温馨提示×

温馨提示×

您好,登录后才能下订单哦!

密码登录×
登录注册×
其他方式登录
点击 登录注册 即表示同意《亿速云用户服务条款》

elasticsearch索引index之put mapping怎么设置

发布时间:2022-04-22 15:06:22 来源:亿速云 阅读:318 作者:iii 栏目:开发技术

本篇内容主要讲解“elasticsearch索引index之put mapping怎么设置”,感兴趣的朋友不妨来看看。本文介绍的方法操作简单快捷,实用性强。下面就让小编来带大家学习“elasticsearch索引index之put mapping怎么设置”吧!

    mapping机制使得elasticsearch索引数据变的更加灵活,近乎于no schema。mapping可以在建立索引时设置,也可以在后期设置。后期设置可以是修改mapping(无法对已有的field属性进行修改,一般来说只是增加新的field)或者对没有mapping的索引设置mapping。put mapping操作必须是master节点来完成,因为它涉及到集群matedata的修改,同时它跟index和type密切相关。修改只是针对特定index的特定type。

    在Action support分析中我们分析过几种Action的抽象类型,put mapping Action属于TransportMasterNodeOperationAction的子类。它实现了masterOperation方法,每个继承自TransportMasterNodeOperationAction的子类都会根据自己的具体功能来实现这个方法。这里的实现如下所示:

    protected void masterOperation(final PutMappingRequest request, final ClusterState state, final ActionListener<PutMappingResponse> listener) throws ElasticsearchException {
            final String[] concreteIndices = clusterService.state().metaData().concreteIndices(request.indicesOptions(), request.indices());
          //构造request
            PutMappingClusterStateUpdateRequest updateRequest = new PutMappingClusterStateUpdateRequest()
                    .ackTimeout(request.timeout()).masterNodeTimeout(request.masterNodeTimeout())
                    .indices(concreteIndices).type(request.type())
                    .source(request.source()).ignoreConflicts(request.ignoreConflicts());
          //调用putMapping方法,同时传入一个Listener
            metaDataMappingService.putMapping(updateRequest, new ActionListener<ClusterStateUpdateResponse>() {
    
                @Override
                public void onResponse(ClusterStateUpdateResponse response) {
                    listener.onResponse(new PutMappingResponse(response.isAcknowledged()));
                }
    
                @Override
                public void onFailure(Throwable t) {
                    logger.debug("failed to put mappings on indices [{}], type [{}]", t, concreteIndices, request.type());
                    listener.onFailure(t);
                }
            });
        }

    以上是TransportPutMappingAction对masterOperation方法的实现,这里并没有多少复杂的逻辑和操作。具体操作在matedataMappingService中。跟之前的CreateIndex一样,put Mapping也是向master提交一个updateTask。所有逻辑也都在execute方法中。这个task的基本跟CreateIndex一样,也需要在给定的时间内响应。它的代码如下所示:

    public void putMapping(final PutMappingClusterStateUpdateRequest request, final ActionListener<ClusterStateUpdateResponse> listener) {
        //提交一个高基本的updateTask
            clusterService.submitStateUpdateTask("put-mapping [" + request.type() + "]", Priority.HIGH, new AckedClusterStateUpdateTask<ClusterStateUpdateResponse>(request, listener) {
    
                @Override
                protected ClusterStateUpdateResponse newResponse(boolean acknowledged) {
                    return new ClusterStateUpdateResponse(acknowledged);
                }
    
                @Override
                public ClusterState execute(final ClusterState currentState) throws Exception {
                    List<String> indicesToClose = Lists.newArrayList();
                    try {
                //必须针对已经在matadata中存在的index,否则抛出异常
                        for (String index : request.indices()) {
                            if (!currentState.metaData().hasIndex(index)) {
                                throw new IndexMissingException(new Index(index));
                            }
                        }
    
                        //还需要存在于indices中,否则无法进行操作。所以这里要进行预建
                        for (String index : request.indices()) {
                            if (indicesService.hasIndex(index)) {
                                continue;
                            }
                            final IndexMetaData indexMetaData = currentState.metaData().index(index);
                  //不存在就进行创建
                            IndexService indexService = indicesService.createIndex(indexMetaData.index(), indexMetaData.settings(), clusterService.localNode().id());
                            indicesToClose.add(indexMetaData.index());
                            // make sure to add custom default mapping if exists
                            if (indexMetaData.mappings().containsKey(MapperService.DEFAULT_MAPPING)) {
                                indexService.mapperService().merge(MapperService.DEFAULT_MAPPING, indexMetaData.mappings().get(MapperService.DEFAULT_MAPPING).source(), false);
                            }
                            // only add the current relevant mapping (if exists)
                            if (indexMetaData.mappings().containsKey(request.type())) {
                                indexService.mapperService().merge(request.type(), indexMetaData.mappings().get(request.type()).source(), false);
                            }
                        }
                //合并更新Mapping
                        Map<String, DocumentMapper> newMappers = newHashMap();
                        Map<String, DocumentMapper> existingMappers = newHashMap();
                //针对每个index进行Mapping合并
                        for (String index : request.indices()) {
                            IndexService indexService = indicesService.indexServiceSafe(index);
                            // try and parse it (no need to add it here) so we can bail early in case of parsing exception
                            DocumentMapper newMapper;
                            DocumentMapper existingMapper = indexService.mapperService().documentMapper(request.type());
                            if (MapperService.DEFAULT_MAPPING.equals(request.type())) {//存在defaultmapping则合并default mapping
                                // _default_ types do not go through merging, but we do test the new settings. Also don't apply the old default
                                newMapper = indexService.mapperService().parse(request.type(), new CompressedString(request.source()), false);
                            } else {
                                newMapper = indexService.mapperService().parse(request.type(), new CompressedString(request.source()), existingMapper == null);
                                if (existingMapper != null) {
                                    // first, simulate
                                    DocumentMapper.MergeResult mergeResult = existingMapper.merge(newMapper, mergeFlags().simulate(true));
                                    // if we have conflicts, and we are not supposed to ignore them, throw an exception
                                    if (!request.ignoreConflicts() && mergeResult.hasConflicts()) {
                                        throw new MergeMappingException(mergeResult.conflicts());
                                    }
                                }
                            }
    
                            newMappers.put(index, newMapper);
                            if (existingMapper != null) {
                                existingMappers.put(index, existingMapper);
                            }
                        }
    
                        String mappingType = request.type();
                        if (mappingType == null) {
                            mappingType = newMappers.values().iterator().next().type();
                        } else if (!mappingType.equals(newMappers.values().iterator().next().type())) {
                            throw new InvalidTypeNameException("Type name provided does not match type name within mapping definition");
                        }
                        if (!MapperService.DEFAULT_MAPPING.equals(mappingType) && !PercolatorService.TYPE_NAME.equals(mappingType) && mappingType.charAt(0) == '_') {
                            throw new InvalidTypeNameException("Document mapping type name can't start with '_'");
                        }
    
                        final Map<String, MappingMetaData> mappings = newHashMap();
                        for (Map.Entry<String, DocumentMapper> entry : newMappers.entrySet()) {
                            String index = entry.getKey();
                            // do the actual merge here on the master, and update the mapping source
                            DocumentMapper newMapper = entry.getValue();
                            IndexService indexService = indicesService.indexService(index);
                            if (indexService == null) {
                                continue;
                            }
    
                            CompressedString existingSource = null;
                            if (existingMappers.containsKey(entry.getKey())) {
                                existingSource = existingMappers.get(entry.getKey()).mappingSource();
                            }
                            DocumentMapper mergedMapper = indexService.mapperService().merge(newMapper.type(), newMapper.mappingSource(), false);
                            CompressedString updatedSource = mergedMapper.mappingSource();
    
                            if (existingSource != null) {
                                if (existingSource.equals(updatedSource)) {
                                    // same source, no changes, ignore it
                                } else {
                                    // use the merged mapping source
                                    mappings.put(index, new MappingMetaData(mergedMapper));
                                    if (logger.isDebugEnabled()) {
                                        logger.debug("[{}] update_mapping [{}] with source [{}]", index, mergedMapper.type(), updatedSource);
                                    } else if (logger.isInfoEnabled()) {
                                        logger.info("[{}] update_mapping [{}]", index, mergedMapper.type());
                                    }
                                }
                            } else {
                                mappings.put(index, new MappingMetaData(mergedMapper));
                                if (logger.isDebugEnabled()) {
                                    logger.debug("[{}] create_mapping [{}] with source [{}]", index, newMapper.type(), updatedSource);
                                } else if (logger.isInfoEnabled()) {
                                    logger.info("[{}] create_mapping [{}]", index, newMapper.type());
                                }
                            }
                        }
    
                        if (mappings.isEmpty()) {
                            // no changes, return
                            return currentState;
                        }
                //根据mapping的更新情况重新生成matadata
                        MetaData.Builder builder = MetaData.builder(currentState.metaData());
                        for (String indexName : request.indices()) {
                            IndexMetaData indexMetaData = currentState.metaData().index(indexName);
                            if (indexMetaData == null) {
                                throw new IndexMissingException(new Index(indexName));
                            }
                            MappingMetaData mappingMd = mappings.get(indexName);
                            if (mappingMd != null) {
                                builder.put(IndexMetaData.builder(indexMetaData).putMapping(mappingMd));
                            }
                        }
    
                        return ClusterState.builder(currentState).metaData(builder).build();
                    } finally {
                        for (String index : indicesToClose) {
                            indicesService.removeIndex(index, "created for mapping processing");
                        }
                    }
                }
            });
        }

    到此,相信大家对“elasticsearch索引index之put mapping怎么设置”有了更深的了解,不妨来实际操作一番吧!这里是亿速云网站,更多相关内容可以进入相关频道进行查询,关注我们,继续学习!

    向AI问一下细节

    免责声明:本站发布的内容(图片、视频和文字)以原创、转载和分享为主,文章观点不代表本网站立场,如果涉及侵权请联系站长邮箱:is@yisu.com进行举报,并提供相关证据,一经查实,将立刻删除涉嫌侵权内容。

    AI