首页 > 其他 > 详细

【ES】查询

时间:2020-07-13 18:26:35      阅读:57      评论:0      收藏:0      [点我收藏+]

查询接口

本文为翻译官方文档

一个基本的查询构造

1、创建SearchRequest,不带参数,表示查询所有索引
2、添加大部分查询参数到 SearchSourceBuilder,接收QueryBuilders构建的查询参数
3、添加 match_all 查询到 SearchSourceBuilder
4、添加 SearchSourceBuilder 到 SearchRequest

SearchRequest searchRequest = new SearchRequest(); 
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder(); 
searchSourceBuilder.query(QueryBuilders.matchAllQuery()); 
searchRequest.source(searchSourceBuilder); 

SearchRequest 有一些可选参数

// 指定查询“posts”索引
SearchRequest searchRequest = new SearchRequest("posts"); 
// 设置路由
searchRequest.routing("routing"); 
// IndicesOptions 设置如何解析未知的索引及通配符表达式如何扩展
searchRequest.indicesOptions(IndicesOptions.lenientExpandOpen()); 
// 设置偏好参数,如设置搜索本地分片的偏好,默认是在分片中随机检索
searchRequest.preference("_local"); 

使用SearchSourceBuilder

控制搜索行为的大多数选项都可以在 SearchSourceBuilder 上设置,该构建器或多或少包含与Rest API 的search request中的选项等价的设置。下面是一些通用设置选项:

// 使用默认参数创建 SearchSourceBuilder
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder(); 
// 可以设置任何类型的QueryBuilder查询参数
sourceBuilder.query(QueryBuilders.termQuery("user", "kimchy")); 
// 设置查询的起始位置,默认是0
sourceBuilder.from(0); 
// 设置查询结果的页大小,默认是10
sourceBuilder.size(5); 
// 设置当前查询的超时时间
sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));

设置完上面 的SearchSourceBuilder ,只需要将其设置到 SearchRequest中即可。

SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("posts");
searchRequest.source(sourceBuilder);

构建查询参数QueryBuilder

使用QueryBuilder构建查询参数,QueryBuilder包括所有Elasticsearch’s Query DSL支持的类型。

可以使用QueryBuilde构造器创建一个QueryBuilder:

// 构建一个全文检索Match Query, 查询匹配kimchy的user字段
MatchQueryBuilder matchQueryBuilder = new MatchQueryBuilder("user", "kimchy");

// 可以针对创建的QueryBuilder对象设置查询参数
// 开启模糊查询
matchQueryBuilder.fuzziness(Fuzziness.AUTO); 
// 设置查询前缀长度
matchQueryBuilder.prefixLength(3); 
// 设置模糊查询最大扩展
matchQueryBuilder.maxExpansions(10); 

可以使用工具类QueryBuilders,采用流式编程的形式构建QueryBuilder

QueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("user", "kimchy")
                                                .fuzziness(Fuzziness.AUTO)
                                                .prefixLength(3)
                                                .maxExpansions(10);

不管使用上述两种方式的哪种来创建QueryBuilder, QueryBuilder都必须按如下方式设置到SearchSourceBuilder.

searchSourceBuilder.query(matchQueryBuilder);

Building Queries 页给出了所有的查询QueryBuilder及辅助类QueryBuilders的表达方式。

指定排序

SearchSourceBuilder允许增加一或多个排序参数SortBuilder,有四个具体实现FieldSortBuilder, ScoreSortBuilder, GeoDistanceSortBuilder 和 ScriptSortBuilder。

// 默认排序。根据_score倒序
sourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC)); 
// 根据_id升序
sourceBuilder.sort(new FieldSortBuilder("id").order(SortOrder.ASC)); 

使用Source字段过滤

_source字段

默认情况下,查询请求会返回_source字段的全部内容,但是该行为可以被覆写,比如,你可以完全关掉该字段的索引(不推荐,该行为,原因参考上面的链接)

sourceBuilder.fetchSource(false);

该方法fetchSource也可以接收组通配模式来以更细粒度地方式控制哪些字段被包含或者被排除。

String[] includeFields = new String[] {"title", "innerObject.*"};
String[] excludeFields = new String[] {"user"};
sourceBuilder.fetchSource(includeFields, excludeFields);

搜索结果突出(Highlighting)

通过向 SearchSourceBuilder 设置 HighlightBuilder 可以实现查询高亮的效果。通过设置一个或多个 HighlightBuilder.Field 对象到HighlightBuilder 中实现不同的突出行为。

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
HighlightBuilder highlightBuilder = new HighlightBuilder(); 
// 设置需要突出的字段
HighlightBuilder.Field highlightTitle = new HighlightBuilder.Field("title"); 
highlightTitle.highlighterType("unified");  
highlightBuilder.field(highlightTitle);  
HighlightBuilder.Field highlightUser = new HighlightBuilder.Field("user");
highlightBuilder.field(highlightUser);
// 设置HighlightBuilder到SearchSourceBuilder
searchSourceBuilder.highlighter(highlightBuilder);

详细的选项可以参考Rest API的文档。

高亮的文本片断可以在下面的 later be retrieved SearchResponse中获取。

实际样例

    // 获取数据实现高亮功能
    public List<Map<String, Object>> searchPageHighlightBuilder(String keyword, int pageNo, int pageSize)
        throws IOException {
        if (pageNo <= 1) {
            pageNo = 1;
        }

        keyword = URLDecoder.decode(keyword, "UTF-8");

        // 条件搜索
        SearchRequest searchRequest = new SearchRequest("jd_goods");
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();

        // 分页
        searchSourceBuilder.from(pageNo);
        searchSourceBuilder.size(pageSize);

        // 精准匹配
        TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("title", keyword);
        searchSourceBuilder.query(termQueryBuilder);
        searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));

        // 高亮
        HighlightBuilder highlightBuilder = new HighlightBuilder();
        highlightBuilder.field("title");
        highlightBuilder.requireFieldMatch(true);// 多个高亮显示
        highlightBuilder.preTags("<span style=‘color:red‘>");
        highlightBuilder.postTags("</span>");
        searchSourceBuilder.highlighter(highlightBuilder);

        // 执行搜索
        searchRequest.source(searchSourceBuilder);
        SearchResponse search = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);

        // 解析结果
        ArrayList<Map<String, Object>> list = new ArrayList<>();
        for (SearchHit documentFields : search.getHits().getHits()) {

            // 解析高亮的字段
            Map<String, HighlightField> highlightFields = documentFields.getHighlightFields();
            HighlightField title = highlightFields.get("title");
            Map<String, Object> sourceAsMap = documentFields.getSourceAsMap();
            if (title != null) {
                Text[] fragments = title.fragments();
                String n_title = "";
                for (Text text : fragments) {
                    n_title += text;
                }
                sourceAsMap.put("title", n_title);
            }
            list.add(sourceAsMap);
        }
        return list;

    }

请求聚合(Requesting Aggregations)

通过构建AggregationBuilder对象并设置到SearchSourceBuilder中可以实现聚合查询。

下面的例子创建了terms的聚合:聚合各公司下员工的平均年龄

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
TermsAggregationBuilder aggregation = AggregationBuilders.terms("by_company").field("company.keyword");
aggregation.subAggregation(AggregationBuilders.avg("average_age").field("age"));
searchSourceBuilder.aggregation(aggregation);

Building Aggregations页给出了聚合对象AggregationBuilder及辅助类AggregationBuilders的对应关系及使用方式。

后面会介绍如何从SearchResponse中获取聚合结果。 access aggregations

请求建议Requesting Suggestions

在查询请求中可以设置请求Suggestions,通过使用SuggestBuilders辅助类,或者SuggestionBuilder构造器,将其设置到SuggestBuilder,最后将SuggestBuilder设置SearchSourceBuilder中。

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
// 为字段user和文本kmichy创建 TermSuggestionBuilder 
SuggestionBuilder termSuggestionBuilder = SuggestBuilders.termSuggestion("user").text("kmichy"); 
SuggestBuilder suggestBuilder = new SuggestBuilder();
// 添加TermSuggestionBuilder到suggestBuilder中,并命名为suggest_user
suggestBuilder.addSuggestion("suggest_user", termSuggestionBuilder); 
searchSourceBuilder.suggest(suggestBuilder);

后面会介绍如何从SearchResponse中检索建议retrieve suggestions

Profiling Queries和aggregations

Profile API可以配置某个具体的查询或聚合请求的执行过程。如果想使用该功能,需要将SearchSourceBuilder的开关打开。

SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.profile(true);

SearchRequest执行后,SearchResponse会包含配置的执行信息。contain the profiling results

同步查询执行Synchronous execution

下面是同步查询执行方式,客户端会等待SearchResponse的结果返回后,才继续执行后面的代码。

SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);

high-level REST client 的同步请求可能抛出IOException。

异步查询执行Asynchronous execution

SearchRequest也可以以异步方式执行。用户需要指定listenert到异步请求中,告诉异步请求如何处理响应或潜在的错误。

client.searchAsync(searchRequest, RequestOptions.DEFAULT, listener); 

searchRequest执行完成后会调用ActionListener

异步方式不会阻塞,当该异步调用结果后,ActionListener会被调用,如果执行成功,则onResponse会被调用,如果失败,则onFailure被调用。一个典型的search listener如下:

ActionListener<SearchResponse> listener = new ActionListener<SearchResponse>() {
    @Override
    public void onResponse(SearchResponse searchResponse) {
        
    }

    @Override
    public void onFailure(Exception e) {
        
    }
};

SearchResponse

SearchResponse提供了查询执行的细节以及返回的文档结果。首先,SearchResponse包括当前查询的执行细节,比如HTTP响应码、执行时间、或者是否超时等:

RestStatus status = searchResponse.status();
TimeValue took = searchResponse.getTook();
Boolean terminatedEarly = searchResponse.isTerminatedEarly();
boolean timedOut = searchResponse.isTimedOut();

其次,SearchResponse提供了查询涉及的分片统计信息,包含执行成功和执行失败的分片。执行的错误信息ShardSearchFailures可以通过如下方式迭代处理:

int totalShards = searchResponse.getTotalShards();
int successfulShards = searchResponse.getSuccessfulShards();
int failedShards = searchResponse.getFailedShards();
for (ShardSearchFailure failure : searchResponse.getShardFailures()) {
    // failures should be handled here
}

检索结果Retrieving SearchHits

为了获取到结果中的文档信息,我们首先要获取response中的搜索结果集SearchHits:

SearchHits hits = searchResponse.getHits();

SearchHits提供了所有命中结果的全局信息,包括命中的总数或者最大的score:

TotalHits totalHits = hits.getTotalHits();
// the total number of hits, must be interpreted in the context of totalHits.relation
long numHits = totalHits.value;
// whether the number of hits is accurate (EQUAL_TO) or a lower bound of the total (GREATER_THAN_OR_EQUAL_TO)
TotalHits.Relation relation = totalHits.relation;
float maxScore = hits.getMaxScore();

SearchHits中的单个结果集可以迭代获取:

SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
    // do something with the SearchHit
}

单个结果集SearchHit包含了一些基本信息:索引、文档ID、每个命中结果的score

String index = hit.getIndex();
String id = hit.getId();
float score = hit.getScore();

此外,SearchHit可以以JSON或MAP形式返回文档的source信息。在Map中,普通的字段以字段名作为key,值为字段值。多值字段是以对象列表形式返回,嵌套对象,则以另一个map的形式返回。需要根据实际情况进行强转:

String sourceAsString = hit.getSourceAsString();
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String documentTitle = (String) sourceAsMap.get("title");
List<Object> users = (List<Object>) sourceAsMap.get("user");
Map<String, Object> innerObject = (Map<String, Object>) sourceAsMap.get("innerObject");

获取突出内容(Retrieving Highlighting)

如果设置了highlighting的请求,则返回的SearchHit中可以获取到突出的text文本片断。

SearchHits hits = searchResponse.getHits();
for (SearchHit hit : hits.getHits()) {
    Map<String, HighlightField> highlightFields = hit.getHighlightFields();
    // Get the highlighting for the title field
    HighlightField highlight = highlightFields.get("title"); 
    // Get one or many fragments containing the highlighted field content
    Text[] fragments = highlight.fragments();  
    String fragmentString = fragments[0].string();
}

获取聚合结果(Retrieving Aggregations)

首先从SearchResponse中获取聚合树aggregation tree的根,聚合对象Aggregations,然后可以根据名称获取聚合对象aggregation。

Aggregations aggregations = searchResponse.getAggregations();
// Get the by_company terms aggregation
Terms byCompanyAggregation = aggregations.get("by_company"); 
// Get the buckets that is keyed with Elastic
Bucket elasticBucket = byCompanyAggregation.getBucketByKey("Elastic"); 
// Get the average_age sub-aggregation from that bucket
Avg averageAge = elasticBucket.getAggregations().get("average_age"); 
double avg = averageAge.getValue();

注意,如果使用名称获取Aggregations,需要指定aggregation 接口为请求里设置的aggregation 类型,否则会抛出ClassCastException 异常。

// This will throw an exception because "by_company" is a terms aggregation but we try to retrieve it as a range aggregation
Range range = aggregations.get("by_company"); 

也可以以map的形式获取aggregations,key是aggregation名称。这种情况下,aggregation 接口需要显式的强转。

Map<String, Aggregation> aggregationMap = aggregations.getAsMap();
Terms companyAggregation = (Terms) aggregationMap.get("by_company");

也有方式将所有top level aggregations以列表形式返回:

List<Aggregation> aggregationList = aggregations.asList();

最后,可以迭代所有aggregations:

for (Aggregation agg : aggregations) {
    String type = agg.getType();
    if (type.equals(TermsAggregationBuilder.NAME)) {
        Bucket elasticBucket = ((Terms) agg).getBucketByKey("Elastic");
        long numberOfDocs = elasticBucket.getDocCount();
    }
}

获取建议结果(Retrieving Suggestions)

为了从SearchResponse中获取suggestions,可以使用Suggest对象作为入口。

// Use the Suggest class to access suggestions
Suggest suggest = searchResponse.getSuggest(); 
// Suggestions can be retrieved by name. You need to assign them to the correct type of Suggestion class (here TermSuggestion), otherwise a ClassCastException is thrown
TermSuggestion termSuggestion = suggest.getSuggestion("suggest_user"); 
// Iterate over the suggestion entries
for (TermSuggestion.Entry entry : termSuggestion.getEntries()) { 
    // Iterate over the options in one entry
    for (TermSuggestion.Entry.Option option : entry) { 
        String suggestText = option.getText().string();
    }
}

获取配置结果(Retrieving Profiling Results)

可以使用SearchResponse的getProfileResults()方法获取。返回结果为每个分片包装一个Map,值为ProfileShardResult对象。key是能唯一标识分片的信息。

// Retrieve the Map of ProfileShardResult from the SearchResponse
Map<String, ProfileShardResult> profilingResults = searchResponse.getProfileResults(); 
// Profiling results can be retrieved by shard’s key if the key is known, otherwise it might be simpler to iterate over all the profiling results
for (Map.Entry<String, ProfileShardResult> profilingResult : profilingResults.entrySet()) { 
    // Retrieve the key that identifies which shard the ProfileShardResult belongs to
    String key = profilingResult.getKey(); 
    // Retrieve the ProfileShardResult for the given shard
    ProfileShardResult profileShardResult = profilingResult.getValue(); 
}

ProfileShardResult包含一个或多个profile 结果:

// Retrieve the list of QueryProfileShardResult
List<QueryProfileShardResult> queryProfileShardResults =
        profileShardResult.getQueryProfileResults(); 
// Iterate over each QueryProfileShardResult
for (QueryProfileShardResult queryProfileResult : queryProfileShardResults) { 

}

每个QueryProfileShardResult 中可以获取ProfileResult对象列表:

// Iterate over the profile results
for (ProfileResult profileResult : queryProfileResult.getQueryResults()) {
    // Retrieve the name of the Lucene query
    String queryName = profileResult.getQueryName(); 
    // Retrieve the time in millis spent executing the Lucene query
    long queryTimeInMillis = profileResult.getTime(); 
    // Retrieve the profile results for the sub-queries (if any)
    List<ProfileResult> profiledChildren = profileResult.getProfiledChildren(); 
}

QueryProfileShardResult也可以获取Lucene collectors的信息:

// Retrieve the profiling result of the Lucene collector
CollectorResult collectorResult = queryProfileResult.getCollectorResult();  
// Retrieve the name of the Lucene collector
String collectorName = collectorResult.getName();  
// Retrieve the time in millis spent executing the Lucene collector
Long collectorTimeInMillis = collectorResult.getTime(); 
// Retrieve the profile results for the sub-collectors (if any)
List<CollectorResult> profiledChildren = collectorResult.getProfiledChildren(); 

QueryProfileShardResult可以获取详细的aggregations tree执行信息:

// Retrieve the AggregationProfileShardResult
AggregationProfileShardResult aggsProfileResults =
        profileShardResult.getAggregationProfileResults(); 
// Iterate over the aggregation profile results
for (ProfileResult profileResult : aggsProfileResults.getProfileResults()) { 
    // Retrieve the type of the aggregation (corresponds to Java class used to execute the aggregation)
    String aggName = profileResult.getQueryName(); 
    // Retrieve the time in millis spent executing the Lucene collector
    long aggTimeInMillis = profileResult.getTime(); 
    // Retrieve the profile results for the sub-aggregations (if any)
    List<ProfileResult> profiledChildren = profileResult.getProfiledChildren(); 
}

【ES】查询

原文:https://www.cnblogs.com/z00377750/p/13294561.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!