网站导航
> 文章列表 > elasticsearch(es)高级查询api

elasticsearch(es)高级查询api

网友: 文章列表 2024-03-21 21:32:28

elasticsearch(es)高级查询api

yml配置

  #es配置
spring:elasticsearch:rest:uris: 192.168.16.188:9200

添加依赖

<dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId>
</dependency>

使用编程的形式设置连接的ES服务器，并获取客户端对象，配置ES服务器地址与端口9200，记得客户端使用完毕需要手工关闭。由于当前客户端是手工维护的，因此不能通过自动装配的形式加载对象

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SpringBootTest.class)
public class JeecgTest {@Test
//    @SneakyThrowspublic void name() {HttpHost host = HttpHost.create("http://localhost:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);try {client.close();} catch (IOException e) {throw new RuntimeException(e);}}
}

使用客户端对象操作ES，例如创建索引

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SpringBootTest.class)
public class JeecgTest {private RestHighLevelClient client;@Testvoid testCreateIndex() throws IOException {HttpHost host = HttpHost.create("http://localhost:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);CreateIndexRequest request = new CreateIndexRequest("books");client.indices().create(request, RequestOptions.DEFAULT); client.close();}
}

添加文档，添加文档使用的请求对象是IndexRequest，与创建索引使用的请求对象不同

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SpringBootTest.class)
public class JeecgTest {@Testpublic void testCreateIndex() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);HashMap<String, Object> map = new HashMap<>();map.put("name", "张三");map.put("age", 20);IndexRequest request = new IndexRequest("user");String json = JSON.toJSONString(map);request.source(json, XContentType.JSON);IndexResponse index = client.index(request, RequestOptions.DEFAULT);client.close();}
}

/增量更新文档

  @Testpublic void testUpdateDoc() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);UpdateRequest updateRequest = new UpdateRequest("user", "88");updateRequest.timeout("1s");User user = new User();user.setAge(222);updateRequest.doc(JSON.toJSONString(user), XContentType.JSON);UpdateResponse update = client.update(updateRequest, RequestOptions.DEFAULT);client.close();}

批量添加文档：批量做时，先创建一个BulkRequest的对象，可以将该对象理解为是一个保存request对象的容器，将所有的请求都初始化好后，添加到BulkRequest对象中，再使用BulkRequest对象的bulk方法，一次性执行完毕

    @Test//批量添加文档public void testCreateDocAll() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);List<Map> list = new ArrayList<>();HashMap<String, Object> map = new HashMap<>();for (int i = 0; i < 3; i++) {map.put("aa" + i, i);list.add(map);}BulkRequest bulk = new BulkRequest();for (Map map1 : list) {IndexRequest request = new IndexRequest("user");String json = JSON.toJSONString(map1);request.source(json, XContentType.JSON);bulk.add(request);}client.bulk(bulk, RequestOptions.DEFAULT);client.close();}

按id查询1文档：根据id查询文档使用的请求对象是GetRequest

    @Test//按id查询public void getById() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);GetRequest request = new GetRequest("user", "88");GetResponse response = client.get(request, RequestOptions.DEFAULT);String json = response.getSourceAsString();System.out.println(json);client.close();}

通过id查询2

    @Test//按id查询public void getById2() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.termQuery("_id",88));request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

按条件查询文档：按条件查询文档使用的请求对象是SearchRequest，查询时调用SearchRequest对象的termQuery方法，需要给出查询属性名，此处支持使用合并字段，也就是前面定义索引属性时添加的all属性

    @Test//按条件查询public void getBySearch() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.matchPhraseQuery("name","张三"));request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

按条件高亮查询

    @Test//按条件高亮查询public void getBySearch2() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.matchPhraseQuery("name","张三"));HighlightBuilder highlightBuilder = new HighlightBuilder();searchSourceBuilder.highlighter(highlightBuilder.field("name"));request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {Map<String, HighlightField> highlightFields = hit.getHighlightFields();System.out.println(highlightFields);}client.close();}

全文高亮查询

    @Test//全文高亮查询public void getBySearch2() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.multiMatchQuery("张三"));HighlightBuilder highlightBuilder = new HighlightBuilder();searchSourceBuilder.highlighter(highlightBuilder.field("*"));request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {Map<String, HighlightField> highlightFields = hit.getHighlightFields();System.out.println(highlightFields);}client.close();}

分页查询

    @Test//分页查询public void getBySearch3() throws IOException {// 开始查询的记录数//页码Integer pageNum=1;//页数Integer pageSize=2;int start = (pageNum - 1) * pageSize;HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.matchAllQuery());searchSourceBuilder.from(start);searchSourceBuilder.size(pageSize);request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}client.close();}

MultiQuery 全部字段联合搜索

    @Test// MultiQuery 全部字段联合搜索 public void getBySearch4() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.multiMatchQuery("三"));request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

MultiQuery 多字段联合搜索 ，使用多字段查询的时候，查询的字段要和查询的内容类型一直，不然就会报错，类似age字段是int类型，和name字段是string类型查询就会报错

    @Test// MultiQuery 多字段联合搜索public void getBySearch5() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.multiMatchQuery("王","name","file"));request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

多域联合查询的时候，可以通过 boost 来设置某个域在计算得分时候的比重，比重越高的域当他符合条件时计算的得分越高，相应的该记录也更靠前。通过在 fields 中给相应的字段用 ^权重倍数来实现

	@Test// MultiQuery 多字段联合搜索和设置权重public void getBySearch6() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(QueryBuilders.multiMatchQuery("广","name","address").field("name",10));request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

如果我们既要对一些字段进行分词查询，同时要对另一些字段进行精确查询，就需要使用布尔查询来实现了。布尔查询对应于Lucene的BooleanQuery查询，实现将多个查询组合起来，有三个可选的参数：must：文档必须匹配must所包括的查询条件，相当于 “AND”should：文档应该匹配should所包括的查询条件其中的一个或多个，相当于 "OR"must_not：文档不能匹配must_not所包括的该查询条件，相当于“NOT”

```erlang
GET user/_search
{"query": {"bool": { // 布尔查询"must": [ // 查询条件 must 表示数组中的查询方式所规定的条件都必须满足{"multi_match": {"query": "王小妹","minimum_should_match": "50%","fields": ["name^10","title"]}},{"match": {"address": "广州"}}]}}
}

    @Test// 布尔查询 BoolQuerypublic void getBySearch7() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 搜索方式// 首先构造多关键字查询条件MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("王小妹", "name", "title").field("name", 10);// 然后构造匹配查询条件MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("address", "广州");// 组合两个条件，组合方式为 must 全满足BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.must(multiMatchQueryBuilder);boolQueryBuilder.must(matchQueryBuilder);// 将查询条件封装给查询对象searchSourceBuilder.query(boolQueryBuilder);request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

定义过滤器查询，是在原本查询结果的基础上对数据进行筛选，因此省略了重新计算的分的步骤，效率更高。并且方便缓存。推荐尽量使用过虑器去实现查询或者过虑器和查询共同使用，过滤器在布尔查询中使用，下边是在搜索结果的基础上进行过滤：

GET user/_search
{"query": {"bool": {"must": [{"multi_match": {"query": "王小妹","minimum_should_match": "50%","fields": ["name^10","title"]}}],"filter": [{// 过滤条件：studymodel 必须是 201001"match": {"address": "广州"}},{// 过滤条件：年龄 >=10 <=100"range": {"age": {"gte": 10,"lte": 100}}}]}}
}

    @Test// 过滤器public void getBySearch8() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 搜索方式// 首先构造多关键字查询条件MultiMatchQueryBuilder multiMatchQueryBuilder = QueryBuilders.multiMatchQuery("王小妹", "name", "title").field("name", 10);// 构造匹配查询条件MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("address", "广州");// 构造范围查询条件RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gt(10).lt(100);BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.must(multiMatchQueryBuilder);boolQueryBuilder.filter(matchQueryBuilder);boolQueryBuilder.filter(rangeQueryBuilder);// 将查询条件封装给查询对象searchSourceBuilder.query(boolQueryBuilder);request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

注意：range和term一次只能对一个Field设置范围过虑

排序，在查询的结果上进行二次排序，支持对 keyword、date、float 等类型添加排序，text类型的字段不允许排序

GET user/_search
{"query": {"bool": {"filter": [{"range": {"age": {"gte": 10,"lte": 100}}}]}},"sort": [{"age": "desc" }]
}
    @Test// 排序public void getBySearch9() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest request = new SearchRequest("user");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();// 构造范围查询条件RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gt(10).lt(100);BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.filter(rangeQueryBuilder);searchSourceBuilder.sort("age", SortOrder.DESC);// 将查询条件封装给查询对象searchSourceBuilder.query(boolQueryBuilder);request.source(searchSourceBuilder);SearchResponse response = client.search(request, RequestOptions.DEFAULT);SearchHits hits = response.getHits();for (SearchHit hit : hits) {String source = hit.getSourceAsString();System.out.println(source);}client.close();}

根据查询条件来删除

@Test// 根据条件物理删除public void deleteBySearch() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);DeleteByQueryRequest request = new DeleteByQueryRequest("efshdx"); // 替换成您的索引名称MatchPhraseQueryBuilder matchPhraseQueryBuilder = QueryBuilders.matchPhraseQuery("catagoryId", "1640927455451201537");request.setQuery(matchPhraseQueryBuilder);BulkByScrollResponse response = client.deleteByQuery(request, RequestOptions.DEFAULT); // 替换成您的Elasticsearch客户端实例long deleted = response.getStatus().getDeleted();System.out.println("删除掉的es数:"+deleted);client.close();}

更新，或者新建属性

    @Test// 使用es的Update Mapping API来更新日期映射,或者新建属性public void updateDateMapping() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);PutMappingRequest request = new PutMappingRequest("person1"); // 替换成您的索引名称XContentBuilder mappingBuilder = XContentFactory.jsonBuilder();mappingBuilder.startObject();{mappingBuilder.startObject("properties");{mappingBuilder.startObject("time1");{mappingBuilder.field("type", "date");mappingBuilder.field("format", "yyyy/MM/dd"); // 更新日期格式}mappingBuilder.endObject();mappingBuilder.startObject("time2");{mappingBuilder.field("type", "date");mappingBuilder.field("format", "yyyy/MM/dd"); // 更新日期格式}mappingBuilder.endObject();}mappingBuilder.endObject();}mappingBuilder.endObject();request.source(mappingBuilder);AcknowledgedResponse response = client.indices().putMapping(request, RequestOptions.DEFAULT); // 替换成您的Elasticsearch客户端实例client.close();}

创建索引并添加属性

@Test// 创建索引并添加属性public void addIndexAndMapping() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);CreateIndexRequest request = new CreateIndexRequest("person1"); // 替换成您的索引名称XContentBuilder mappingBuilder = XContentFactory.jsonBuilder();mappingBuilder.startObject();{mappingBuilder.startObject("properties");{// 添加字符串类型字段mappingBuilder.startObject("string-field");{mappingBuilder.field("type", "text");}mappingBuilder.endObject();// 添加整数类型字段mappingBuilder.startObject("integer-field");{mappingBuilder.field("type", "integer");}mappingBuilder.endObject();// 添加日期类型字段mappingBuilder.startObject("date-field");{mappingBuilder.field("type", "date");mappingBuilder.field("format", "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis");}mappingBuilder.endObject();// 添加嵌套类型字段mappingBuilder.startObject("nested-field");{mappingBuilder.field("type", "nested");mappingBuilder.startObject("properties");{mappingBuilder.startObject("nested-string-field");{mappingBuilder.field("type", "text");}mappingBuilder.endObject();}mappingBuilder.endObject();}mappingBuilder.endObject();}mappingBuilder.endObject();}mappingBuilder.endObject();request.mapping(mappingBuilder);CreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT); // 替换成您的Elasticsearch客户端实例client.close();}

单属性多条件查询，类似in查询

@Test// 单属性多条件查询，类似in查询public void queryByMultiFieldValue() throws IOException {HttpHost host = HttpHost.create("http://192.168.16.188:9200");RestClientBuilder builder = RestClient.builder(host);client = new RestHighLevelClient(builder);SearchRequest searchRequest = new SearchRequest("person1"); // 替换成您的索引名称SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();List<String> list = new ArrayList<>();list.add("我40");list.add("我50");TermsQueryBuilder termsQueryBuilder = QueryBuilders.termsQuery("newName", list);searchSourceBuilder.query(termsQueryBuilder);searchRequest.source(searchSourceBuilder);SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT); // 替换成您的Elasticsearch客户端实例client.close();}

在SpringMVC中，可以使用ResponseEntity对象将异步操作的结果返回给前端。以下是示例代码：

@RestController
@RequestMapping("/es")
public class EsController {private final RestHighLevelClient client;@Autowiredpublic EsController(RestHighLevelClient client) {this.client = client;}@PostMapping("/search")public ResponseEntity<Map<String, Object>> search(@RequestBody SearchRequest request) {Map<String, Object> result = new HashMap<>();CountDownLatch countDownLatch = new CountDownLatch(1);ActionListener<SearchResponse> listener = new ActionListener<SearchResponse>() {@Overridepublic void onResponse(SearchResponse searchResponse) {// 在这里处理异步响应结果SearchHits hits = searchResponse.getHits();Long total = hits.getTotalHits().value;List<Map<String, Object>> record = new ArrayList<>();for (SearchHit hit : hits) {Map<String, Object> sourceAsString = hit.getSourceAsMap();if (sourceAsString != null) {sourceAsString.put("id", sourceAsString.get("metaId"));}record.add(sourceAsString);}result.put("record", record);result.put("total", total);countDownLatch.countDown();}@Overridepublic void onFailure(Exception e) {countDownLatch.countDown();// 处理异步响应失败情况result.put("error", e.getMessage());}};client.searchAsync(request, RequestOptions.DEFAULT, listener);countDownLatch.await();// 返回异步操作的结果return ResponseEntity.ok(result);}
}

在以上示例代码中，定义了一个返回类型为ResponseEntity<Map<String, Object>>的/search POST映射方法，并使用Map<String, Object>对象来存储异步操作的结果。然后，创建了一个ActionListener的匿名实现对象，并使用client.searchAsync()方法以异步方式执行搜索操作。在onResponse()方法中，将搜索结果存储到result Map中。如果异步响应失败，则使用onFailure()方法处理异常情况，并将错误消息存储到result Map中。
最后，在异步操作完成之前，使用ResponseEntity.ok(result)返回一个HTTP 200 OK响应对象，并将result Map作为响应体返回给前端。

在Java中操作Elasticsearch进行异步搜索，如果需要等待搜索结果返回再继续执行，可以使用CompletableFuture来实现。具体步骤如下：

@RestController
@RequestMapping("/es")
public class EsController {private final RestHighLevelClient client;@Autowiredpublic EsController(RestHighLevelClient client) {this.client = client;}@PostMapping("/search")public ResponseEntity<Map<String, Object>> search(@RequestBody SearchRequest request) {Map<String, Object> result = new HashMap<>();CompletableFuture<SearchResponse> future = new CompletableFuture<>();ActionListener<SearchResponse> listener = new ActionListener<SearchResponse>() {@Overridepublic void onResponse(SearchResponse searchResponse) {// 在这里处理异步响应结果SearchHits hits = searchResponse.getHits();Long total = hits.getTotalHits().value;List<Map<String, Object>> record = new ArrayList<>();for (SearchHit hit : hits) {Map<String, Object> sourceAsString = hit.getSourceAsMap();if (sourceAsString != null) {sourceAsString.put("id", sourceAsString.get("metaId"));}record.add(sourceAsString);}result.put("record", record);result.put("total", total);future.complete(searchResponse);}@Overridepublic void onFailure(Exception e) {future.completeExceptionally(e);// 处理异步响应失败情况result.put("error", e.getMessage());}};client.searchAsync(request, RequestOptions.DEFAULT, listener);SearchResponse searchResponse = future.get();// 返回异步操作的结果return ResponseEntity.ok(result);}
}
网络标签：

相关问题