SpringCloud:ElasticSearch之自动补全
当用户在搜索框输入字符时,我们应该提示出与该字符有关的搜索项,如图:
这种根据用户输入的字母,提示完整词条的功能,就是自动补全了。
1.拼音分词器
要实现根据字母做补全,就必须对文档按照拼音分词。在GitHub
上恰好有elasticsearch
的拼音分词插件。地址:https://github.com/medcl/elasticsearch-analysis-pinyin
安装方式与iK
分词器一样,分三步:
①下载解压
②上传到虚拟机中,elasticsearch
的plugin
目录
③重启elasticsearch
④测试
详细安装步骤可以参考IK
分词器的安装过程。
测试用法如下:
POST /_analyze
{"text": "我爱北京天安门","analyzer": "pinyin"
}
结果:
2.自定义分词器
默认的拼音分词器会将每个汉字单独分为拼音,而我们希望的是每个词条形成一组拼音,需要对拼音分词器做个性化定制,形成自定义分词器。
elasticsearch
中分词器(analyzer
)的组成包含三部分:
character filters
:在tokenizer
之前对文本进行处理。例如删除字符、替换字符tokenizer
:将文本按照一定的规则切割成词条(term
)。例如keyword
,就是不分词;还有ik_smart
tokenizer filter
:将tokenizer
输出的词条做进一步处理。例如大小写转换、同义词处理、拼音处理等
文档分词时会依次由这三部分来处理文档:
声明自定义分词器的语法如下:
PUT /test
{"settings": {"analysis": {"analyzer": { // 自定义分词器"my_analyzer": { // 分词器名称"tokenizer": "ik_max_word","filter": "py"}},"filter": { // 自定义tokenizer filter"py": { // 过滤器名称"type": "pinyin", // 过滤器类型,这里是pinyin"keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"name": {"type": "text","analyzer": "my_analyzer","search_analyzer": "ik_smart"}}}
}
测试:
总结:
如何使用拼音分词器?
-
①下载
pinyin
分词器 -
②解压并放到
elasticsearch
的plugin
目录 -
③重启即可
如何自定义分词器?
-
①创建索引库时,在
settings
中配置,可以包含三部分 -
②
character filter
-
③
tokenizer
-
④
filter
拼音分词器注意事项?
- 为了避免搜索到同音字,搜索时不要使用拼音分词器
3.自动补全查询
elasticsearch
提供了Completion Suggester查询来实现自动补全功能。这个查询会匹配以用户输入内容开头的词条并返回。为了提高补全查询的效率,对于文档中字段的类型有一些约束:
-
参与补全查询的字段必须是
completion
类型。 -
字段的内容一般是用来补全的多个词条形成的数组。
比如,一个这样的索引库:
// 创建索引库
PUT test
{"mappings": {"properties": {"title":{"type": "completion"}}}
}
然后插入下面的数据:
// 示例数据
POST test/_doc
{"title": ["Sony", "WH-1000XM3"]
}
POST test/_doc
{"title": ["SK-II", "PITERA"]
}
POST test/_doc
{"title": ["Nintendo", "switch"]
}
查询的DSL
语句如下:
// 自动补全查询
GET /test/_search
{"suggest": {"title_suggest": {"text": "s", // 关键字"completion": {"field": "title", // 补全查询的字段"skip_duplicates": true, // 跳过重复的"size": 10 // 获取前10条结果}}}
}
4.实现酒店搜索框自动补全
现在,我们的hotel
索引库还没有设置拼音分词器,需要修改索引库中的配置。但是我们知道索引库是无法修改的,只能删除然后重新创建。
另外,我们需要添加一个字段,用来做自动补全,将brand
、suggestion
、city
等都放进去,作为自动补全的提示。
因此,总结一下,我们需要做的事情包括:
-
修改
hotel
索引库结构,设置自定义拼音分词器 -
修改索引库的
name
、all
字段,使用自定义分词器 -
索引库添加一个新字段
suggestion
,类型为completion
类型,使用自定义的分词器 -
给
HotelDoc
类添加suggestion
字段,内容包含brand
、business
-
重新导入数据到
hotel
库
4.1.修改酒店映射结构
代码如下:
// 酒店数据索引库
PUT /hotel
{"settings": {"analysis": {"analyzer": {"text_anlyzer": {"tokenizer": "ik_max_word","filter": "py"},"completion_analyzer": {"tokenizer": "keyword","filter": "py"}},"filter": {"py": {"type": "pinyin","keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"id":{"type": "keyword"},"name":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart","copy_to": "all"},"address":{"type": "keyword","index": false},"price":{"type": "integer"},"score":{"type": "integer"},"brand":{"type": "keyword","copy_to": "all"},"city":{"type": "keyword"},"starName":{"type": "keyword"},"business":{"type": "keyword","copy_to": "all"},"location":{"type": "geo_point"},"pic":{"type": "keyword","index": false},"all":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart"},"suggestion":{"type": "completion","analyzer": "completion_analyzer"}}}
}
4.2.修改HotelDoc
实体
HotelDoc
中要添加一个字段,用来做自动补全,内容可以是酒店品牌、城市、商圈等信息。按照自动补全字段的要求,最好是这些字段的数组。
因此我们在HotelDoc
中添加一个suggestion
字段,类型为List<String>
,然后将brand
、city
、business
等信息放到里面。
代码如下:
package cn.itcast.hotel.pojo;import lombok.Data;
import lombok.NoArgsConstructor;import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;@Data
@NoArgsConstructor
public class HotelDoc {private Long id;private String name;private String address;private Integer price;private Integer score;private String brand;private String city;private String starName;private String business;private String location;private String pic;private Object distance;private Boolean isAD;private List<String> suggestion;public HotelDoc(Hotel hotel) {this.id = hotel.getId();this.name = hotel.getName();this.address = hotel.getAddress();this.price = hotel.getPrice();this.score = hotel.getScore();this.brand = hotel.getBrand();this.city = hotel.getCity();this.starName = hotel.getStarName();this.business = hotel.getBusiness();this.location = hotel.getLatitude() + ", " + hotel.getLongitude();this.pic = hotel.getPic();// 组装suggestionif(this.business.contains("/")){// business有多个值,需要切割String[] arr = this.business.split("/");// 添加元素this.suggestion = new ArrayList<>();this.suggestion.add(this.brand);Collections.addAll(this.suggestion, arr);}else {this.suggestion = Arrays.asList(this.brand, this.business);}}
}
4.3.重新导入
重新执行之前编写的导入数据功能,可以看到新的酒店数据中包含了suggestion
:
4.4.自动补全查询的JavaAPI
之前自动补全查询的DSL
,而没有对应的JavaAPI
,这里给出一个示例:
@Testvoid testSuggest() throws IOException {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().suggest(new SuggestBuilder().addSuggestion("suggestions",SuggestBuilders.completionSuggestion("suggestion").prefix("h").skipDuplicates(true).size(10)));// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析响应System.out.println("response = " + response);}
而自动补全的结果也比较特殊,解析的代码如下:
@Testvoid testSuggest() throws IOException {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().suggest(new SuggestBuilder().addSuggestion("suggestions",SuggestBuilders.completionSuggestion("suggestion").prefix("h").skipDuplicates(true).size(10)));// 3.发送请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析响应//System.out.println("response = " + response);Suggest suggest = response.getSuggest();// 4.1 根据名称获取补全结果CompletionSuggestion suggestions = suggest.getSuggestion("suggestions");// 4.2 获取options并遍历for (CompletionSuggestion.Entry.Option option : suggestions.getOptions()) {// 4.3 获取一个option的text,,也就是补全的词条String string = option.getText().string();System.out.println(string);}}
4.5.实现搜索框自动补全
1)在cn.itcast.hotel.web
包下的HotelController
中添加新接口,接收新的请求:
@GetMapping("suggestion")
public List<String> getSuggestions(@RequestParam("key") String prefix) {return hotelService.getSuggestions(prefix);
}
2)在cn.itcast.hotel.service
包下的IhotelService
中添加方法:
List<String> getSuggestions(String prefix);
3)在cn.itcast.hotel.service.impl.HotelService
中实现该方法:
@Override
public List<String> getSuggestions(String prefix) {try {// 1.准备RequestSearchRequest request = new SearchRequest("hotel");// 2.准备DSLrequest.source().suggest(new SuggestBuilder().addSuggestion("suggestions",SuggestBuilders.completionSuggestion("suggestion").prefix(prefix).skipDuplicates(true).size(10)));// 3.发起请求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析结果Suggest suggest = response.getSuggest();// 4.1.根据补全查询名称,获取补全结果CompletionSuggestion suggestions = suggest.getSuggestion("suggestions");// 4.2.获取optionsList<CompletionSuggestion.Entry.Option> options = suggestions.getOptions();// 4.3.遍历List<String> list = new ArrayList<>(options.size());for (CompletionSuggestion.Entry.Option option : options) {String text = option.getText().toString();list.add(text);}return list;} catch (IOException e) {throw new RuntimeException(e);}
}