编程知识 cdmana.com

Open source general search engine based on elasticsearch

Purpose of writing

Recently in use ElasticSearch( hereinafter referred to as ES) When building a real-time data warehouse , For the data in the index are written code or query statements to search , inefficiency . So I want to encapsulate the search process , And display the search results directly in the front end .

chart 1: Data list page
 Data list page
chart 2: Data details page
 Data details page


Function introduction and highlights

1、 Users can enter data directly in the search box , That's right ES Library to search , The search results will be displayed in a list . The list field can be configured by itself ( The specific configuration method will be discussed later ).
2、 Add time filtering and sorting functions , Users can filter and sort the data directly in the selected time period of the page . The premise is that the time field should exist in the data and be configured .
3、 Data classification display , It is convenient to retrieve and view the corresponding data .
4、 Advanced search , You can enter “&&” or “||” Conduct “ And with or with ” Logical search for , amount to MySQL Medium “or and and”. At the same time, users can use “people.name= Li Qifeng ” You can specify the specific fields to search for .


Usage method

1、 After synchronizing the code locally , find application.yml And modify the configuration , Replace the data with your local style .
 Insert picture description here
2、 After code synchronization , find search.sql File and import to mysql In the library ( The library needs to be built in advance , Any library name ), Two tables appear after successful import :table_info、field_info.
among table_info The table structure is as follows :
 Insert picture description here
field_info The table structure is as follows :
 Insert picture description here





Project address

https://github.com/mrliqifeng/searchWebForEs

Feeling

This search engine is very fast 、 Difficulty is not great , It's nothing more than right ElasticSearch Interface call . The purpose of writing is only to serve the real-time data warehouse , You can view the data in the data warehouse faster and more conveniently .
But in the process of Engine Implementation , Many data problems have been exposed , such as Too much dirty data It leads to inefficient search 、 A single piece of data is too large Lead to http Request fault 、 The time field is confusing It can't filter and sort by time 、 There are too many useless or low frequency fields in the data It takes up a lot of space , Too many watches It leads to the low efficiency of business users .
So in the process of realizing the search engine , Do it together Data governance , Data tiering , Data integration Etc . Wait until the search engine is implemented , The data problem described above has been basically solved .
The final conclusion is to make a good search engine , Or you want to use data with ease , Data governance is the top priority .


版权声明
本文为[osc_ 7ehwx8hw]所创,转载请带上原文链接,感谢
https://cdmana.com/2020/12/20201224112939127r.html

Scroll to Top