salvere / search_engine Goto Github PK
View Code? Open in Web Editor NEWThis project forked from yujianjun1025/ysimple
基于java的搜索引擎demo
License: GNU General Public License v2.0
This project forked from yujianjun1025/ysimple
基于java的搜索引擎demo
License: GNU General Public License v2.0
博客主页:http://blog.csdn.net/minna_d 引言: 最近想自己基于java完全重底层开始实现一套搜索, 目前我已写了一个最最基本的大体框架, 内部也做过一些内存、响应时间方面的优化,重提交记录能看出来其实做过很多的尝试, 我深感搜索系统的复杂, 个人精力有限,后面还有很长的路要走,诚邀志同道合之士一起开发。 没人加入也无所谓,路过的点个赞我就很满足啦, 要是能够加入我一起开发提交点代码、fix_bug就更爽歪歪啦 目前的情况: 自带search_data.txt文本5497479行单field数据, 稳定内存2.8G, 响应时间稳定在8毫秒 git地址:https://github.com/yujianjun1025/search_engine 运行方式: 0. 修改logback.xml fileDir 指定日志目录 1. tar -xzvf search_data.txt.tar.gz ./ 2. 启动耗费3.3G内存左右, 建议export MAVEN_OPTS=-Xms50000m,默认MAVEN_OPTS=-Xms10000m 3. 启动执行./run.sh 4.访问:http://127.0.0.1:8080/test.json?world=北京&limit=5 ( limit 为空时,默认返回top3000条数据) 返回数据格式: { ret: true, data: [ { docId: 164334, rank: 24900.37237152425 }, { docId: 699048, rank: 23614.956482398586 }, { docId: 1903045, rank: 18891.965185918867 }, { docId: 2894797, rank: 16583.64799943515 }, { docId: 311685, rank: 16583.64799943515 } ] } 日志格式: limit限制为5条时搜索北京、上海、山海日志: [2015-12-05 22:46:07.536] [621747766@qtp-669663528-2] [INFO ]<logger:195 doSearch>查询词:北京, 过滤到符合要求的docId耗时:2.819毫秒, 结果数5 [2015-12-05 22:46:07.794] [621747766@qtp-669663528-2] [INFO ]<logger:153 doSearch>查询词:北京, 求交得到所有docIds耗时:2.999毫秒, 结果数16359 [2015-12-05 22:46:07.795] [621747766@qtp-669663528-2] [INFO ]<logger:195 doSearch>查询词:北京, 过滤到符合要求的docId耗时:1.728毫秒, 结果数5 [2015-12-05 22:46:08.280] [621747766@qtp-669663528-2] [INFO ]<logger:153 doSearch>查询词:北京, 求交得到所有docIds耗时:3.101毫秒, 结果数16359 [2015-12-05 22:46:08.282] [621747766@qtp-669663528-2] [INFO ]<logger:195 doSearch>查询词:北京, 过滤到符合要求的docId耗时:1.87毫秒, 结果数5 [2015-12-05 22:46:24.148] [621747766@qtp-669663528-2] [INFO ]<logger:153 doSearch>查询词:山海, 求交得到所有docIds耗时:3.438毫秒, 结果数4093 [2015-12-05 22:46:24.149] [621747766@qtp-669663528-2] [INFO ]<logger:195 doSearch>查询词:山海, 过滤到符合要求的docId耗时:0.77毫秒, 结果数5 [2015-12-05 22:46:25.348] [621747766@qtp-669663528-2] [INFO ]<logger:153 doSearch>查询词:山海, 求交得到所有docIds耗时:3.694毫秒, 结果数4093 [2015-12-05 22:46:25.349] [621747766@qtp-669663528-2] [INFO ]<logger:195 doSearch>查询词:山海, 过滤到符合要求的docId耗时:0.884毫秒, 结果数5 [2015-12-05 22:46:33.002] [621747766@qtp-669663528-2] [INFO ]<logger:153 doSearch>查询词:上海, 求交得到所有docIds耗时:3.481毫秒, 结果数14477 [2015-12-05 22:46:33.003] [621747766@qtp-669663528-2] [INFO ]<logger:195 doSearch>查询词:上海, 过滤到符合要求的docId耗时:1.331毫秒, 结果数5 接下来需要做的事: 1、内存、响应时间是需要持之以恒的,优化永无止境 2、分词,目前还是java自带的最最基本的string.toArray(),下一步需要做的是将数字分到一起、英文分开, 汉字依然单字 3、查询语法,目前仅支持紧密度查询, 如“AB”@all_filed这种查询语法,下一步需要丰富查询语法的解析与索引 4、检索字段,目前仅支持单field检索,下一步还需要多文本域检索及属性字段检索 个人邮箱:[email protected]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.