Comments (2)
除了使用lucene的indexWriter,还有一种办法把lucene文件加入到一个空的shard,步骤如下
- 将索引close
- 删除shard中原先的lucene目录和translog目录
- 将lucene文件和translog文件拷贝到shard对应的目录中
4.将索引open,就可以查询了
不过上面的方法有个问题,close和open的操作需要经过master,很容易对master造成压力,任务数较多的情况下,会出现master处理不过来的情况
from es-fastloader.
试了一下,好像有点问题,还得再请教一下,看看是不是我哪里没理解对~
我的测试索引共有2个分片,我在reduce端启动了2个task,通过map端的路由,把所有数据分成了2份,第一个reduce写入到本地启动的es之后,文件中有2个分片,分片0有数据,分片1是空的,第二个reduce是分片1有数据,分片0是空的。
1、在线上es集群新建一个索引(无副本),关闭索引,通过查询获知新索引的2个分片所在机器
2、把数据从hdfs下载到线上es集群的2个节点上,解压。
3、把第一个reduce的0分片中的index和translog拷贝到0分片所在的机器目录下,此reduce的1分片数据未操作
4、把第二个reduce的1分片中的index和translog拷贝到1分片所在的机器目录下,此reduce的0分片数据未操作
5、打开索引
6、查询过程中发现总条目数争取,但是查询明细的时候总有一个shard报错。。。。
"reason": {
"type": "null_pointer_exception",
"reason": null
}
类似上面的错误信息
from es-fastloader.
Related Issues (13)
- 工程中存在公司内部服务相关配置 HOT 1
- Failed to execute goal on project mr2es HOT 2
- 代码不完整,无法将流程串起来 HOT 7
- es高级版本不能routing算法不能再用2.X的代码了 HOT 2
- 打包部署运行问题 HOT 1
- elasticsearch-6.6.1缺失lib文件夹
- 什么时候可以开源es集群管理平台和es收集query查询分析的网关呀 HOT 3
- There is a vulnerability in elasticsearch 6.8.13,upgrade recommended
- There is a vulnerability in log4j 2.11.1,upgrade recommended
- why not use reflect to get InternalEngine and IndexWriter? HOT 1
- 使用spark跑会出现内存溢出的情况,不知道大佬们遇到过没
- xpack权限问题
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from es-fastloader.