Comments (7)
FastIndexLoadDataCollector是有一个定时任务启动的,由于是内部服务,我没有把相关代码放进项目中,不过你可以任务这个认识是每分钟执行一次
ZeusUtils也是滴滴内部的服务,你可以任务是一个在远端节点上执行脚本的工具
from es-fastloader.
好的,感谢。关于expandfactor参数,代码里最大值是20,这个是基于什么考虑呢,如果改成比如200,会有什么影响吗
from es-fastloader.
Server中定时任务调度是用的xxl-job实现的吗?
from es-fastloader.
好的,感谢。关于expandfactor参数,代码里最大值是20,这个是基于什么考虑呢,如果改成比如200,会有什么影响吗
限制20主要是为了不让reducer任务过多,占用过多资源
from es-fastloader.
Server中定时任务调度是用的xxl-job实现的吗?
调度任务是我们内部自研的
from es-fastloader.
感觉生成了lucene文件并上传到hdfs之后的流程由于有滴滴内部的诸多服务的原因,导致代码有点断层,希望楼主能在github项目里用一个类似markDown的方式简单阐述一下流程就好~,感谢
from es-fastloader.
已更新开发文档,欢迎交流
from es-fastloader.
Related Issues (13)
- 工程中存在公司内部服务相关配置 HOT 1
- 请教一个问题,关于nodemanager端本地生成的lucene文件加载到es本地节点之后的操作 HOT 2
- 打包部署运行问题 HOT 1
- elasticsearch-6.6.1缺失lib文件夹
- 什么时候可以开源es集群管理平台和es收集query查询分析的网关呀 HOT 3
- There is a vulnerability in elasticsearch 6.8.13,upgrade recommended
- There is a vulnerability in log4j 2.11.1,upgrade recommended
- Failed to execute goal on project mr2es HOT 2
- why not use reflect to get InternalEngine and IndexWriter? HOT 1
- 使用spark跑会出现内存溢出的情况,不知道大佬们遇到过没
- xpack权限问题
- es高级版本不能routing算法不能再用2.X的代码了 HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from es-fastloader.