mozilla-platform-ops / dxr-infra Goto Github PK

I think mach build is using (the wrong) gmake to build, which causes failures. 1) need logs of where it fails, 2) is there a releng version of gmake in /tools to use instead?, 3) can we get mach to use a different make/gmake?

For testing new builds, it would be nice to figure out how to easily get a new entrypoint-config.yml onto the image that doesn't involve rebuilding the entire container.

If there are no changes to a repo we should consider skipping it. This assumes we already have a clone in the source_folder and presumably an existing index. We might want to verify an index exists, if that's feasible.

Related, possibly worth skipping a build if a tree is closed due to bustage.

There were a bunch of -central trees on MXR that provided a convenient parent directory to bundle a large, and automatically maintained, list of repos. It allowed one to search across all repos there-in, and meant that no manual intervention was required to add new trees. This is, imo, especially important with the l10n repos given the sheer number of them. This might help the Jenkins/multi-scm problems we've been having with the build and nss repos, too.

I'm currently thinking that we do NOT rely on Jenkins to poll the SCMs for change, and instead go with the old timed model. Then maybe index.sh can work some magic to scrape hg.m.o/github for a list of repos? I'm not sure how to pass through that info though, especially for build/* where there is a mix of hg and github repos.

Jenkins often fails when creating a new job because it is unable to create top level directories in /data/dxr_data. Fix that.

changing mozconfig entries in config.yml updates all of the necessary configs, but jenkins doesn't overwrite .mozconfig or otherwise check for differences

DXR's actually expecting ver 3.5 of clang/llvm. hitch is that using the specific versions means that llvm-config isn't found (it's llvm-config-3.5, I think). same goes for clang++. so we need to account for that.

We should take things like es host list from ENV variables to allow people to run this locally and point to their own ES instance.

workers, default_tree, plugins?, es_hosts, es_catalog_replicas, es_indexing_timeout, es_refresh_interval

2018/02/07 14:59:12 [error] 18112#18112: *38 lua entry thread aborted: runtime error: /usr/share/lua/5.1/resty/hmac.lua:4: module 'crypto' not found:
no field package.preload['crypto']
no file '~/lua/crypto.lua'
no file '/usr/share/lua/5.1/crypto.lua'
no file '/usr/local/openresty/site/lualib/crypto.lua'
no file '/usr/local/openresty/site/lualib/crypto/init.lua'
no file '/usr/local/openresty/lualib/crypto.lua'
no file '/usr/local/openresty/lualib/crypto/init.lua'
no file './crypto.lua'
no file '/usr/local/openresty/luajit/share/luajit-2.1.0-beta2/crypto.lua'
no file '/usr/local/share/lua/5.1/crypto.lua'
no file '/usr/local/share/lua/5.1/crypto/init.lua'
no file '/usr/local/openresty/luajit/share/lua/5.1/crypto.lua'
no file '/usr/local/openresty/luajit/share/lua/5.1/crypto/init.lua'
no file '/usr/share/lua/5.1/crypto.so'
no file '/usr/lib64/lua/5.1/crypto.so'
no file '/usr/local/openresty/site/lualib/crypto.so'
no file '/usr/local/openresty/lualib/crypto.so'
no file './crypto.so'
no file '/usr/local/lib/lua/5.1/crypto.so'
no file '/usr/local/openresty/luajit/lib/lua/5.1/crypto.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
coroutine 0:
[C]: in function 'require'
/usr/share/lua/5.1/resty/openidc.lua:705: in function 'openidc_load_jwt_and_verify_crypto'
/usr/share/lua/5.1/resty/openidc.lua:812: in function 'authenticate'
/usr/local/openresty/nginx/conf/openidc_layer.lua:17: in function </usr/local/openresty/nginx/conf/openidc_layer.lua:1>,

As of January 1 2019, Mozilla requires that all GitHub projects include this CODE_OF_CONDUCT.md file in the project root. The file has two parts:

1. Required Text - All text under the headings Community Participation Guidelines and How to Report, are required, and should not be altered.
2. Optional Text - The Project Specific Etiquette heading provides a space to speak more specifically about ways people can work effectively and inclusively together. Some examples of those can be found on the Firefox Debugger project, and Common Voice. (The optional part is commented out in the raw template file, and will not be visible until you modify and uncomment that part.)

If you have any questions about this file, or Code of Conduct policies and procedures, please see Mozilla-GitHub-Standards or email [email protected].

(Message COC001)

rust-lang gets built from scratch for indexing, but it's not installed nor otherwise available to build other rust projects to index, so we should install rustc in the builder image.