Comments (1)
It's already split off, here's how we initialize the queue right now:
Minitest.queue = CI::Queue::Redis.new(
TestGlobs.all_tests(seed: ENV['BUILDKITE_COMMIT']),
build_id: build_id,
worker_id: ENV.fetch('BUILDKITE_PARALLEL_JOB'),
redis: CIRedis.connection,
**queue_config,
)
All queue implementations just stupidly take a list of strings (test identifiers). If we want to filter out parts of the tests, it can and should be done beforehand.
The only downside is that for resiliency purposes, all workers can potentially be elected master, which mean they all compute the list of tests, but only one actually get to populate the shared queue with it.
So if we were to have a costly way to reduce the list it would be a bit wasteful.
what do you think?
IMO if we want to filter out the tests, it should be the responsibility of a dedicated service which can hold state, record all tests runs results and refine it's heuristics based on that data.
e.g. curl https://example.com/commits/abcdef092332432/tests.txt
.
from ci-queue.
Related Issues (20)
- DRY redis setup in ci-queue tests HOT 1
- Sometimes one worker goes rogue and doesn't stop at the end, making my build fail HOT 11
- Q: Smallest unit of work in an RSpec suite HOT 6
- Expire old test runs HOT 1
- Q: Is ordering of tests in the queue based on previous timing statistics? HOT 3
- Project root gets greedily stripped from failure backtraces in JUnit Reporter
- Do you use this in conjunction with Rails 6 parallel tests? HOT 4
- Ruby ci-queue requires gems that are not runtime dependencies HOT 3
- Support multiple `-I` flags in `minitest-queue` HOT 1
- `minitest-queue` `ENOENT` error if `log` folder does not exist HOT 1
- ci-queue: Classification Check HOT 3
- Add CHANGELOG
- Re-add Python lint and 3.8 support
- Redis 4.6 is causing CI warnings of deprecation HOT 2
- Update ciqueue for pytest
- Error thrown when no worker-id is present when redis > 5.0.0 HOT 6
- More aggressively retry Redis commands
- Remove Ruby 2.7 support HOT 2
- 0.28.0 version is not compatible with redis 4.8.1+ HOT 1
- Memory leak from Redis due to missing key expirations HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ci-queue.