Comments (7)
This is an issue that greatly interests me as well. I have at least one use case where I've had to do the ugly hack of spawning multiple threads in a single phase.
My understanding is that the way threading works in the test executor and the way the framework talks to the station server it would probably be very difficult to make this refactor.
I also however think this would be a hugely beneficial feature.
For now, you can consider an approach of parallelism at the test vs phase level. That should work, but still doesn't play too well with the station server.
from openhtf.
I would add that in addition to and somewhat related to supporting parallelism would be an easier way of defining test flow. Nesting phase nodes essentially creates a DAG and we could think about a more sophisticated execution approach where rather than just traversing the tree in order and executing phases sequentially, we instead create pools to execute independent phases and basically have each phase be triggered to execute immediately when the phases it depends on complete.
Building the infra from the ground up in this way might also lend itself towards better logging/post analysis. Imagine a test record that preserves the DAG structure rather than just giving you a flattened list of phases you can't really correlate back to the original phase node descriptors.
from openhtf.
Thanks for your comments @gtpalmer.
Especially on production lines, time is money, so I agree this kind of support would be beneficial.
I've considered using Test objects for this purpose, but this would make the management of our complete test more complex (managing the threads, logging callbacks, station manager etc.). I'll go with this option eventually, if I find no other solution.
from openhtf.
Good points here, thanks all for commenting.
We are also interested in parallel phases, and looked into how it could be done into OpenHTF. The conclusion was that it would basically require rewriting large parts of the OpenHTF executor to support true parallel phases. At that point, we would also want to redesign some of the APIs to better work with potentially parallel phases. Currently we're not planning to do such a change for OpenHTF.
A couple of workarounds can be used:
- Running entire tests in parallel - think of a "multi-up" test bench.
- Managing your own threads from inside a phase - a bit clunky as you've to manage your own threads and synchronization.
If and when there's a ground-up rework, parallel phases would definitely be on the feature list.
from openhtf.
This is possible with the PhaseNode implementations I added a few years back.
This can be accomplished with a new Node type:
ParallelPhaseNode, a PhaseCollection that has a list of phases to run in parallel. It should run each node it finds in a separate thread. If there are nested collections (i.e. sequences or groups), those collections should run in order.
The parallel node needs an option to determine how fatal errors affect the parallel processing; either it should stop parallel phases immediately or continue running.
The implementation could be handled by breaking up the TestExecutor a bit to allow for parallel copies, creating copies for new nodes. It's especially important for there to be slightly different TestState objects for the parallel phases.
from openhtf.
@arsharma1 Do you have code for this parallel phase collection or is it just hypothetical? Seems like a possible solution - though it could still be problematic in terms of sharing the same test state and displaying updates on the webfront (what would the currently running phase be?).
More generally I agree with fact that the test_api interaction would have to be tweaked. My proposal is along these lines:
- Add some infrastructure to create "local" test APIs whose state can be merged after a phase and/or collection is executed.
- To make it easier to add new types of phase nodes, and even have user defined nodes, have the execution of the phase node be defined by the node itself.
- A phase node (collection or descriptor) can return a record object directly. This object would always have a top level result enum, and always be serializable. This you can easily merge it into a test record object or an outer node record.
- Update the executor to basically just execute a phase node and then merge the node record and test API into the global test state.
from openhtf.
I'd love to work on a prototype of this if people are interested, though it's complicated right now because I had been working for Argo AI... I'm not sure if I'll be using openhtf at my next role.
from openhtf.
Related Issues (20)
- If plug configuration is dynamic (e.g. configs loaded from a file), how can the decorators be updated accordingly? HOT 2
- frontend_example with PyInstaller HOT 3
- Request: Add ability to FAIL_AND_CONTINUE instead of STOP when a Phase reaches max_repeats
- How to build a test running multiple test groups concurrently, but showing I/O data of all test groups on the same webpage? HOT 1
- HIL testing? HOT 5
- multiple monitors for a single phase?
- KeyError in `PlugManager.provide_plugs` HOT 6
- monitors decorator does not fully copy the specification of the monitored phase
- Feature Request(?): Provide a mechanism for displaying images in the front-end HOT 1
- Progress bar/completion status stops if test is marked as FAIL. HOT 4
- How could I call a PhaseDescriptor explicitly? HOT 3
- runtime reporting/logging for each phase during test HOT 6
- Having trouble using the unit test decorator @test.patch_plugs to mock a device I'm testing. HOT 3
- where PyOpenSSL is used?
- Unused `six` import in monitor code
- Replacing `M2Crypto` dependency with `cryptography`
- Status LED on failure HOT 1
- Multiple tests or one test and multiple subtests?
- Best practice on how to organize product-dependent tests
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openhtf.