paritytech / zombienet-sdk Goto Github PK
View Code? Open in Web Editor NEWZombieNet SDK
Home Page: https://paritytech.github.io/zombienet-sdk/zombienet_sdk/
License: GNU General Public License v3.0
ZombieNet SDK
Home Page: https://paritytech.github.io/zombienet-sdk/zombienet_sdk/
License: GNU General Public License v3.0
Allow to isolate the single peer (or a group of peers) from the rest of the network for the given period
Component of Configuration Crate
As part of the building of the new SDK (#1), we want to create a PoC
with a subset of features (spawning only) to collect feedback from the internal teams.
The subset of features we want to achieve in this Poc
:
./crates/configuration
)./crates/orchestrator
)examples
from https://github.com/paritytech/zombienet/tree/main/examplesComponent of Orchestrator Crate
Ones we have docker as provider (#38), we should change the setup to use it as default since is the easiest way to start using zombienet (in both mac/linux).
Component of Provider Crate
Component of Configuration Crate
In order to automate the package installation for this new service we will turn it into helm charts that easily can be deployed to whatever cluster we want
Network
high-level specs:
We should allow to use a builder pattern to construct the network and the build
method should create all the needed configuration to spawn the network.
spawn
method should check that all the needed artifacts are ready, and if not we should build
internally.
info
method should return all the running config info (endpoints/chain-spec paths/ log paths/etc)
destroy
method should tear-down all the network and clean-up all the resources. (we want to allow to save the logs/database for a later analysis)
rebuild
should allow to rebuild the network
instance from the same json we get from info
.
This schema is a proposal of an architecture that could be done for the SDK:
Open for your suggestions !
Component of Provider Crate
This issue will track the progress of the new ZombieNet SDK.
We want to create a new SDK for ZombieNet
that allow users to build more complex use cases and interact with the network in a more flexible and programatic way.
The SDK will provide a set of building blocks
that users can combine in order to spawn and interact (test/query/etc) with the network providing a fluent api to craft different topologies and assertions to the running network. The new SDK
will support the same range of providers
and configurations that can be created in the current version (v1).
We also want to continue supporting the CLI
interface but should be updated to use the SDK
under the hood.
We plan to divide the work phases to. ensure we cover all the requirement and inside each phase in small tasks, covering one of the building blocks and the interaction between them.
After [team's discussion] (#22) the following architecture was agreed to be followed as a good-initial plan:
## Prototype building blocks
Prototype each building block with a clear interface and how to interact with it
Building blocks are replaced with the structure depicted in the image above with crates and crate components;
All issues of Prototype building blocks
are closed and replaced with the ones mentioned in the next paragraph;
./crates/support
)./crates/configuration
)./crates/providers
)./crates/orchestrator
)./crates/test-runner
)We want to integrate the interactions for all building blocks and document the way that they work together.
CLI
and ensure backwards compatibilityRefactor the CLI
module to use the new SDK
under the hood.
Component of Test Runner Crate
Component of Orchestrator Crate
Component of Provider Crate
Responsible for executing tests and reporting their results (./crates/test-runner
)
Note: Depends on Support
, Configuration
crates
Represent an individual node
in the network topology, inherit from the default_*
config of the network
(iif is attached to one) and we should have nice defaults
to allow run a single node
easily.
Depends on #38
We want then to add some smoke
test in GHA for this provider.
Component of Test Runner Crate
Component of Configuration Crate
Component of Provider Crate
Component of Orchestrator Crate
Component of Test Runner Crate
In the future, it would be nice for zombienet to provide an API for injecting network issues.
Zombienet clusters and internal testnets are usually spawned within a single data center, where network speeds are often close to optimal. Unfortunately, in the real world things are not so ideal. Therefore, I'd propose to extend zombienet functionality w/ ability to simulate bad & adversary network conditions. Sort of how Jepsen provides an API for splitting the network into halves (or majorities).
Here is the list of things I think we want in the first approximation:
tc & netem
on Linux) for all connections of the given peeriptables
on Linux)iptables
on Linux)(1) can be simplified into multiple predefined options: "slow", "50_loss", etc.
Slightly unrelated, but killing random nodes (chaos engineering; using tools like https://github.com/asobti/kube-monkey) also sound like a good idea to me.
Curious to hear your thoughts on this ๐
How hard would it be to implement something like this? Are there existing tools for Kubernetes that give you control over network? What zombienet API would look like?
UPDATE:
Adding individual issues to track progress:
Component of Orchestrator Crate
Component of Support Crate
Component of Support Crate
Add support to set:
latency
packet loss
jitter
We should support in the available providers:
k8s
through chaos-mesh CRD (installed in the cluster)
native
( tc & netem on Linux)
podman
(need exploration).
Component of Configuration Crate
Component of Provider Crate
Component of Support Crate
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.