Visual Testing for modern web apps. Review visual changes in your development workflow.
Home Page: https://app.argos-ci.com
License: MIT License
Argos is a visual testing solution that fits in your workflow to avoid visual regression. Takes screenshots on each commit and be notified if something changes.
Changes and improvements are more than welcome! Please make your changes in a specific branch and request to pull into main! To learn more about it, have a look at our CONTRIBUTING.md.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
As a user, I have a simple and inefficient way to filter out flaky tests.
While flaky tests can be fixed, I would expect any new product using the service to have quite some unpredictable screenshots. The solution I'm proposing is far from perfect but can be easily implemented.
Sentry should get the context of the job in order to debug easily.
Job should be marked in error in database when it fails 3 times.
Notifications should be done in an other job in order to track notifications in database and to be able to replay them if an error occurs.
ETQU, I can download screenshots from another build using the CLI
I can start a local server from CLI that display "hello world".
As a user, I can click on the build summary link and go back to GitHub.
We constantly switch between services, the easier the better.
We gonna need to get the repository base url.
When I tape "argos diff --branch master ./my-screenshots", I can see in my browser a local interface presenting diff.
Running the CLI locally helps getting faster feedback and avoid using our server ressources.
Also add index on "users.login".
ETQU, I have a build number per repository and not a shared one for the platform.
It's simpler for users.
We are going to need a new column in the build table.
Collapse the card when the score is 0.
ETQU, Instead of being redirecting to the homepage of argos, I'm redirected to the specific build.
It saves time for users.
We gonna need a path builder, we could use rails as inspiration.
ETQU, I can signin / login using my Github account.
Linked to #19.
In a separated repository, create a command line interface named "argos-cli".
When I type "argos" or "argos --help", I can see the help of the CLI.
name
commit
branch
screenshots[]
{
id
name
commit
branch
jobStatus
}The goal is to be able to run and process some jobs using RabbitMQ.
When a diff is computed, resulting images should be uploaded to s3 and referenced in "screenshot_diffs" table.
We have to find a solution, but requeuing is not a good one.
Jest is :
As a user, after logged on, I'm redirected to /.
I can see a list of the repositories I have access to and that are enabled for Argos.
Gives users the possibilities to navigate to a repository.
We need to request GitHub to get repositories the user have access too.
Then, we need to look into the repositories table.
We expose a GraphQL entry point.
query {
repositories {
name
token
enabled
}
}Right now, we have that ...
As a user, I have access to a validation button that is going to accept the changes.
So we can keep track of what was accepted and what wasn't.
As a user, I can connect using a button with GitHub logo.
We need an authentification method and we want to integrate to GitHub first.
When I am a new user, an account with my information is created and I am logged in for one month.
When I am not a new user, my account is updated and I am logged for one month.
If I am not logged, I see a login button in the header, if I am logged, I see a logout button.
When I click on the logout button, I am logged out.
As a user, I can see my profile and the repositories I own.
Once we can display the repositories I own, we can propose user to enable the integration with Argos.
Create a page /profile/:profileId that display profile information:
Model: https://travis-ci.org/profile
When I tape "argos whoami", I can see information about the current project.
Help users understanding what's going one.
Technically, a ".argosrc.json" containing an entry "token" is read and the API "http://api.argos-ci.com/whoami" is called using the token.
The job will take two screenshot ids and will generate a screenshot diff.
We need to be flexible and to simplify the integration.
Stop retrying the jobs after 3 fails. If we fail more than 3 times, we should flag an error.
Following our postmortem, we shouldn't retry the screenshotDiffs job indefinitely.
We made 800k screenshots in a single day and had many errors:
As a developper, I can always use a seed to start working on the project.
Save time, shorter feedback loop.
Run the seed in the CI to prevent regression.
Used internally by the CLI.
[
{
id
name
commit
branch
jobStatus
}
]In tests, we should use factories.
As a user, I can the history of the builds for a given repository.
Helps making sure things are working as expected.
We gonna expose a graphQL api.
When I tape "argos upload --token xx", screenshots are uploaded and a build is created.
Be able to run a build from a CI service.
We call POST https://api.argos-ci.com/builds with all screenshots.
It's important to keep data very consistent and this from the beginning, the idea is to migrate database and add model validation to start with consistent data.
We should provide a string, not an object.
As a user I can see the organization I'm in. I can navigate to them.
The same as #43.
One the "/profile/:profileId" page, list organizations and some information:
At the bottom add a link to change settings.
Model: https://travis-ci.org/profile
As a user, I can see an ordered list of repositories.
Repositories wheren't created equals. Display first the one that have the newest build sounds like a good choice.
Display a button to sync repositories, on hover, display the last date of synchronisation.
Model: https://travis-ci.org/profile
Synchronisation must be done in a job and must be launch at login, the button should display the status of synchronisation.
Save money
It's inside the screenshotDiff job.
https://elements.heroku.com/addons/rabbitmq-bigwig
Scaling should be better with it, number of message is unlimited.
Allow running the CLI tool locally.
argos-bucket-${branch}-${id}.zipAs a user, I can quickly understand if a build introduce potential regressions or not.
Travis is doing it very well. We could use red when a build is marked at wrong, orange when we don't know, green when it's good and grey when it's in progress/pending.
As a user, I can compare two screenshots using one of the following methods.
Helps better catching regression.
ETQU, I can see a list of screenshot diffs for a single build.
That's the whole point of this project.
The interface could be using a Grid list. Something close to Google Photo should do it.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.