Advanced declarative web scraping
License: Apache License 2.0
Server for advanced web scraping.
Open API defintion.
Be aware, that the project is under heavy development.
There is no documentation and some things may change in the final release.
You can download latest binaries from here.
ferret-server --db=http://0.0.0.0:8529![dependabot-preview[bot] avatar](https://avatars.githubusercontent.com/in/2141?v=4 "dependabot-preview[bot]")
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Create the possibility to create a collection keeping all output data which is dedicated to a specific script .
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: github.com/go-openapi/[email protected] requires
github.com/docker/[email protected]: reading github.com/docker/go-units/go.mod at revision v0.4.0: unknown revision
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
github.com/MontFerret/ferret-server/server: cannot find module providing package github.com/MontFerret/ferret-server/server
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
github.com/MontFerret/ferret-server/server: cannot find module providing package github.com/MontFerret/ferret-server/server
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Run script execution using web hooks.
Dependabot couldn't resolve your project's dependencies as it couldn't access strfmt.
You can provide additional git credentials in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'. If you use a custom token for the host github.com make sure it has read access to this repo, too.
Add the possibility to notify when a scripts starts or ends or gets cancelled by sending an email.
This is a reminder and one of the milestones towards 1.0 release.
Guidelines, API documentations and examples.
Preferably as a GitHub Page with nice theme.
With link or integration with Ferret Documentation
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: golang.org/x/[email protected]: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /opt/go/gopath/pkg/mod/cache/vcs/ed42bd05533fd84ae290a5d33ebd3695a0a2b06131beebd5450825bee8603aca: exit status 128:
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
github.com/MontFerret/ferret-server/server: cannot find module providing package github.com/MontFerret/ferret-server/server
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Add the possibility to create webhooks for 3rd party systems to notify when a scripts starts or ends or gets cancelled.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Design API for remote persistence.
Add a collection that keeps all past and active executions.
It must have the following data:
And add an API endpoints:
projects/{project}/history - Query all entriesprojects/{project}/history/{scriptId} - Query all entries for a given scripthistory is not a final segment name.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: github.com/go-openapi/[email protected] requires
github.com/go-openapi/[email protected]: reading github.com/go-openapi/analysis/go.mod at revision v0.19.10: unknown revision
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Create a dedicated Docker image and publish it on Docker Hub registry.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
github.com/MontFerret/ferret-server/server: cannot find module providing package github.com/MontFerret/ferret-server/server
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: github.com/go-openapi/[email protected] requires
github.com/go-openapi/[email protected]: reading github.com/go-openapi/jsonpointer/go.mod at revision v0.19.3: unknown revision
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Do not have particular ideas about authentication with some roles.
Just a draft:
Add the possibility to query the results data using native AQL.
There are some security concerns.
Probably, we will need to implement #2 first, in order to limit access to system collections.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: github.com/rs/[email protected] requires
github.com/zenazn/[email protected]: reading github.com/zenazn/goji/go.mod at revision v0.9.0: unknown revision
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Add the possibility to validate script results before saving them using JSON Schema.
Must be optional.
Dependabot couldn't resolve your project's dependencies as it couldn't access flag.
You can provide additional git credentials in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'. If you use a custom token for the host github.com make sure it has read access to this repo, too.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: github.com/arangodb/[email protected] requires
github.com/dgrijalva/[email protected]+incompatible: reading github.com/dgrijalva/jwt-go/go.mod at revision v3.2.0: unknown revision
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
POST projects/{projectId}/runtime/stop/{scriptId}
Return nothing.
Fails if the execution does not exist.
The execution must be stored in history even though there is no script id.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
github.com/MontFerret/ferret-server/server: cannot find module providing package github.com/MontFerret/ferret-server/server
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
POST projects/{projectId}/runtime/start/{scriptId}
With the possibility to redefine param values.
Should return a unique id that can be used to check the execution status and retrieve output data.
Create scripts scheduling.
There is a caveat.
If we do in-memory scheduling - it wont' work well on horizontal scaling (if we deploy multiple instances of the server).
If we do 3rd party scheduling (like using Chronos) - we will increase complexity of deployment.
ArangoDB, unfortunately, does not support change events which would simplify the task.
Design system settings:
What to store:
Added end to end integration tests for API endpoints.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
downloaded: h1:FnLRMYxmWxiuv10sNS9XgEy4/x01qgBcLgrIzqbzwHA=
go.sum: h1:9t7z7TqeVYgabjFQz9UytgbUotR0sA/k/3EUlPRV4aw=
SECURITY ERROR
This download does NOT match an earlier download recorded in go.sum.
The bits may have been replaced on the origin server, or an attacker may
have intercepted the download attempt.
For more information, see 'go help module-auth'.
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
verifying github.com/MontFerret/[email protected]/go.mod: checksum mismatch
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot can't resolve your Go dependency files.
As a result, Dependabot couldn't update your dependencies.
The error Dependabot encountered was:
go: github.com/go-openapi/[email protected] requires
github.com/google/[email protected]: reading github.com/google/uuid/go.mod at revision v1.1.1: unknown revision
If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.
Dependabot couldn't resolve your project's dependencies as it couldn't access negroni.
You can provide additional git credentials in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'. If you use a custom token for the host github.com make sure it has read access to this repo, too.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.