Comments (2)
mvdbos
commented on August 19, 2024
Please check out the link checker example. It uses a custom request handler, that configures the default Guzzle request handler to not fail on 4XX and 5XX responses. This way, these responses will be persisted too and don't stop processing. This can be used for link checking or simply to be resistant agains failures.
from php-spider.
dougblackjr
commented on August 19, 2024
@mvdbos Perfect, thank you!
from php-spider.
Related Issues (20)
- Add debug logging
- Add prefetch filter to cache downloads with a max age
- Is it possible to skip creation of the results files and just report if the links are valid? HOT 3
- Fatal error: Uncaught TypeError: Example\GuzzleTimerMiddleware::onResponse(): Argument #3 ($response) must be of type GuzzleHttp\Promise\FulfilledPromise, GuzzleHttp\Promise\RejectedPromise given
- Confusing separate event dispatchers HOT 1
- Not showing result in https web HOT 1
- How php-spider work with proxy? HOT 2
- Wrong Event Arguments HOT 4
- suitable as link checker? HOT 4
- Is it possible to count/track the number of links pointing to a page? HOT 2
- How can I pass a form like login or signup? HOT 4
- "EventDispatcher::dispatch() must be an object, string given" HOT 3
- Cannot retrieve href attribute of source page HOT 3
- Robots.txt filtering? HOT 8
- Follow only internal redirects HOT 2
- Upgrading to support Symfony 6. HOT 4
- Pass Query Parameters HOT 3
- StatsHandler has been deleted (mistakenly?) HOT 1
- get source url HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from php-spider.