Comments (3)
It's possible to narrow down which URLs are crawled by taking advantage of the SimpleCrawler options. For instance, some JS files were throwing errors that I wasn't interested in, so I removed these from the URLs being crawled:
'link-checker': {
postDeploy: {
site: 'mydomain.com',
options: {
callback: function (crawler) {
crawler.addFetchCondition(function (url) {
return !url.path.match(/\.js$/i);
});
}
}
}
}
More reading and options: https://github.com/cgiffard/node-simplecrawler#excluding-certain-resources-from-downloading
from grunt-link-checker.
@danken00 thanks for responding to this!
from grunt-link-checker.
Pleasure. Thanks for writing the plugin :)
from grunt-link-checker.
Related Issues (16)
- Check for valid #anchor links HOT 1
- De-duplicate URL fetching when only difference is fragment identifier HOT 1
- Default config throwing error HOT 4
- getting 404 for css files HOT 2
- Decouple from Grunt HOT 2
- CONTRIBUTING.md needs work HOT 1
- Anchors throwing 404s HOT 2
- Fatal error: "name" and "value" are required for setHeader(). HOT 12
- Check redirects too HOT 2
- Rename task to `linkChecker` HOT 4
- Fatal error: Request path contains unescaped characters. HOT 1
- readme.md HOT 1
- grunt-link-checker could benefit from a progress indication
- Fatal error: Maximum call stack size exceeded HOT 5
- Reports links with anchors (hashes) in the URL as 404
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grunt-link-checker.