Comments (2)
Submitted by calvin
Logged In: YES
user_id=9205
Hm, the used functions to construct or merge urls are all
standard functions in urllib. Anyway, this is funky stuff I
have to check out.
from linkchecker.
Submitted by calvin
Logged In: YES
user_id=9205
Using "wget -S http://users.compaqnet.be/robots.txt" reveals
that the last redirect is a broken URL:
http://www.belgacom.netindex.html?l1=search&a=error404&error=http://users.compaqnet.be/robots.txt
This raises a httplib.InvalidUrl exception which is not
handled in the robotparser routines.
I checked in a fix that handles such exceptions to
linkcheck/robotparser.py, rev. 1.17
from linkchecker.
Related Issues (20)
- This Repo is DEAD, please go to new Repo
- This LinkChecker Project is **DEAD**, please go to new Project
- python2.7 安装失败 HOT 1
- /usr/include/stdio.h:33:21: fatal error: stddef.h: No such file or directory HOT 1
- Linkchecker just find just 1 Link
- re-try HOT 1
- LookupError: unknown encoding: cp65001
- ImportError: cannot import name md5 HOT 1
- quote or backtick in linkchecker logs ?
- Website plugins page is broken
- How do I check for only 404 (not 403 errors)?
- Checking local files [usage notes]
- Unable to install. Is it not compatible with pip3 ? HOT 1
- Feature request: Automatic daily check for broken links with email notification
- Error: OSError: Could not find a suitable TLS CA certificate bundle HOT 1
- Option to ignore tag: <meta name="robots" content="nofollow"> HOT 1
- Internal error HOT 1
- This LinkChecker Project is **DEAD**, please go to new Project
- Internal Error- linkchecker HOT 1
- Error: SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from linkchecker.