Comments (3)
I just added the following to the setting.py and hey presto
# Disable S3
AWS_ACCESS_KEY_ID = ""
AWS_SECRET_ACCESS_KEY = ""
Hope this helps someone else
Newbie to Py and Scrapy
Richard
from scrapybook.
Answering to myself, it seems that the boto library is showing this error when it can't connect to an S3/AWS host. Just set the variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to an empty string and your are good to go:
$ export AWS_ACCESS_KEY_ID="" && export AWS_SECRET_ACCESS_KEY="" && scrapy shell http://web:9312/properties/property_000000.html
(I suppose that there is a more elegant way to bypass this error, but that line works for me :)
from scrapybook.
Thanks a lot @juananpe for the clarification. This refers to page 113 of the e-book, or page 92 of the printed book.
You are right on exporting AWS_*
credentials. This is exactly what I do as well in settings.py for Chapter 5 and any other chapter. As you say - it's just annoying boto detail and a non-elegant workaround is fine.
Hopefully this won't be a very common problem. If you run your scrapy shell
command from within ch05/properties
directory, it should work fine because scrapy shell
includes automatically settings.py
. My guess is you ran scrapy shell
from a top-level directory.
It's great clarification. Thanks a million!
from scrapybook.
Related Issues (20)
- vagrant up error HOT 1
- can't access http://scrapybook.s3.amazonaws.com/properties/ 403 forbidden HOT 1
- there is an Syntax Error on page 16
- is it because of socks5?
- seems that I have the same problem: HOT 1
- install panda
- OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to d1kby793vfk4bq.cloudfront.net:443 HOT 1
- Virtual machine has terminated unexpectedly during startup with exit code 1 (0x1) HOT 1
- Can't deploy 'properties' spider to scrapinghub.com from the docker container (chapter 6)
- Vagrant up --no-parallel You are trying to forward a host IP that does not exist. HOT 1
- package 'openssh-server' is not installed
- The problem of setting up the environment HOT 1
- Cloning into 'algo-cs503'... fatal: unable to access 'https://github.com/saqibutm/algo-cs503.git/': error setting certificate verify locations: CAfile: D:/4th semster/ds/Git/mingw64/ssl/certs/ca-bundle.crt CApath: none this is the issue can plzz solve the issue
- Vagrant Setup - Resolving port conflicts on Mac HOT 1
- !!
- can't visit http://web:9312/ HOT 1
- how to connect local github with github id
- VAGRANT UP ERROR 2022
- vagrant up --no-parallel command not working
- vagrant up --no-parallel command not working HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from scrapybook.