image-downloader's People
Forkers
cool3rocks sdtm1016 lilykissme hongqiqu jingle1267 hero9868 ivantong1101 chuhuangvio dharniho arip12 prakash-android acd87 fengw518 captainteknics devmanro 7drrrrrrr nuguril jeyhunmikayil abhishekshil githubnotme sungchanjo q4232134 mrsoapman mayanklal yuandongzhong liipod int16384 endermine2007200 tayfunuyar knightcn1983 ansarker ainoviq fallenfolkfox arshdeepsandhu deepak-garuda liuhang2 adnoh joh7h imagedirect holding-1-at-a-time pongsatorn-r cedev935image-downloader's Issues
login forms
how can i batch download images from a site that requires a login
Getting "TypeError: 'encoding' is an invalid keyword argument for this function 192-168-1-103" in my terminal, and no images....
Hi guys,
Total newb here, (this would be maybe 10th time I've ever opened terminal...lol)
I have downloaded/installed latest version of Python(3.12.0), I have my .csv file in the right place, it has the correct columns, and is with the "image-downloader.py" file. But I don't get any images downloaded into the created folder, and when I look at the readout from terminal this is what it says:
INFO:executed by python 2
INFO:importing data from scraped_data.csv
Traceback (most recent call last):
File "image-downloader.py", line 133, in
main(sys.argv)
File "image-downloader.py", line 125, in main
download_csv_file_images(csv_filename)
File "image-downloader.py", line 111, in download_csv_file_images
with open(filename, "r", encoding="utf8") as csvfile:
TypeError: 'encoding' is an invalid keyword argument for this function
I'm on a pretty ancient mac - maybe that's the problem?
2012 Macbook Pro / MacOS Catalina 10.15.7
Please help! :)
I will go and try on my new Mac mini and see if that helps.....
renaming files trough web-scraper-order column
Hi there,
First of all thank you very much for this script. I really did helped me with my job.
However currently I'm having troubles to rename files through "web-scraper-order" column. If I put there instead of default (f.e.: 1608498486-1648) data desired future file name (f.e.: 3245/285/330) script can't proceed and download the images. Apparently it has something to do with UTF-8 coding and slash - "/".
Can anybody give me an advice how to adjust the script so I can do such thing?
Any advice would be appreciated!
WARNING: Image download error: HTTP Error 401: Unauthorized
The subject error occurs when executing the script.
How do I modify the script?
Problem with format .webp
I have a problem with .webp format when I download :)
Example:
.jpg.pagespeed.ic.r1QtDCQg3m.webp
Handling webp images
You may want to add handling for webp images - I am seeing them in my scrapes. They are often .jpg in the url, but the format that comes down and the Content Type is webp. Easy change in the code is:
if content_type == 'image/jpeg' or content_type == 'image/jpg':
ext = 'jpg'
elif content_type == 'image/png':
ext = 'png'
elif content_type == 'image/gif':
ext = 'gif'
elif content_type =='image/webp':
ext = 'webp'
That way they come down with a webp extension and can be converted.
I would put in a pull request, but don't know if you want to deal with that, so offering it here.
PS: Thank you for all this, it has really helped me!
WARNING:Image download error. HTTP Error 403: Forbidden
i get following error
WARNING:Image download error. HTTP Error 403: Forbidden
INFO:downloading image
Dies when I try....
It just dies when i run it. My file is 3million lines long and 300mb? Was hoping it would work.... Any ideas
screenshot here... just dies... https://imgur.com/V6XFGmA
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.