smahesh29 / web-scraping-python

It contains some web scraping examples implemented using Python.

Web-Scraping-Python

It contains some web scraping examples implemented using Python.

1. Internshala :

Python codes for Scraping Internship from Internshala

Additional Python Libraries Required :

• BeautifulSoup

  pip install beautifulsoup4

• Pandas

  pip install pandas

• Requests

  pip install requests

Usage :

This directory contains 2 python files :

1. python_django_internship.py :

This program is used to get details of all the Python/Django category internships from first page. It will store the details in python_django_internship.xlsx sheet. To Run the program :

• Open your Command Prompt or Terminal and change directory to the same folder where you have saved the program.
• Use Command :

                    python python_django_internship.py

2. keyword_internship.py :

This program is used to get details of all the internship of specified Keyword. The keyword should be specified during the execution of program. The program will ask for keyword, after running it. It will store the details in keyword_internship.xlsx sheet. To Run the program :

• Open your Command Prompt or Terminal and change directory to the same folder where you have saved the program.
• Use Command :

                  python keyword_internship.py

2. Google image Downloader :

A Python program for downloading images from a Google Search.

Additional Python Libraries Required :

• BeautifulSoup

  pip install beautifulsoup4

• Requests

  pip install requests

Usage :

Open your Command Prompt or Terminal and change directory to the same folder where you have saved the program. Type the following command in your CMD or Terminal:

 python image_downloader.py -n 50 -s "apples" -d "E:\images"

Here, change the following as per your needs:

-n: Number of Images to be downloaded (Default = 10).

-s: Search Query (Default = red roses).

-d: Directory where to save the images(Default = Download - the directory present with code).

3. Flipkart

A Python program to extract the Price, Name, and Rating of Laptops from Filpkart website.

Pre-requisites :

1. Additional Python Libraries Required :

• Selenium

  pip install selenium

• BeautifulSoup

  pip install beautifulsoup4

• Pandas

  pip install pandas

2. chromedriver (For Windows operating system you can download the chromedriver from this repository)

Usage :

• Open your Command Prompt or Terminal and change directory to the same folder where you have saved the program.
• Use Command :

                  python laptop_details.py              

• The laptops details will be stored in laptop_details.csv file.

4. Youtube :

A Python program to extract the information such as title, duration, channel, url, views, description, etc. of youtube trending videos.

Additional Python Libraries Required :

• BeautifulSoup

  pip install beautifulsoup4

• Pandas

  pip install pandas

• Requests

  pip install requests

Usage :

• Open your Command Prompt or Terminal and change directory to the same folder where you have saved the program.
• Use Command :

                  python youtube_trending.py             

• The videos details will be stored in youtube_trending.xlsx file.

web-scraping-python's Issues

Help

Can you please share your knowledge on how to build this project as a application and how to deploy in cloud !

Getting empty data frame

Hello Mahesh,
I am getting laptop_details.csv generated but its empty and dont have any data in it. I have followed all the instructions but seems something is missing here.
Also, getting this error everytime
DevTools listening on ws://127.0.0.1:61528/devtools/browser/730b1e58-ccab-44ca-8cf1-7a40256c0b3a.