parth-vader / fb-spider

Accepts a page name and shows latest posts and comments in a new browser window.

FB-Spider

A program which accepts the id of a Facebook page and transforms into a table of the latest 5 posts and the respective latest 5 comments per post. The table will be in .html format

The number of posts and comments can be changed by editing the graph.py file.

Table of contents

• Get Started
• Requirements
• Instructions
• Contribute

Get Started

(Back to top)

You would require a Facebook developer account to get an access token : https://developers.facebook.com/

Register your app and replace the 'YOUR_ACCESS_TOKEN' in graph.py by your User Token : https://developers.facebook.com/tools/accesstoken/.

Requirements

(Back to top)

Other than requiring Python3.x , you require the following libraries:

• facepy
  
• json
  
• json2html
  
• webbrowser
  
• PyQt4
  
• SIP
  

Instructions

(Back to top)

Instructions for the CLI Version

1. Clone the repository to your machine.

2. Open your terminal and change directory to your cloned project folder.

3. $ pip3 install --editable . This will install all the requirements listed.

4. Now run the program using $ fbspider.

5. Instructions for using the CLI

$ fbspider

When starting the program for the first time, use: $ fbspider initialise

This will ask for your graph ACCESS TOKEN, which you can find here

You can see your saved values by : $ fbspider show [OPTIONS]

Options:
  --token  Shows the user access token stored.
  --npa    Shows the default no of pages for given input.
  --npo    Shows the default no of top post in the output.
  --help   Show this message and exit.

To edit these values, use : $ fbspider edit [OPTIONS]

Options:
  --token  Edits the user access token stored.
  --npa    Edits the no of pages.
  --npo    Edits the no of top post.
  --help   Show this message and exit.
  

To search for a page, use : $ fbspider search [OPTIONS] <Page name>

Options:
  --npa INTEGER  the no of pages.
  --npo INTEGER  the no of top post.
  --help         Show this message and exit.
  

To see the help message, use: $ fbspider --help

Usage: fbspider [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  edit        edits the specified value stored.
  initialise  Initialise with the required info.
  search      Search the page.
  show        Shows the specified value stored.

Please note that this app will only work for public pages and not from profiles of other people.
That would require permission from the user.

6. Enjoy the contents in a table format. A file called 'Table.html' would be made in the folder itself. It is opened automatically after running a valid search.

Instructions for the GUI Version

(Back to top)

For the GUI Version, follow the steps

1. Clone the repository to your local machine, open the terminal and change directory to your cloned project folder.

2. Use a virtual environment virtualenv --python=python3.5 --no-site-packages venv

3. PyQt4 is not supported in Virtual Environment, so you need to setup PyQt manually in the virtualenv.

4. Two dependencies need to be downloaded,PyQt4 and SIP.

5. Download the binaries from the given website - (Choose the Source Packages)

   PYQT4
   SIP

6. In the project folder, make a folder named build, and extract the PyQt4 and SIP, in the build folder.

7. Activate virtualenv using . venv/bin/activate

8. Install some prerequisites

   • sudo apt-get install python2.7-dev libxext-dev python-qt4 qt4-dev-tools build-essential

9. Go to build/SIP directory, and type the following commands: python configure.py sudo make sudo make install

10. Similarily for PyQt4, go to the build/PYQT4 directory and type the commands: python configure.py sudo make sudo make install

11. After the installation, run the following file python3 main.py

12. A GUI will open, enter your Access Code, and press Go.

13. Enter the page you want to search, and at last the choice of page number.

14. Press Generate HTML, to generate the HTML File.

Thanks to the Youtube Tutorial for preparing the instruction manual: How to Install PyQt4 Inside Virtualenv

Contribute

(Back to top)

You can contribute to the repo via this really-simple steps:

• Star and fork this repo.
  
• Join the (slack channel) . Discuss your ideas on how to improve the project with members or ask your doubts regarding the project.
  
• Clone your repo.
  
• Tackle down the issues or add your own innovations.
  
• Pull Request
  

fb-spider's Issues

Update Readme.md

Update the README.md for the changes appearing due to addition of setup.py and removal of requirements.txt.

adding setup.py instead of requirments.txt

UnicodeDecodeError : charmap codec could not decode byte 0x8d in Windows

Improve the README.md

Write all the steps and give all the necessary instructions

Add Dependecny List

Add a requirements.txt file, so that it is easier to run the program.

No variable named YOUR_ACCESS_TOKEN

There is no variable named YOUR_ACCESS_TOKEN in graph.py.

json2html.json2html escapes HTML tags by default

When using json2html.json2html it converts HTML tags like to <a> and so on. This means that Page doesn't show embedded links and images but HTML code

Measure the bandwidth usage for FB-Spider vs Normal Web browsing

If we can somehow measure how much bandwidth does it require to open a page and see it's last n posts and comments on a web browser (this would have to be isolated - i.e. no other internet using app or webpage should be open) and through FB-Spider (GUI and CLI) then maybe we can justify a use case for FB-Spider.

HTML page to be modified.

1)The page needs to be prettier
2)Different color schemes can be used
3)The links of the commenters and posts would be shown.

Changes to be made in the table.

1)Unwanted elements should be removed.
2)The rows and columns must be updated with common names.
3)A separate column for showing the date,time,person of the comment/post.
4)Option of different colors for the table.
5)Image comments/posts should be displayed

Page not found error

When no page is found for a given input there is an error while traversing the search_res['data'] list as nothing is present and program stops abruptly showing error message

Styling the Output

The Output html file(Data.html) is not beautiful. It would be great if Output is properly structured and page looks beautiful

Make it Python3 compatible

Update the README.md

Updated the Instructions for the CLI Version of README.md for the updated cli support...

• add explanation of all commands and options
• add gif for each

Getting `not JSON serializable` for all requests

Additions to README.md

Wanted to add some points regarding the installation of FB-Spider.

Remove unwanted elements from the table.

A string containing a name of the page would be accepted.

The top 5 results related to the accepted 'string' would be accepted and top 5 results would be shown.
The user would be able to select from the options shown.

New columns to be added

Columns indicating date,time and the person who posted and commented.

Common terms must be used to identify rows and columns.

Adding images to the table

Creation of FB-Spider Logo

I am adding a FBSpider Logo, Please review.

i think we should have an option in starting of app where it should ask whether to display the latest comments or comments with most likes..

In this i am trying to say that if users type ''1'' then it should show the 5 comments with most like and else it should be 5 latest ones

Make a CLI version of FB-Spider

make a cli version of the project so that it can be distributed and can be used as a CLI application.

Creating a simple GUI for the same program

PyQt could be used for it.