4slimbu / limbook-api

Api for social networking app

Limbook Api

Limbook api is a minimal REST Api for creating social app like facebook or twitter. It has basic features like posts, comments and react.

Motivation

I had met quite a few people who have wanted to have their own private social network for their office or fun project. So, to help them, I had create this minimal version of the social API which they can integrate easily into their design and have the app up and running in no time. Also, they can take this as a starter and build on top of this api.

Technology Used

• Python
• Flask
• Flask-SQlalchemy
• Redis
• Redis Queue
• Postgresql
• Auth0

All python code follows PEP8 style guidelines

For documentation of API visit here:

API DOCUMENTATION

Features

• Post
• Comment
• React
• Image Manager

Upcoming Features

• Chat
• Notification
• Activity
• User Bot

System Requirements

• Python >= 3.7
• Pip >= 19.0
• redis-server

Note: may run in lower version but haven't tested.

Installation

Using virtual environment

# go to project directory and create venv
$ virtualenv venv
$ (or) python3 -m venv path_to_project/venv
# source venv from project directory
$ source venv/bin/activate

Install dependencies

$ pip install -r requirements.txt

Export secrets

# in ~/.profile add your env variables:
export SECRET_KEY='my_secret_key' #any random string
export DATABASE_URL= #db path
export REDIS_URL= # redis url
export MAIL_SERVER= # mail server host
export MAIL_PORT= # mail server port
export MAIL_USERNAME= # mail username
export MAIL_PASSWORD= # mail password
export DEMO_USER_PASSWORD= # password for demo seed user
# logout and login or
$ source ~/.profile

Run migration

# initialize and run migration
flask db init
flask db migrate
flask db upgrade

Seed demo data

flask seed run
# Security Note: seed data has some default users with password.
# [email protected]/password
# [email protected]/password
# [email protected]/password

Run Redis Server and Worker

# if you want to use redis queue you need to enable redis server and worker
# to run redis-server
redis-server

# to run worker
python worker.py

# or you can disable redis in the config: 
USE_REDIS=False

Run app

# using python
python run.py
# using flask
export FLASK_APP=limbook_api
flask run

This should bring the api up and running at:

http://localhost:5000

Test

# Note: unittest may not detect all the tests. So use PYTEST:
# Simply run pytest from the root directory.
pytest

Debugging with python interpreter

# in the command line
python

# inside python interpreter set app context
from run import app

# now you can test and try
from limbook_api.setup_db import db
from limbook_api.models import Post
post = Post(user_id="id",content="my post")
db.session.add(post)

Deployment: Heroku

• Create new app in heroku
• Add Postgresql and Redis as addons
• Connect github to the app
• Set config vars (secret_key, db url, mail credentials etc)
• Create pipeline and add app to the pipeline
• Choose auto-deploy master branch
• Make sure both web and worker dyno are running:
  heroku ps:scale web=1 worker=1

Contribution

If you want to contribute, just fork the repository and play around, create issues and submit the pull request. Help is always welcomed.

Security

If you discover any security related issues, please email [email protected] instead of using the issue tracker.

License

The scripts and documentation in this project are released under the MIT License

Author

Sudip Limbu

limbook-api's Issues

Improve file upload security

As of now, we are checking the file extension to allow only certain file type to be uploaded. We can do much better.

Should be able to react and reply to comments

• User should be able to react and reply to comments
• The reply to comments should be only one level nested

Implement own user management along with auth0

Improve friends module

Use pivot table between user model to achieve the result.

Add File or Image manager

Right now there is no image manger to handle file upload. We need to create one that has easy api to create file of different sizes and retrieve them. e.g:

Save

image = Image->add(image)
image->save(app.config.get('image_sizes'))

Retrieve

image = Image->query->get_using_id(image_id)

User should be able to send friends message

User should be able to send friend message to other users.

Use OOP approach

Right now other than model, everything is function based.
Transform the code to OOP

Need to refactor Model.py

As of now model.py has all the Models. It can be divided into smaller models for each package.

Add notifications

• Can implement third party service like Pusher
• Better, if can implement own web-socket server to handle push notification
• User should receive notification when someone interact with their post or send them message

Test not passing for postgresql database

Handle images using s3

Instead of saving images locally, saving it on s3 is a better choice

Auto generated posts, comments and reacts should have authentic id

Functions to generate post, comment and react for testing or automation purpose is generating post_id and user_id using rand_int which cannot be linked to Post or User object. If it can be linked to Post or User object, then it can be really helpful in seeding purpose.

Seeing payload.get('sub') in a lot of place.

I am seeing payload.get('sub') in a lot of places. Its time to extract it to its own method that returns the user_id and throws 401 otherwise.

Using postgresql as test db freezes test

Error while running migration

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "image" does not exist

[SQL:
CREATE TABLE post_image (
post_id INTEGER NOT NULL,
image_id INTEGER NOT NULL,
PRIMARY KEY (post_id, image_id),
FOREIGN KEY(image_id) REFERENCES image (id),
FOREIGN KEY(post_id) REFERENCES post (id) ON DELETE cascade
)

Add pagination and filter to get resource

• Right now resource is returning all items. We need to return around 10 items (the no. of items to return need to be configured from the config).
• use the query parameter to achieve this

Add scheduler

We need scheduler to do background tasks like:

• clean up posts, notifications, or chat messages
• send emails and may be run bot users (if that's included in future)

Suggestion:

• Should be two scheduler, one to run urgent tasks other to run general background tasks.

Implement Cache

• May be use redis or memcache

I cannot run all test at once in pycharm

When I right click the test directory and run tests, it doesn't find test. However, If I right click the test file and run test, it works.

Structure of the app directory can be improved

Not sure but may be:

• migrations and seeds should be inside limbook_api directory
• packages like posts, comments etc can be organized into package directory or modules

Multi-thread issue on sqlite

Post not found after making request in test.

Instance <Post at 0x7fc0ec09f090> is not bound to a Session; attribute refresh operation cannot proceed (Background on this error at: http://sqlalche.me/e/bhk3

Post should have visibility

• Post should have visibility: private, friends, public

Permission test is not explicit

If you look at the permission testing for any route, you can see that its testing permission from the token. We need to mock permission so that we can see against which permission we are testing.

Need activity tracker

For each user there should be an activity log:
e.g:
Table schema:
id, user_id, activity, created_at
Activity can be json like:
{ activity: "commented", "post_id": 1, "post_title": "my post"}

• post_title is useful in case of delete where post is unavailable.
  enum: commented, reacted, created, updated, deleted

Tests have redundant variables and setUp function

As of now tests have redundant functions and variables. A base test class can be created to organize those things.

Add version to api end point

The api end point should be:
/v1/api-endpoint
instead of just: /api-endpoint

It's better if later have to support multiple versions.

Add chat

• User should be able to send private message to other user
• There should be two copies of message for sender and receiver
• User should be able to delete their copy of message
• User should also be able to delete message they sent from receiver copy
• May integrate AI reply

Production environ uses postgres where as test environ uses sqlite

An issue occured where all tests passed using sqlite but failed on production.
Friend.is_friend == 1 in one query passed for sqlite but failed for postgres due to boolean field constraint.

To prevent such issues. Production and Test environ should be as similar as possible

Cannot register

Always returns bad request