pr0gramista / memes-api

API for scrapping common 🇵🇱 meme sites

Memes API

API for scrapping common meme sites. Written in Python using parsel and Flask. Currently supports:

• demotywatory.pl
• kwejk.pl
• mistrzowie.org
• jbzd.pl
• 9gag.com
• ifunny.co

No sites to be supported, suggest one?

API

/

Response: available sites

[
  "/kwejk",
  "/jbzd",
  "/9gag",
  "/9gagnsfw",
  "/demotywatory",
  "/mistrzowie",
  "/anonimowe",
  "/ifunnyco"
]

Then you can access them by accessing fe. /kwejk

//shortened response
{
  "memes": [
    {
      "title": "Czasy się zmieniają",
      "url": "https://kwejk.pl/obrazek/3387625/czasy-sie-zmieniaja.html",
      "view_url": "/kwejk/3387625",
      "author": {
        "name": "Torendil",
        "url": "https://kwejk.pl/uzytkownik/torendil"
      },
      "comment_count": 18,
      "content": {
        "contentType": "IMAGE",
        "url": "https://i1.kwejk.pl/k/obrazki/2019/05/lJUqdnyKqJf1Katl.jpg"
      },
      "points": 205,
      "tags": [
        {
          "name": "#obrazek",
          "url": "https://kwejk.pl/tag/obrazek"
        },
        {
          "name": "#humor",
          "url": "https://kwejk.pl/tag/humor"
        },
        {
          "name": "#mem",
          "url": "https://kwejk.pl/tag/mem"
        },
        {
          "name": "#true",
          "url": "https://kwejk.pl/tag/true"
        }
      ]
    }
  ],
  "next_page_url": "/kwejk/page/40878",
  "title": "Ministerstwo memów, zdjęć i innych śmiesznych obrazków - KWEJK.pl"
}

Development

1. Install dependencies with pipenv
2. Run development server with python main.py
3. Make your changes
4. Write and run tests with pytest in project directory
5. Format your code using black
6. If you added new packages run pipenv run pipenv_to_requirements -f
7. Make a pull request and be happy :)

Deploying Memes API

There are couple ways to deploy Memes API. For now supported options are:

• Docker image (Dockerfile)
• ZEIT Now (now.json)
• Google App Engine (app.yaml)

memes-api's Issues

JSONDecodeError occurs while trying to fetch 9gag memes

Hello,
I noticed, that an error occurs when trying to fetch content from /9gag and /9gagnsfw endpoints.
This is thrown while requesting one of endpoints written above.

Can you fix this err?
Thanks,
Matt.
PS. Kawał dobrej roboty 😄

Consider moving to Play Framework

Play Framework is maybe more suitable for Memes API than Spring.

Add next page url validation

Sometimes next page url is actually wrong for some reason. Validation could be helpful to determine whether next page url is actually what we wanted.

Kwejk is priority here.

Upgrade to Java 11

Fix 9gag tests

Page scrapper tests were removed, because new API was introduced.

Add not seen fork woman back

kwejk new site

9gag title is missing

Response contains empty title element.

Consider moving to AWS Lambda/Serverless

9gag doesn't work

Probably API was changed

Jbzd galleries

Jbzd galleries are interpreted as single image.

Add caching

5 minute cache should be ok.

Kwejk comments contains HTML escaped characters

Superkobieta bezczelnie zerżnięta z Cyanide & Happiness Show https://youtu.be/Hbv--yJhGRs?t=3m4s

Koteł wygląda na "nieco" mądrzejszego.