andrewsosa / celebrity-death-tracker Goto Github PK

Turns out something is still wrong with the NLTK part

Deploying NLTK on AWS Lambda is too hard with Serverless. Let's take the NLTK
bits and deploy them as their own microservice on Now (supports Dockerfiles)
and have the NLTK bits invoked via RESTful HTTP. To speed up the process,
we'll have separate routes for:

• Extracting entities from headlines
• Checking wikipedia for a person's death

This will be done with the nameko microservice library for python.

We should automatically test services via CI and then deploy with CD.

Unit Testing

• NLTK/Detection Service
• API Endpoints

Will add more integrations later

Need to secure the POST endpoint to prevent other people from uploading things. Use http-auth express middleware.

Right now we check:

• Does headline have a 'death' keyword using array of words
• Does the wikipedia article have '[current year] deaths' category
• Does the person have [today] in their summary

This is all very 'dumb'. This needs to become a more robust system.

Currently uses a big ubuntu image. Gotta figure out how to install numpy on Alpine.

Needs to be rewritten to use Now.sh-deployed NLTK service, then deployed to run on cron via Serverless AWS.

When deployed, functions still have AWS domains assigned. Need to select a domain to alias
these endpoints.