devoteam-se / devolunch

Devolunch is an app that presents today's lunch using data from scraped restaurant websites.

Devolunch

Devolunch is an app that presents today's lunch menu on scraped restaurants. It's hosted on Google Cloud Platform.

The working version can be found at: https://www.malmolunch.se.

This repository consist of three services.

• Scraper
• Website
• Slack notifier

If you want to deploy it to GCP, head over to ./terraform.

Scraper

This service is using Puppeteer to scrape all the restaurants that exists in ./apps/functions/scraper/src/restaurants, and saves the output to a .json file hosted on a Google Cloud Bucket (to reduce cost).

It's hosted on a Google Cloud Functions v2 and triggered daily at 10:00 using a Google Cloud Scheduler.

Check out the scraper on how to add restaurants.

Technologies:

• Node w/ Typescript
• puppeteer
• pdf-parse
• Cloud Functions v2
• Cloud Scheduler
• Cloud Storage Buckets
• Terraform

Website

Needs Scraper to work

The website is a React app built with Typescript that presents the lunch menu of the scraped restaurants. It's hosted on Cloud Run and is served by a simple Node express app.

Technologies:

• React w/ Typescript
• Vite
• Node w/ Express
• Docker image on Artifact Registry
• Cloud Run
• Terraform

Slack notifier

Needs Scraper to work

The Slack notifier is a simple service that retrieves the data scraped by the Scraper and posts it to a Slack channel. The Terraform setup is configured to create a Google Cloud Scheduler which sends a message at 10:30 AM to the specified Slack channel.

Technologies:

• Node w/ Typescript
• Cloud Functions v2
• Cloud Scheduler
• Terraform

Contribute

Excited to work alongside you! Follow the instructions in CONTRIBUTING and code away.

TODO

• Make open source
  • Implement Terraform support
  • Add instructions on how to make a scraper
  • Move scrape call to Cloud Function
  • Add deploy on Github release
  • Block main branch from anyone outside of the org (unless default)
• Add husky/lint-staged and make sure pre-commit is triggered to run lint/tests
• Add API definition
• Add API versioning
• Change filesOverride to be read from environment instead of code
• Change scraper to run PDF parse from within browserScrapeFunction
• Fix shared npm packages for functions
• Host images in a bucket
• Add a separate service account for Cloud Run instead of using the default Compute Engine service account
• Add Cloud Function deploy step
• Serve images via the backend
• Split up terraform configurations into multiple files for readability
• Figure out a good way of handling restaurant chains

devolunch's Issues

[Bug]: Hyllie Bryggeri Bistro scrape is broken

What happened?

Hyllie bryggeri now displays their menu as an image instead of text or pdf, breaking the scraper.

The image looks like this.
There might be a way to read the text of the image using something like Tesseract

Relevant log output

No response