soup-bowl / hot-this-week

🔥 Posts last.fm details to a Mastodon account.

🔥 soup-bowl's Hot This Week

An experimental bot that posts a rundown of your musical week on Mastodon and/or Twitter.

⚠️ The Twitter integration will be phased out as Twitter has taken an agressive stance against automatic tweeting. My development account has been suspended, so I'm no longer able to develop the integration. Mastodon will be the sole focus going forward.

🤔 What does this do?

This clever bot does the following:

• Phones the last.fm API 📲, exchanges pleasantries, asks how their cat is doing 🐈...
• Oh yeah, "can you tell me what has listened to this week?" 🎶
• API kindly hands over the info (or gives us a whack of the handbag if we have no API key).
• We sneakily scrape the last.fm website for the artist pictures 🤫 (better solutions welcome).
• We do some arts and crafts wizardary 🪄 to formulate a collage picture.
• Lastly, the app phones up Mastodon 📞, asks how their turtle is hanging 🐢, and posts the info and picture.

⭐ Collage is made using the power of Python using Pillow for image manipulation, Twython/Mastodon and urllib3 for API communication, and lxml for scraping the internet.

🚀 Set-up

There's no official service or method of usage yet! Watch this space 👀

🐋 Docker/Podman

docker run -v ${PWD}/config.json:/opt/app/config.json soupbowl/hot-this-week:latest

The command above uses the Dockerhub image. You can swap soupbowl out for ghcr.io/soup-bowl for GitHub container registry.

Append -h right at the end to see usage instructions. Change 'latest' for 'edge' to get the latest development version (possibly unstable).

Natively

The project depends on having Python 3 installed, and don't forget to run pip3 install -r requirements.txt to grab the project dependencies.

See the configuration example to see how to setup the tool. The following configurations are required for this to work in your own environment:

• last.fm: global LASTFM_KEY, LASTFM_SECRET, and per-user LASTFM_SCAN_USER_NAME.
  • You can register an API key here.
• Mastodon: global MASTODON_URL, MASTODON_KEY and MASTODON_SECRET.
  • Easily register an application in Preferences, then Development.
  • Permissions needed are write:media and write:statuses.
• Twitter: global TWITTER_CONSUMER_KEY, TWITTER_CONSUMER_SECRET, and per-user TWITTER_ACCESS_TOKEN, and TWITTER_ACCESS_TOKEN.
  • You can register for Twitter API keys here.
  • Ensure your access token has read and write capabilities (default is read only).

With everything set, you can just run python3 -m htw from CLI, and all the magic should happen. You can see the optional arguments by running python3 -m htw --help.

This uses pytest for Unit Testing, and pylint for linting.

hot-this-week's Issues

Investigate: Issues moving Docker build to Alpine

To improve size-efficiency I attempted to migrate the Docker image to Alpine. This worked well for AMD64, but appears to have a strange issue on ARM64. In both local tests and on GH Actions, the build takes ^10 mins to finish on ARM64. Aware that this is via QEMU, but this still appears to be incredibly slow.

The Dockerfile is:

FROM python:3-alpine

WORKDIR /opt/app

COPY htw                htw
COPY assets             assets
COPY requirements.txt   requirements.txt

RUN apk add python3-dev libc-dev zlib-dev jpeg-dev freetype-dev gcc \
	libxml2-dev libxslt-dev

RUN pip install --no-cache-dir -r requirements.txt

ENTRYPOINT [ "python", "-m", "htw" ]

Testing this on genuine ARM hardware should hopefully reveal what the true nature of the issue is.

Discogs Integration?

Discogs provides a general API for collecting artist information, which could be good as a way of genuinely collecting artist images.

Needs further investigation - so far my quick tests have given me blank artist image URLs. Perhaps they have taken the Last.fm approach of rejecting providing resources. This appears to be my inproper testing, FAQ specifies that image URLs are signed.

Main/latest container failing - Edge is fine

server:/$ /usr/bin/docker run -v <redacted>:/opt/app/config.json ghcr.io/soup-bowl/hot-this-week:latest
Processing soup-bowl
- Scraping from last.fm...
- Generating collage...
- Composing tweet...
- Posting to Twitter...
PHP Fatal error:  Uncaught Error: Class 'HotThisWeek\TwitterOAuth' not found in /opt/app/src/TwitterAPI.php:68
Stack trace:
#0 /opt/app/src/CLI.php(100): HotThisWeek\TwitterAPI->postToTwitter()
#1 /opt/app/main.php(89): HotThisWeek\CLI->main()
#2 {main}
  thrown in /opt/app/src/TwitterAPI.php on line 68

Looks like the mainline image isn't grabbing Composer dependencies. Edge container appears to be fine.

When run from Docker - Fatal error: Uncaught Intervention\Image\Exception\NotReadableException: Unable to init from given binary data

When you run the Docker image, you get hit with the following:

Attaching to lastfm-twitter_bot_1
bot_1  | - Scraping from last.fm...
bot_1  | - Generating collage...
bot_1  | 
bot_1  | Fatal error: Uncaught Intervention\Image\Exception\NotReadableException: Unable to init from given binary data. in /opt/tbot/vendor/intervention/image/src/Intervention/Image/Gd/Decoder.php:131
bot_1  | Stack trace:
bot_1  | #0 /opt/tbot/vendor/intervention/image/src/Intervention/Image/AbstractDecoder.php(82): Intervention\Image\Gd\Decoder->initFromBinary('\xFF\xD8\xFF\xE0\x00\x10JFIF\x00\x01\x01\x00\x00...')
bot_1  | #1 /opt/tbot/vendor/intervention/image/src/Intervention/Image/AbstractDecoder.php(335): Intervention\Image\AbstractDecoder->initFromUrl('https://lastfm....')
bot_1  | #2 /opt/tbot/vendor/intervention/image/src/Intervention/Image/AbstractDriver.php(66): Intervention\Image\AbstractDecoder->init('https://lastfm....')
bot_1  | #3 /opt/tbot/vendor/intervention/image/src/Intervention/Image/ImageManager.php(54): Intervention\Image\AbstractDriver->init('https://lastfm....')
bot_1  | #4 /opt/tbot/vendor/intervention/image/src/Intervention/Image/ImageManagerStatic.php(58): Intervention\Image\ImageManager->make('https://lastfm....')
bot_1  | #5 /opt/tbot/vendor/tzsk/collage/src/Con in /opt/tbot/vendor/intervention/image/src/Intervention/Image/Gd/Decoder.php on line 131
lastfm-twitter_bot_1 exited with code 255

Presumably this is coming from the fact the Collage process is fed image URLs, rather than a local system or binary collection. Might need to rejig the import handler to see why this doesn't work. When tried on a local system running Kubuntu 20.04, it works fine.

Possibly related: Intervention/image#634

Passing invalid configurations (last.fm and Twitter, consumer and app).

As of writing this ticket, unknown to author what transpires when missing or incorrect information is fed to last.fm/Twitter API. Assuming an uncaught exception.

• last.fm - no key.
• last.fm - Invalid key.
• last.fm - User specified incorrect Lfm name.
• Twitter - no application identity.
• Twitter - Invalid app ID.
• Twitter - Client auth keys are incorrect.

🐦❌ Remove Twitter

Hardly worth keep it now since I a) can't test it and b) would likely need to pay to use it, hardly likely.

Spotify Integration

Spotify is the market leading music platform, and it would make sense to - if possible - provide an option to integrate directly into the platform.

(I'm currently using Last.fm hooked into Spotify).

Hardening

This has been built in an expected format, but I'd like to take a preventative approach to try and catch some bugs that are inevitably lurking about.

• #7
• last.fm user data does not return 5 (can happen if the client is new or clears their data).
• last.fm artist has no image?

Still generate a collage if the LastFM output is less than 5

Currently, the tool will fail if a response of less than 5 comes from the LastFM API, which can happen on weeks with not much listened to.

Instead of an error, this should produce a collage with less images.

Alert - Rewriting Codebase to Python

Branch 'experimental'. Once it can achieve the same level as edge, it will be merged in.