shpigford / plylst Goto Github PK

The GetMoreTracksWorker worker currently makes a ton of Spotify API calls which, traditionally, have a habit of throwing rate-limit errors.

Problem there is it forces the entire worker to get re-queued to run again, essentially creating an endless loop of calls.

That's problematic.

Each block of calls need to be broken out in to separate workers so they don't cause issues.

Basic functionality needs to be added for deleting a playlist you've created in PLYLST.

Unfortunately the playlists in Spotify can't be deleted (there's no API endpoint) but we do need to allow deleting a playlist in PLYLST in the event the user doesn't want them to keep being regenerated in their Spotify account.

after new account is created a I click the button create new playlist
and this error appeared

Some artist doesn't get added when use the rule Artist Name / contains

For the purposes of helping folks get up and running, create some one-click starter templates folks can use after they sign up.

Spotify has no webhooks or even the ability to sort saved tracks by add date. So you have to basically run the full "import" process to add those tracks.

https://developer.spotify.com/documentation/web-api/reference/library/get-users-saved-tracks/

Right now the easiest thing is probably to run ProcessAccountWorker daily.

Is it not possible to create a playlist with many different artists as a rule?

Add an option to ignore duplicate tracks in playlists. Most efficient way to do this is likely just by track name.

Love the idea so far but I'm running into an issue where each of my playlists are broken in the app. Not sure if there are other factors, but it seems to happen consistently when I create a new playlist with:

• a genre
• a play count
• a play count filter of "greater than"

Now I have three playlists that all return "We're sorry, but something went wrong." when I try to access them. Maybe this also happens for other rulesets but I haven't tested them all.

I've saw that some rules and sorting are only applied if Songs I've liked is selected.
While it absolutely make sense for the Days Ago (How many days ago was the song added?) rule, there are some more rules which also could be applied if one of the other catalogs are selected (Play count and Last played). There're also some sorting which also could be used with other catalogs than Songs I've liked.

The main reason is that the play statistics are held in the follows table. The follows table only contains a document when the user have liked the song.
I saw that there is another table streams which holds all plays of a song for a user. Maybe we could refactor the rules mentioned above to use the streams table instead of follows.
What do you think @Shpigford?

Just trying to figure out how genres work, as some of my playlists return zero tracks. If I'm missing something obvious, lmk 😁

Poking around, I think I understand that genres match to the artist listing, not to the track or album itself. So when some genres return zero results, I tried checking the spotify api to make sure those genres existed for the artist.

I've also double-checked that I have several albums for the following artists saved in my Spotify library.

Example 1 — works ✅

Artist: Mutemath (artist id: 1J9Z9bhqkgyqseEi85cbyx)
Spotify API genres:

"genres": [
    "dreamo",
    "modern rock"
  ],

PLYLST results: playlists each with 'dreamo' or 'modern rock' return the full artists collection

Example 2 — no results 😵

Artist: Jonathan Coulton (artist id: 292sg99iIOc93zcd30r4Oz)
Spotify API genres:

"genres": [
    "comic",
    "pop rock"
  ],

PLYLST results: any playlist each with 'comic' or even 'pop rock' returns zero results.

Example 3 — no results 😵

Artist: Tycho (artist id: 5oOhM2DFWab8XhSdQiITry)
Spotify API genres:

"genres": [
    "chillwave",
    "downtempo",
    "intelligent dance music"
  ],

PLYLST results: zero results for all three genres

@Shpigford mentioned in a previous issue that pltlst supports pipe-separated values.

After some experimentation, I've found this only works (using artists) with the contains modifier option (not is). Which works fine for most values but not all. E.g. Having a value of Sia results in lots of false hits, such as Staasia Daniels and Artemisia. Similarly Air and even Lorde have lots of false hits (which "breaks" the playlist effectively). Note, adding spaces around the term does not work (| Air |).

One possible fix for this is to support the pipe syntax with the is modifier (currently it returns zero results for even simple pipe queries), assuming it's possible via the Spotify API.

Alternatively some sort of additional syntax could also overcome it (word boundaries or startswith or endswith chars - ^ & $), etc.

I've got a bunch of playlists already in my Spotify. Is it possible to scan those for common genres, then offer those as quick selections next to the genre picker?

Right now the rule builder defaults to AND, meaning tracks have to match every rule.

The app should support OR as well. I believe the rule building javascript library (queryBuilder.js) supports this from a UI perspective, but the actual track filtering method would need some adjustments.

https://github.com/Shpigford/plylst/blob/master/app/models/playlist.rb#L15

I just noticed that all of my generated playlists are public and need to be manually changed to private. It would be helpful to either have a toggle for private when creating. Or maybe a global default setting so all future playlists can be private unless changed? Might be cool.

When searching the Genius API for song/artist, right now we just snag the first result and do nothing to verify if it's actually a match.

https://github.com/Shpigford/plylst/blob/master/app/workers/get_lyrics_worker.rb#L16-L17

We should check that result and make sure the song name and artist name do in fact match. If it doesn't, we should check the next result, and so on.

At the moment, if we try to load all the songs for a given playlist, it will generally time out when there are more than ~500 songs.

So, there's are a hard limit of 250 songs within the app (though actual playlist generation in Spotify still works as it should).

We should progressively load tracks on the /playlists/show pages to address this.

This is one feature I use in use iTunes a lot.

Plylst doesn't appear to allow this, although I'm happy to be proven wrong.

#42 is one possible approach. Another is to allow some sort of simplified regex option so artists can be pipe separated in a single rule:

artist -- contains -- artist x|artist y|artist z

Thanks for considering.

For example. I want tracks that are:

Danceability: Very
OR
Danceability: Super

This would also mean adding support for Operators (and / or) for each variable. (#7)

There's a popularity attribute that tracks have in Spotify and it changes over time.

Need to create a worker that periodically updates tracks to refresh that since it's used in the playlist builder.

I'm guessing a popularity_last_checked_at field and then a worker that grabs tracks to update every 7 days (at least as a starting point).

Could potentially spread it out, though. If a track was released 10 years ago, its popularity likely won't change much, so could reduce some API calls be spreading that out.

Hey, thanks a lot for your great work on this project! 🙂
I'm trying to create a playlist with all the songs I have listened to at least 3 times, but they are not in my library.
I haven't found a filter for "in library". Am I missing something or is it not implemented yet?

Spotify has recently changed the behavior of how liking an album works. A user can now save an album to their library without adding every song. This can create an issue where there's no criteria to filter based on all albums in your library.

If possible, this could be added as another filter criteria.

Currently, PLYLST auto-updates playlists in Spotify every couple of hours, adding/removing tracks as they meet the rules of the playlist set in PLYLST.

However, there are use cases for not auto-updating playlists after the fact. An example is seeding a playlist with the initial rules, but then manually curating it afterwards.

At first pass, adding a boolean option to the playlist create/edit form would likely do the trick here.

Worth noting is that updating/saving the playlist on PLYLST would still update the the tracks on Spotify.

Hello.

I've started to use Plylst few days ago and I'm in love in it's idea. Thank you so much for this app.
But I faced with a huge problem. I think playlists work wrong for me. Eg I created playlist with following rules:

1. Energy is Insane, Danceability is Super.
2. Uses only songs I've liked.
3. Sorted randomly.

And that what's I got. This is definately not a dance track.

Same story with playlist that should sort tracks which has not played in past 90 days (https://plylst.app/playlists/n6z7ghzq38). This playlist contain following rules:

1. Last Played Days Ago is greater than 90.
2. Uses only songs I've liked.
3. Sorted randomly.

Okay, 2nd and 3rd rules works fine. It's all my liked tracks but all of them has been played few days ago.

Spotify has a "popularity" attribute on tracks and artists.

Users should be able to set a filter to only pick songs with popularity less than, greater than or between X values (0-100).

Build a playlist based on artists similar to a given artist.

An option to include only one artist per playlist. This way you discover more artists and avoids duplicated tracks.

Thanks for this great playlist tool.

Use the danceability metric from the audio features as a "greater than / less than"

Use case:

Many Spotify users don't use the Save to Library feature.

I instead used the "Star" feature for years before Spotify removed it. I have instead just kept using the Starred playlist and add tracks to that playlist instead.

I'd like to be able to use my Starred playlist as my "library" for building smart playlists out of.

SaveTracksWorker is a highly used process (5.5k/hr) using alot of bandwidth (9.7s/job). I think uniqueness constraints in rails are biting us here.

Here is the beginning of the block in question that lasts ~10s for most jobs. This begins a transaction (3s), attempts to create a follow (3s), then rolls back in 97% of cases (3s).

I think the rollback is occurring because of the uniqueness constraint. Rather than offloading that uniqueness to postgres, rails does many queries to validate it in app land, but doesn't actually delegate that uniqueness to the database (source).

The actual insert, when it does succeed, takes 800ms. I'd be interested to see what performance looks like when relying on a postgres uniqueness contraint vs. the rails constraint.

This will only get more ridiculous as more rules are added. We really need a "rule builder" where you add rules one at a time instead of seeing the whole dang form.

Will require a decent amount of javascript and that's, unfortunately, not my strong suit. Would love some help on this!

Hello!
The first case:
I've created a playlist https://plylst.app/playlists/v63oqhe0v6
There must be tracks which I liked and played more than 1 time.
However, it says "No songs match your filters." I've been using plylist about a month or even more, so I'm sure there is a whole bunch of tracks which I've listened to 2+ times during this period.
Also, I have a Last.fm account connected to Spotify and it kinda proves my points. My stats for March (and that's even less than a month for now!): [deleted].
So, it seems like something is definitely wrong with play counts.

The second case:
I've changed that playlist so it's based on the "Any songs from artists I've liked" option and I've got really weird results.
https://plylst.app/playlists/48yp7hpy28
The resulting playlist contains 1000 tracks of just three artists (Bonobo, Sigur Ros, The Shins) although I'm not subscribed to any of them. And, of course, I haven't listened to any of these tracks more than once since I've been using Plylst, I'm pretty sure about that.
Moreover, tracks got duplicated there for some reason.
[deleted]
Sort by song title and you'll see.

Certain attributes are ultimately numeric (at least with how they're stored) and as such we can use lt/gt/between operators on them.

• Key
• Danceability
• Acoustic Likeliness
• Energy

Changing them in the rule builder is straight forward. The bigger change will be in the Playlist model for the actual SQL queries.

If you remove a Saved Track, PLYLST currently doesn't have a way to know that.

First thought on how to handle this is to hit the /tracks/contain endpoint with an array of song IDs (max is 50), and see if they're still in there.

If not, set the "follow" record's active column to false.

https://developer.spotify.com/documentation/web-api/reference/library/check-users-saved-tracks/

Could be run daily (or as often as the Spotify rate limits allow).

I'd like to create a playlist sorted by the most recently played songs but this doesn't appear to be an option right now. Seems to be possible based on the initial "latest streams" but I'm not sure how to recreate it.

Not sure if it'd be helpful for anyone else, but I know that when a PLYLST is deleted or even renamed the playlist still exists in Spotify. When renaming lists with slightly different names, it may be confusing to know exactly which one is untracked and can be manually removed.

It might be helpful to add a [old] or 😵 or some other kind of string in the name to designate that the playlist is no longer connected to the app. That way the user will know they can remove or rename as they need and PLYLST won't try to regenerate it anymore.

Right now we generate a list of genres for the user to pick from based on their current collection augmented with the most popular genres across the user base.

However, it can be really tedious to select all of those genres as they are hyper-specific (there are over 4,500 genres).

So if you're trying to make a "jazz" playlist, there's actually not a "jazz" genre...there are dozens of _sub-_genres like indie jazz, jazztronic, jazz rap, nu jazz, uk contemporary jazz, and on and on.

The query builder operator is currently in, and you're able to select as many genres as you'd like. What we need to do is add a contains operator which then changes the input type to string and let the user just type in the string they want to match on.

Places that will need modification:

• application.js (~L240)
• playlist.rb (~L127)

QueryBuilder

The query builder uses QueryBuilder. Where it's currently problematic (and where I keep getting stuck) is on how to change the input after changing the operator. Currently it makes use of the select picker plugin so you can easily filter the list of genres, but if you change the operator to contains it should no longer use that plugin or a dropdown and instead just have a plain text input field.

Bounty

As a test on offering payment for higher-priority items, I'm offering a payment of $200 for completing this.

Deliverable: PR that successfully modifies the genre picker to making use of the "contains" operator which changes the "input" from a "select" to "text". You don't have to handle the backend code part (in playlist.rb), though you're welcome to.

First person to submit a verified functional PR gets the bounty.

Payout will be made via PayPal, Venmo, CashApp or as a donation to a charity/non-profit of your choosing.

It's pretty easy to get up and running locally using your own Spotify account, but it's nearly impossible to test performance issues with just one account.

I need to figure out some way to create a similarly sized data set to production for local testing.

Seems pulling down the current dataset and then just anonymizing all the user accounts would be sufficient. And then I could make that dataset available somehow for others to use?

I admittedly have no clue how to go about that, though. So, would love some input/tips there.

Required for #6

Allow users to share their playlists in a public directory for others to copy.

Hello.

I tried to create smart playlist based on my most played tracks and only one rule that I found is: play count greater than.
Main problem of this rule is that you always need to edit count of plays. Much better to create special rule which will be include most played songs in amount of ... (set it too by yourself). Just set it up once and forget about edits.

Maybe I'm thinking about this wrong, but after creating a playlist with a specific genre I'd expect for all of the songs from an artist of that genre to be included. Instead, only a subset of the artist's songs are included (a few from each album).

From what I could tell, genres are applied to the artist and not the tracks themselves. How do I get a genre-based playlist to include an entire artist (or even album) from that genre?

As a user I often don't know exactly which genres all of my music fits into, or what exactly it's called. It might be helpful to get an autocompleted list of the available genres as you type. Maybe even more than just a forward search, so searches like "rock" could also suggest all the crazy sub-genres of rock?

For work and studying I prefer to have songs without lyrics and a filter for that would be nice.

I've used plylst sparingly, just checking in every once in a while to see how the project is progressing. Something that always gets me stuck is having to a name a playlist before I've had a chance to explore the options for creating a playlist.

Since the playlist name is editable from the second screen, maybe we can just remove the first screen and give it a preset name like "Untitled playlist" or something? That'd get folks quicker to the playlist builder screen.

For use in both the PLYLST UI and in the Spotify playlist descriptions, the app should generate descriptive text of what rules were used to make the playlist.