vinceau / project-clippi

An automation framework for Super Smash Bros. Melee. Detect an event → execute an action.

Project Clippi

An Automation Framework for Super Smash Bros. Melee

Project Clippi is your must-have Slippi toolkit for tapping into real-time game data and processing replay files. Automatically execute actions when an in-game event occurs. Automatically find and curate combos for that sick combo video. Automatically rename and reorganise your SLP files.

Everything is open source and fully-customisable. The possibilities are endless.

Screenshots

Detect events and execute actions!

Built in replay processor helps you find combos and organise your SLP replays!

Easily play and record SLP files with a single click!

Highlights

• Detect in game events and execute arbitrary actions
• Customisable profiles for easy combo curation and combo video generation
• Automatically record Dolphin gameplay footage without black frames
• One-click sort and rename your SLP files with ease
• Twitch and OBS integration

Download

Project Clippi is available on Windows, MacOS, and Linux. Check out the releases page to download the latest version.

Usage

To make combo videos using the Project Clippi, check out this guide on using the Replay Processor.

To turn SLP files into video using the Project Clippi, check out this guide on using the Playback Queue.

To take advantage of real-time game events using the Automator, you will need to connect Project Clippi to one of the following console sources:

• Slippi Dolphin (guide)
• Wii Console Relay via Slippi Launcher (guide)

Contributing

If you found Project Clippi useful and would like to give back here's how you can:

• Spread the word! The more people that use it, the more Project Clippi can improve!
• If you have ideas for new features, or would like to report a bug, please tweet at @ProjectClippi.
• If you are gifted in the way of code, you can help by adding more detectable events and more executable actions. Dive into the docs for the slp-realtime library which underpins this project and the source code for the Project Clippi front-end.
• PRs are welcome and encouraged! Do make sure to read through the development guidelines though.

Development

Start the development server using:

yarn run start

The development server includes HMR and auto-reloading so changes to both the main and renderer processes should auto-update.

To create a binary package:

yarn run dist

For more detailed instructions on development and building the project from source, check out the Build Process section.

Acknowledgements

Project Clippi wouldn't be here without the work of Jas Laferriere, the rest of the Project Slippi team, and all the Project Clippi supporters.

License

This software is released under the terms of MIT license.

project-clippi's Issues

Ability to trigger a keyboard shortcut

Love the project, it makes my life a lot easier. It would be really nice if I could trigger a keyboard shortcut with an action in the program, that would allow me to do all sorts of recording tasks within the game without having to use my keyboard. How feasible is this?

Thanks,
Doc

Deleting SLP files through File Processor should be restorable

Currently when "Delete files with no highlights" option is enabled, the deleted SLP files cannot be restored. This could be incredibly destructive if there was an issue like #63 where combos aren't correctly being matched. It could result in someone essentially losing all their SLP files.

We should use a package like trash to send files to the recycle bin/trash so that users can restore the files if they so wish.

Alternatively, we should add very strong warnings/prompts when a user enables that option.

Error when trying to scrape through replays

Getting this odd error when trying to scrape through replays or when trying to play a .json file. Only became an issue since 1.67 update of slippi desktop app.

Clippi should indicate that Twitch is not connected

After #66 users must first connect with Twitch before it can auto-clip. This is different from the previous behaviour where users just had to log in once that was it. There should be an indication or user flow that notifies the user that Twitch is not currently connected.

Similar to the OBS actions, the Twitch action should also probably show a "Connect to Twitch" button if not connected.

Another possibility is to add a checkbox/toggle for "Auto-connect to Twitch on app start". Sufficient error messaging should also appear if a clip attempt is made while Twitch is disconnected.

Ticking "Load into Dolphin" and then disabling "Find Highlights" still opens Dolphin

Ticking the box

Then unselecting the category

Still loads dolphin

Integration with OBS messed up, timing off on recording

I've just downloaded project clippi, and followed the guide in the readme about creating videos with it, but when I try to create separate videos for each clip, a lot of the time it either doesn't record the clip, or it starts recording too early, or too late, etc.

Hot module reloading doesn't work in Electron 7

I noticed that hot module reloading (HMR) doesn't work all the time anymore. HMR used to work fine and would correctly reload the app whether changes were made to the renderer thread or the main thread. Somewhere along the lines, HMR broke. I've managed to narrow down the culprit to one of these commits.

My understanding is that this issue with HMR likely stems from the fact that we are using this hacky workaround introduced in 2e4d3e0:

if (isDevelopment) {
  app.prependOnceListener("ready", startUp);
} else {
  app.on("ready", startUp);
}

As a recap, in #100 we upgraded Electron from version 4.1.1 to 7.3.2, due to vulnerabilities found in version 4. The twitch-electron-auth-provider package works with Electron as a peer dependency up to versions < 9.0.0, so we could have installed any Electron up til version 8. So why did we settle on 7.3.2? As evident in the commits, initially we had upgraded to Electron to 8.4.1, but it caused the console to be spammed with DevTools failed to parse SourceMap warning messages. At the time of writing, it seems like there is still no fix available for this, and in addition, this issue also plagues later versions as well including version 9 and version 10.

And so, we're stuck with version 7 of Electron. But version 7 has this very strange bug with Windows 10 dark mode, where the ready event doesn't fire in dev mode (yes, really) and thus not opening the app. So we use the hacky workaround as initially described here to get around opening the app in dev mode.

Which leads me to repeat that I believe this workaround is the cause of HMR not working. I'm still not sure what the best path forward in regards to this.

In conclusion: I hate Electron.

Clippi does not save name tag filter

Whenever I add a name tag filter and save the profile, my input is wiped after I switch screens and return to the combo filter. I also noticed that the name tag filter field is empty when I export the profile. I am not able to use name tag in my combo filter.

After closing settings and then returning to the combo filter.

After clicking on an input field, clicking on a button deselects the input field, but does not activate the clicked button

Specifically the Combo Filter tab. This lead to me believing the save functionality was broken, because I would only click the button once. Only after clicking it twice does it fire the Saved test profile. toast.

Loading replayCache.json does nothing

Hi I generated a combo json using default settings. Then I switch to the Playback Queue and I select load JSON and select the file. Nothing happens (that I can tell). I can select 1 or more slp files to add the queue no problem. It is just the JSON file that seems to be the issue. It might be because the file is very large... (I analyzed ~5000 slp games to create the JSON). I have attached the replayCache here:

replayCache.zip

Quitting Clippi prematurely should restore OBS filename format

If recording is in progress and Clippi is closed, the OBS filename format is not restored, likewise if the program is force quit or if it crashed.

We should add a hook which detects when the Electron app is about to be closed/quit and send a message to OBS to restore the filename.

This method won't be able to handle crashes or force quits however. We could restore the most recently backed up filename format on first boot, however that is not guaranteed to be up to date and the user may have changed it between now and the last time they ran Clippi.

Generating a combo.json file should show the total duration

This is useful for combo video recordings

Moving to trash is broken on Windows

According to this tweet there's an issue with the current method of using trash to move to the Recycle bin.

Also it appears that we don't need to rely on an external library for such functionality as it's built into Electron.

Update Electron to v9 since v4 has reached EOL

Might want to update the typescript dependency to the latest version too.

Combo filter: character color

It would be nice to be able to filter by character color too.

Stored Twitch access tokens aren't being expired

Currently we don't check for token expiry when storing a token to the store. We should store the expiry and the access scopes attached to the token, so that if either the expiry has passed or if we're expecting a different set of scopes, then we refresh/re-request the Twitch access token.

Ask to overwrite combos.json

Hey, just spent about 30 minutes letting it go through a number of slippi files, and accidentally clicked "Process Highlights" again after it finished processing. Without a prompt, the combos file was overwritten and I had lost the data from the run.

A prompt to overwrite would likely be helpful here, thank you!

Playing a clip requires waiting for the scrollbar to catch up

I'm using Slippi 2.1.1 & Clippi 1.3.7.

When loading a clip into slippi, I have to wait for the scrollbar at the bottom to go to the time when the clip starts. It doesn't display the match and the time is sped up quite a bit, but for clips later in the match it can take a while.

Clippi not detecting any highlights

So Clippi was working for a while on my computer and suddenly it has stopped detecting any highlights. I have tested using what I am almost certain are the same settings with the same replay files I have used in the past to create the already-existing highlight files (which btw the program still previews fine), but nothing shows up. It is also much quicker than before, as if skipping over the files that immediately exclude the settings like it doesn't have the right character in it, but with every replay file. I even tried just setting the combo settings to 1 hit with a minimum of 0% damage with any character and tried just this month on both combos and conversions, and it showed up with nothing. I tried reinstalling the program but it did nothing.

Only parse replays in a certain time window

I'm not sure if this is a native feature outside of maybe automation. Could there be an added option to check the age of replay files and only filter through a user-selected time range(Ex, from a month ago or feb 12 -> march 30) when parsing.

This would allow uses to do a yearly highlight reel pretty easily

Auto-generated Twitch clip links should be posted to Twitch chat

When auto-creating Twitch clips, due to the API restriction it is impossible to programmatically publish Twitch clips. This means that after a streaming session, one would have to manually click on each link on the Project Clippi Twitch integration page, and then manually check and publish the Twitch clip.

We can work around this limitation in the Twitch API by manually posting a link to the Twitch chat containing the clip. We would have to request the additional chat:read and chat:edit scopes, and the user must also be able to post links to chat.

Updating Project Clippi

Is there a command to update the application if you already have it installed on your computer or is the only solution to redownload the project?

Add Player "display name" or ConnectCode in Rename Files

I want "name" and "connect code" for "Rename files" option.
like {playerName}, {opponentName}, {playerConnectCode}, {opponentConnectCode}

I meant here.

e.g
format: {{YYYY}}{{MM}}{{DD}}-{{HH}}{{mm}}_{{playerName}}({{playerShortChar}})-vs-{{opponentName}}({{opponentShortChar}})_{{shortStage}}.slp
result: 20011122-0233_twitch.tv-oyost(Samus)-vs-Kounotori(Fox)_BF.slp

though the chars which cannot be used as filename in playerName have to be replaced a certain char...

List of tags for "Write to file" option

I'm trying to understand how the different actions work, and I'm very confused with the "write to file" action.

I've seen in one of your medium articles that are some tags such as {{date}}, {{time}} or {{clipID}}.

Can we get an extensive list of these with a little explanation? Or I can extract them from the code if you can tell me where to find them.

Thanks!

Add reset button to the Dolphin Playback path

Users are dumb.

Should probably change the description to include "This path should be the same as the Dolphin Playback path in the Slippi Desktop App".

The tutorial needs a entry to set melee ISO path

I've read the tutorial 3 times and couldn't figure out what was wrong, until I guessed that iso hasn't been set. You should add this to your readme since I can't PR to medium

Failed to delete file: for every file

When I pick "Delete files with no highlights", Clippi fails to delete anything.
It prints the "Failed to delete file:" message
Windows 10 Pro

I came across this bug in Electron, and tried all combinations of Recycling Bin properties, which didn't help.

Cannot access netplay name tags

Netplay tags aren't natively a part of the Slippi protocol but are still embedded in Slippi dolphin generated SLP files. We should be able to process them with the file processor.

Loading a file into dolphin should automatically select Melee

This will require an input field for specifying the melee iso file, potentially a hash check as well.

Renaming replays results in some having random string appended

Renaming files results in some replay filenames having _<5 random characters>.slp at the end of the filename.

This is the format string I'm using: "{{YY}}{{MM}}{{DD}}{{HH}}{{mm}}{{playerName}}{{playerShortChar}}v{{opponentName}}{{opponentShortChar}}_({{shortStage}}).slp"

Here are some of the results I get:

nick@nick-lenovo:/data/slp$ find . -name '*)_?????.slp' | head
./202201/220126_0230_Mr Plow_Falco_v_mang_Falco_(BF)_28ul1.slp
./202201/220112_2101_n3zmodgod_Yoshi_v_mang_Fox_(YS)_21cox.slp
./202201/220107_1807_mang_Fox_v_KoDoRiN_Marth_(BF)_51dpj.slp
./202201/220118_2152_mang_Fox_v_TheRealThing_Falco_(DL)_u9wji.slp
./202201/220130_1506_mang_Falco_v_Lucky_Fox_(DL)_pz91p.slp
./202201/220121_2248_mang_Fox_v_TheRealThing_Falco_(BF)_qr8ms.slp
./202201/220122_0214_mang_Fox_v_TheRealThing_Falco_(BF)_1v9t5.slp
./202201/220129_2056_Lucky_Fox_v_mang_Falco_(BF)_vk610.slp
./202201/220112_2335_mang_Fox_v_@boyo_boost_Marth_(FoD)_avvy1.slp
./202201/220120_2018_mang_Falco_v_aye or die_Fox_(BF)_m3uoq.slp

Most other files are renamed fine. I did rename ~20k replays at once so maybe that affected things.

I cant seem to set where clippi looks for dolphin

I do not have dolphin in the default location, how do i set clippi to use dolphin in a non-default location? I cant find a setting, tho it would not surprise me if one existed

Cannot read property "players" of null

I keep on getting this error

Potential issues with delay blocking

Currently delays will block the list of actions. e.g. 20 second delay before clipping, and then a 10 second delay before OBS scene changing, will cause the OBS scene change to happen roughly 30 seconds after the initial event was detected.

This might not be the preferred behaviour. You can get around this for now by ordering the actions such that the action with the shortest delay happens first, and subsequent actions would set the delay to be the cumulative delay based on previous actions. e.g. set the 10 second delay for OBS scene changing first, then after that make the Twitch clip action have a 10 second delay, which would have been around 20 seconds cumulative.

Ideally it would be possible to specify whether delaying actions should block the other actions from proceeding. If it is set to not block, we could use setTimeout to trigger the action. This however would cause issues with context sharing. For example, if you expect to have access to the {{ clipID }} context variable, but the Twitch clip is set to not block, the clip ID would not be available when the action is executed.

Fix inverted Slippi Icon

As per Fizzi's request the inverted Slippi icon is not right and needs to be fixed. Slippi's "ribbit inflator" should always be light. This is fine in dark mode but looks strange in light mode.

chokidar throws "Can't resolve 'fsevents'" warning on Windows

After the #45 PR was merged, on Windows machines it will still try to import fsevents despite not being supported. This seems to not impact the building of the binary using yarn run dist but seems to halt yarn run start.

The stack trace of the warning message is as follows:

        WARNING in ./node_modules/@vinceau/slp-realtime/node_modules/chokidar/lib/fsevents-handler.js
        Module not found: Error: Can't resolve 'fsevents' in 'D:\a\project-clippi\project-clippi\node_modules\@vinceau\slp-realtime\node_modules\chokidar\lib'
         @ ./node_modules/@vinceau/slp-realtime/node_modules/chokidar/lib/fsevents-handler.js
         @ ./node_modules/@vinceau/slp-realtime/node_modules/chokidar/index.js
         @ ./node_modules/@vinceau/slp-realtime/dist/index.es.js
         @ ./src/common/fileProcessor.ts
         @ ./node_modules/ts-loader??ref--13-0!./src/common/workers/fileProcessor.worker.ts

A workaround for the issue is to delete lines 8-12 from the node_modules/@vinceau/slp-realtime/node_modules/chokidar/lib/fsevents-handler.js.

try {
  fsevents = require('fsevents');
} catch (error) {
  if (process.env.CHOKIDAR_PRINT_FSEVENTS_REQUIRE_ERROR) console.error(error);
}

Not really sure how to fix this since the error is happening from the actual require() statement and not coming from the catch and subsequent console.error().

This issue only affects Windows.

Error on attempted playback of .json

Creating the .json based on the combo profile works fine, but when it attempts to play it gets the errors shown in the attached images.

then click OK and met with

I have the Melee ISO selected correctly and Slippi Dolphin v2.3.0 is connected to Clippi vs 1.5.3. Not sure what else to try.

Have Clippi remove black frames in videos

Clippi makes the combo JSON, Dolphin plays it back, and OBS records the playback. Makes sense, but Dolphin needs to take time to load up the stage and playback between each clip. So the recording will have black frames that need to be manually removed. FFMPEG can remove these automatically, but is too complex for most users to do without a GUI.

Resources:

• ffmpeg in.mp4 -vf mpdecimate,setpts=N/FRAME_RATE/TB -an out.mp4
  • This detects any frame that is identical to the previous frame, then removes all except one. So it would leave behind 1 black frame.
  • source
• ffmpeg -i in.mp4 -vf blackframe=0,metadata=select:key=lavfi.blackframe.pblack:value=50:function=less -vsync cfr -c:a copy out.mp4
  • This would replace a black frame with the contents of the previous frame
  • source

Combining these two would probably work, but if you looked into FFMPEG's CLI switches documentation there is probably a simpler approach that just removes all black frames entirely in one go.

This would be a nice tool to add in to the sidebar in Clippi to save people time. Ultimately a good compilation will have good editing to chop out the boring bits and reorganize the action to be in a better order, but no matter what, EVERYONE will want the black frames removed, and it would be nice to build that tool in.

Note: FFMPEG has weird licensing. You'll need to have the user opt-in to downloading it themselves. Either by telling Clippi where it is located on disk, or by clicking a button in Clippi that automatically downlaods FFMPEG into Clippi's folder (this is what ScreenToGif and many other tools do).

Ability to choose replay files to find highlights in

Maybe I have a weird setup, but it seems useful to me to be able to select just some replay files to process rather than having to process the whole directory.

My setup is:

1. Slippi is set to put replays in C:\Users\me\Documents\Slippi
2. I never want to delete a replay because I'm totally going to look through all of these replays and take notes to get better. Even though uhh I haven't done that yet.

So when I run project clippi in C:\Users\me\Documents\Slippi it always runs over all of my replays, which takes a while, and I end up with a bunch of combos I have already seen so the "Load output file into Dolphin when complete" option isn't very useful.

It would be better for me if I could manually select which replay files to process or if I could specify a time range like "only look at files modified in the last week".

Thanks for the great project! If this sounds amenable to folks I can try to put a PR together too.

Matching netplay names is not working

The Replay Processor should avoid reparsing replays

I'll often hit dpad down to mark a replay when I want to clip something, so that I can come back to it later with Clippi. Like most Slippi users I have a single folder with thousands of replay files, and every time I want to look for new clips, Clippi reprocesses all of these replays. Parsing my whole replay folder takes something like an hour. It takes significantly less time if I move not-yet-processed files into another folder and point Clippi to the new folder instead. However it's not easy to remember which clips were and weren't processed, and then you also have to move your replays back at the end if you like to keep them all in one place. Overall I think this workflow could be smoother.

Clippi should keep track of files that were previously processed with a given set of search settings, and avoid reprocessing them. This could be achieved by adding additional information (the search settings, filenames or hashes of already-processed replays) to the output file, and allowing Clippi to take in a previous output file as input to update it with new replays.

MINUS (or weird) start/end frame in JSON generated by "Button combinations"

Frequently, Clippi generates a json file with weird frame times like below.

{
  "mode": "queue",
  "replay": "",
  "isRealTimeMode": false,
  "outputOverlayFiles": true,
  "queue": [
    {
      "path": "C:\\Users\\Kounotori\\Game\\Melee\\FM-Slippi-2.0.0-Win\\Slippi\\20200714\\20200714-0921_(Fox)-vs-(Falco)_BF.slp",
      "startFrame": -123,  //??
      "endFrame": 0,  //??
      "gameStartAt": "07/14/20 9:21 am"
    },
    {
      "path": "C:\\Users\\Kounotori\\Game\\Melee\\FM-Slippi-2.0.0-Win\\Slippi\\20200714\\20200714-1424_(Puff)-vs-(Fox)_YS.slp",
      "startFrame": -123,  //??
      "endFrame": 4,  //??
      "gameStartAt": "07/14/20 2:24 pm"
    },
    {
      "path": "C:\\Users\\Kounotori\\Game\\Melee\\FM-Slippi-2.0.0-Win\\Slippi\\20200714\\20200714-1431_(Puff)-vs-(Fox)_PS.slp",
      "startFrame": -123,  //??
      "endFrame": 3,  //??
      "gameStartAt": "07/14/20 2:31 pm"
    },
    {
      "path": "C:\\Users\\Kounotori\\Game\\Melee\\FM-Slippi-2.0.0-Win\\Slippi\\20200714\\20200714-1431_(Puff)-vs-(Fox)_PS.slp",
      "startFrame": -6,  //??
      "endFrame": 834,  //??
      "gameStartAt": "07/14/20 2:31 pm"
    },
    {
      "path": "C:\\Users\\Kounotori\\Game\\Melee\\FM-Slippi-2.0.0-Win\\Slippi\\20200714\\20200714-2148_(Fox)-vs-(Sheik)_YS.slp",
      "startFrame": 1203,  //correct
      "endFrame": 2043,  //correct
      "gameStartAt": "07/14/20 9:48 pm"
    },
    ...
  ]
}

whats this frames?
I think no one inputs the button combination (I use A+Dpad-down) on that frame.

"startFrame": -123, appears most frequently.
And "endFrame": 3, and "endFrame": 4, also appears frequently.

Button combination time throttle doesn't work as expected

Currently when processing replays for button combinations, we are using throttleTime to filter out consecutive presses. This is actually incorrect since processing a file can often take less than a second, so if we throttle for say 5 seconds, we won't pick up any combos past the first one.

Using an invalid Renaming-Format

When using an invalid char in a Filename (I did it with ":") when entering a Renaming Format, there will be no on-screen error etc, but instead the progress-bar will stay at 0% with no possibility of stopping.

Allow individual file playback in Playback Queue or implement "Open in Finder" action

Currently, when loading a combos.json, the "Play" button activates the viewing of all replays in the queue. It would be helpful to be able to select/double-click a single file to view rather than having to reorder the list.

Another possibility is to allow a user to double-click/right-click one of the .slp files in the queue and open a finder window to its location in filesystem.

Issue with recording clips

Every time I go to start the automated recording with OBS, Slippi opens and before the clip can play I get an error.
"Question
Delete the existing file 'C:/Users//Appdata/Roaming/Slippi Launcher/playback/User/Dump/Frames/framedump0.avi'?"

If I press yes, the clip will continue as normal until it is over and then the error repeats on the next clip. This no longer automates the process. If I press no, the error immediately returns and does every subsequent press.

What do I do?

Cannot connect to OBS Web Socket in OBS 28.1.2

Enabling the web socket in OBS, entering in the correct info in the Clippi OBS connection page and clicking "connect" has no result.

It should prompt to download the Slippi Desktop App if it's not installed

Currently if a user tries to use Project Clippi without first having Slippi Desktop App installed, and tries to load something into (a non-existent) Dolphin, it will prompt with "Did you forget to set the Dolphin path?". This error message is not very helpful for a first time user. Instead, a user on Windows or Mac should be prompted to download/install the Slippi Desktop App if Dolphin is not present. On Linux, it should direct them to compile Dolphin.