clarkio / ttv-chat-bot

Twitch livestream bot that can control colors for overlays from Stream Elements, play sound effects, handle custom rewards (like text-to-speech) and more!

ttv-chat-bot

Twitch livestream bot that can control colors for overlays from Stream Elements, play sound effects, handle custom rewards (like text-to-speech) and more!

Clarkio

This chatbot was built with 💙 live on stream with the programming community. Come and hang out with us over on Twitch!

https://twitch.tv/clarkio

Getting Started

1. Clone with: git clone [email protected]:clarkio/ttv-chat-bot.git

2. Goto project: cd ttv-chat-bot

3. Install node dependencies: npm install

4. If on:

   • Debian/Ubuntu run: sudo apt install mpg123
   • Windows install: cmdmp3
   • MacOS you should have afplay already available

5. Login on https://twitch.tv

6. Go to https://twitchapps.com/tmi/

   1. Click "Connect with Twitch"
   2. Click "Authorize"
   3. Copy the token value (starts with "oauth:")

7. Rename the file .env-example to .env

8. Update TTV_CLIENT_TOKEN with the token value you copied before in step 4.c

9. Update TTV_CLIENT_USERNAME to your client username (defaults to "clarkio")

10. For each overlay in streamelements (a.k.a. scenes) copy the URL to the scene and add it to the .env file as a new environment variable for each one.

    Example: mainScene=https://streamelements.com/overlay/abc/123

11. Get Streamelements JWT from your account and update STREAMELEMENTS_JWT in your .env file.

Run the Application

Please make sure you've completed all steps in the "Getting Started" section before attempting to run the app.

From Docker

• (@roberttables) Set OBS_SOCKETS_SERVER=host.docker.internal:
  • (@parithon) Note This dns name (host.docker.internal) only appears to work on Docker Desktop for Mac/Windows and not production environments nor other linux environments (Ubuntu)
  • (@roberttables) In that case then you can use the override through docker0 bridge which exists in linux environments

Windows

MacOS

Linux

From the Command Line (CLI)

1. Change to the directory of the project ttv-chat-bot if you have not already done so.

2. Run npm start

3. Open your browser and go to http://localhost:1337/scenes?sceneName=<your-scene-name-from-.env>

   Example: http://localhost:1337/scenes?sceneName=mainScene

From VS Code

1. Go to the debugger view and confirm the debugger is set to "Server Start"

2. Press the "Start Debugging" button (the green play button)

3. Open your browser and go to http://localhost:1337/scenes?sceneName=<your-scene-name-from-.env>

   Example: http://localhost:1337/scenes?sceneName=mainScene

Trying Things Out

Please make sure you've completed all steps in the "Getting Started" and "Run the Application" sections before attempting to try out the app.

1. Open up a new browser tab or window and navigate to the chat for the Twitch channel you want to use (defaults to "clarkio" in the .env file under TTV_CHANNELS)

   Example: https://www.twitch.tv/popout/clarkio/chat

   If you'd like to have the client connect to another channel you can add it to the TTV_CHANNELS environment variable. The variable is comma-delimited so you can have it as clarkio,"your channel name" to connect to multiple channels

2. Enter the following chat message "!bulb go green" and you should see your overlay change color in your other tab/window which loaded http://localhost:1337/scenes?sceneName=mainScene

   WARNING: the implementation is currently based off of an overlay being blue by default and will alter the hue from that. If your overlay default color is different it will not exactly change to the color you may be intending. We are working on making this better to support different default overlay colors.

Twitch Viewers that have helped contribute to the project (in no particular order):

Want to contribute? Check out our Code of Conduct and Contributing docs. This project follows the all-contributors specification. Contributions of any kind welcome!

Thanks goes to these wonderful people (emoji key):

• sockelo
• frenck
• tallpants
• wwsean08
• l2ival
• ikoakmaindehoas
• codephobia
• styler
• TheMartesLive
• eeevans
• TheMichaelJolley
• @jaredpsimpson

Contributors

Thanks goes to these wonderful people (emoji key):

Jake Nylund 💻	Michael Jolley 💻	CodemanCodes 💻	jaredpsimpson 💻	Edward Evans 💻	Nicholas March 💻	PatPat1567 🐛
Timmy Kokke 💻	Michael duBois 💻	Benjamin Lannon 💻	Jason Allan 💻	Chris Gargotta 💻 🤔	Wingysam 🤔	LuckyNoS7evin 💻
Sean Smith 💻

This project follows the all-contributors specification. Contributions of any kind welcome!

ttv-chat-bot's Issues

Smack21 and Highfive21

We can have a !command like smack21 and it will start smack for 21 times and we can do same with highfive21 and you can add that smack voice (stream)

After writing the command there will be a series of 21 smacks or 21 highfives this could be so fun

Only play viewer join sounds when they send an initial message

To avoid putting people on the spot for joining the stream quietly update the bot to only play their join sound when they send their first message for that stream session

Overlay Color Cycle: Allow Color Override for 30 seconds like Bulb

The Bulb allows a pulse effect or light bulb direct color change to stop the cycle, apply the change to the bulb, let that change stay for 30 seconds and then restart the color cycle again.

Do the same for the overlay color cycle.

Keep a record of how many times each sound effect is played

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

use some kind of database to keep track of the count
send a message when a user activates a sound effect that's never been played before
send a message when a user activates a sound effect and hits a certain count milestone (such as played 100 times)

Overlay-Colors: Create color wheel to adjust overlay colors

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

A visual color wheel that users can click and drag a point within to update the css filter hue-rotate degree value found at `localhost/overlay-colors

Currently, users can enter an exact value or use the "up" "down" buttons to change the degree value. See current UI below:

Refactor: Twitch-Chat class to pass in twitch configuration options on instantiation

Currently, configuration options are set up directly in the class but the caller or creator that's using the class should really pass in the options it wants to be used within the TwitchChat class.

Right now the TwitchChat constructor uses an internal function that gets the configuration options to load. We need to essentially move that code into index.ts to pass in the configuration options object from there when it is new'd up.

Screenshots to help explain:

Current constructor in TwitchChat.ts:

Current function gathering options in TwitchChat.ts:

UI - Better Bulb User Control and Information

Moved over from clarkio/live-stream#5 by @LasseWormark

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

Viewer color control

1. Implement a queue system, and a timer for each light change request, so if multiple people write the command each person get the requested color for ex. 5 seconds.

2. in the HUD / Overlay add a "light controlled by : username" in the time span the light is controlled by a user, and then give points towards how long time the light is controlled to give some viewer credit "only an idea" 💃

Add Timeout Between Bulb Commands

As per the suggestion from @DougTheCoderSA add a configurable (suggested by @lannonbr) setting to implement a timeout between bulb commands.

Overlay-Colors: Store color selections as css to be reused in scenes

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Ease Twitch Chat onboarding

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

As a user I would like to ease the onboarding process by not having each user sign up for a twitch App & Auth account to get a twitch key.

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

If able to login through your twitch account and again the appropriate credentials this would be ideal.

---

Thanks! We'll be in touch soon.

Create a module for loading environment variables

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Refactor: move discord code into its own file

Audio tracks are parsed in parallel for duration

If a lot files are parsed in parallel, this may cause a seriously slow down the system.

In the code the following comment can be found: (Array.forEach is blocking aka synchronous)

https://github.com/clarkio/ttv-chat-light/blob/bbfecb5725f5be39a95805c3c38215c7db26c1cf/src/server/sound-fx.ts#L101-L102

This is not the case, Array.forEach is not blocking.

A for loop is blocking in combination with await. However a for loop would not directly make a difference neither because the expensive task executed in that loop is mp3Duration , which is an asynchronous (non blocking), not returning a promise.

Using await in combination with a for loop is a way to make it synchronous: PR #98.

Update README so others can easily get started and contribute

Custom Epic User Intro Sounds

Custom Intro Sounds

• Specific Media Folder
• FileName = Twitch Name
• Detect first text in chat
• Plays matching sound
• Some limit to sound length (slightly longer than !shock, e.g. 2 sec)
• User has to submit via PR

Stretch goals

• Command to trigger after
  • Only by user !myintro
• Outros also
  • {name}_intro.wav and {name}_outro.wav
  • !myintro !myoutro
• Admin/Mod cmds
  • !user_{name}_{intro/outro}

Refactor: move bot code into its own file

Add Hidden Morse Code Messages via Light

As suggested by @TripleDiamonds. We need to think this one out a bit more as to how it could be implemented.

Maybe use preloaded messages to avoid people abusing it with inappropriate messages.

Testing project automation

chat light doesn't scale

Please provide us with the following information:

When a raid from TimTheTatman comes in the bot kills the computer. We need to scale this up to the roof!

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Get a raid from TimTheTatman or someone else that's awesome

Any log messages given by the failure

Expected/desired behavior

Clarkio's computer does not become a pile of slag from all the heat given off by his CPU

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Join sounds are played more than once

When a user joins for the first time during a stream session the sound plays but the bot will reconnect to the chat room which leads to it thinking the viewer joined back again. When this happens the bot plays their join sound each time.

Refactor: move overlay supported canned colors to environment/config variable

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Split reading components to separate functional parts being overlay and bulb

If I understood it correctly during stream, currently there is only one local chat reading component that would then trigger both the overlay color changing and bulb changing part. Both are different components who just happen to have the same input, being the command in the chat.

Adding a chat reading component in Azure would separate the whole bulb part from the overlay part. Both would read the twitch chat as input, thus separating the functional concerns. You could trigger both local and online services without depending on your local service to be running. In the Azure reading component you could build in a safe guard to trigger only when streaming.

So you would want 2 chat readers, one running on Azure and another running locally. Those would spit out events that you can hook into. On the local reader you can hook in the overlay change by subscribing to the chat command event. On the online reader you can hook in the bot-sending by subscribing to the same chat event.
That way the overlays and the light bulb are completely separated and won't influence each other.

Another option would be to read the input from same online event and trigger on that, instead of having a local reader at all.

Add all contributors to readme

@all-contributors please add @jakegny for code
@all-contributors please add @MichaelCduBois for code
@all-contributors please add @michaeljolley for code
@all-contributors please add @sorskoot for code
@all-contributors please add @ElectricHavoc for code
@all-contributors please add @lannonbr for code
@all-contributors please add @CodemanCodes for code
@all-contributors please add @jaredpsimpson for code
@all-contributors please add @eeevans for code
@all-contributors please add @nmarch213 for code

Special Effects: break down by effect type (e.g. blink) and colors associated to the effect.

Allow the user to save these to a JSON file that can be referenced later at runtime/streamtime

Overlay Color Change: Improve the Synchronization between bulb and overlay color

Overlay color cycle and bulb color cycle don't really stay in sync. Improve this with maths!

Create GitHub ISSUE template

Sometimes the overlay color will not stay to a new color after changing

I suspect this occurs after some special effect/event like when someone follows or subscribes and the overlay cycles between.

More research is needed.

LIFX - Post in chat when light change is successful / failure

Moved from clarkio/live-stream#13 by @lannonbr

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

It would be a good indicator in addition to the visual change to have the stream bot post a message saying that it did changed the bulb color or if there was an issue

Keep a record of how many times the !bulb is used/colors

Similar to #91

Add an .editorconfig to simplify local setup for any devs

Set up overlay to cycle through all colors over time

As suggested by Mikee_k on stream.

Convert to TypeScript

Create GitHub PR template

Update AI model and make AI determine effects

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

This came out of the stream tonight that we need to update the AI to handle more effects (like tips, hosts, raids etc).

Also allow the chat client to send the AI a message like trigger pulse green purple to cause it to pulse green/purple or other similar lifx supported commands

!bulb Black hides overlay details

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

!bulb black

Any log messages given by the failure

N/A

Expected/desired behavior

When overlay color changes to black, you lose the recent follower box and the branding elements. I would expect those would be colored white or another color.

OS and Version?

N/A

Versions

Latest version

Mention any other details that might be useful

https://clips.twitch.tv/BrightEsteemedClintPastaThat

We need a !humpday sound clip

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

Refactor: handle special effects dynamically

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

• Would like to have a way to easily define special effects without having to change the code. Currently, effects are hardcoded names and colors. Not everyone will want to use the same as what we are using today.

Add Text-to-Speech capabilities

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Mention any other details that might be useful

It's been on my mind and very much inspired by similar features in @NinjaBunny9000 and @instafluff's streams where chat can use a command to audibly say a message on stream.

Example: !tts hello everyone

Squirrel command to track how many times stream gets off track

!squirrel command to keep track of how many times the stream gets off track.

Just a fun command, since we love Clarkio so much and the tangents are just as fun as the coding in the stream!

As part of this, maybe a squirrel animation of a squirrel running around the screen

Need pacman visual for when !wakawaka is played

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

Need pacman visual that runs across the screen for when !wakawaka is played

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.

give the ability to upload or give a URL link to their overlay

• URL (just URL for now, can add ability for a local file at another time)

Add Timer to Bulb Commands

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

!bulb setTimer HH:MM:SS
!bulb setTimer MM:SS

Expected/desired behavior

Scenario 1: HH:MM:SS
Given that Clarkio is streaming on TwitchTV
And ttv-chat-light service is active
When a user has triggered a !bulb setTimer command
And the user has passed a parameter in the format of HH:MM:SS
Then ttv-chat-light shall parse the parameter
And ttv-chat-light shall create a timer with the duration provided by the user

Scenario 2: MM:SS
Given that Clarkio is streaming on TwitchTV
And ttv-chat-light service is active
When a user has triggered a !bulb setTimer command
And the user has passed a parameter in the format of MM:SS
Then ttv-chat-light shall parse the parameter
And ttv-chat-light shall create a timer with the duration provided by the user

Scenario 3: Bad Input
Given that Clarkio is streaming on TwitchTV
And the ttv-chat-light server is active
When a user has triggered a !bulb setTimer command
And the user has passed a parameter that DOES NOT meet either format of HH:MM:SS or MM:SS
Then ttv-chat-light shall ignore the parameter
And ttv-chat-light shall communicate with the Twitch TV chat function
And inform the user that an invalid parameter has been passed

Mention any other details that might be useful

This has been created and documented by Yegnaro from Twitch.

Figure out where this log message comes from

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

This is hard to reproduce as I think it may be showing up from an internal module or third-party module but every now and then at start up we'll see this log message in the output

INFO: Could not find files for the given pattern(s).

Any log messages given by the failure

INFO: Could not find files for the given pattern(s).

Expected/desired behavior

Don't show this message?

OS and Version?

Windows 10 and macOS

Versions

0.1.0

Determine more reliable way to capture alerts/events

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Expected/desired behavior

• Instead of relying on a bot in chat to capture alerts/events find a better way to get those directly or via another medium. As the number of chatters can increase this will also increase the processing time on the client for each message to be parsed.

Create a module to store constants

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)
• refactoring

Mention any other details that might be useful

Create a module to store strings and commonly used values. For instance, put the effect strings found in azure-bot.ts (lines 28-33) in an external file.

Feature: Alias to Color

It would be nice to alias certain strings to colors.

e.g.
ElectricHavoc > "Gold" / "#ffd700"
RobertTables > "RebeccaPurple" / "#663399"

Etc...

Stretch Goal.... flashes color on Join. 😁

Implement an HTML templating engine to remove reduncancies

Refactor: move chat related code into its own file

Implement ability to enable/disable AI bot component

Play !byefelicia whenever a user leaves the stream

Please provide us with the following information:

This issue is for a: (mark with an x)

• bug report -> please search issues before submitting
• feature request
• documentation issue or request
• regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Any log messages given by the failure

Expected/desired behavior

Play 'byefelicia' when a user leaves the stream and possibly have the ability for custom sounds for specific users similar to when users join.

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Versions

Mention any other details that might be useful

---

Thanks! We'll be in touch soon.