mrphlip / lrrbot Goto Github PK
View Code? Open in Web Editor NEWLoadingReadyRun Twitch chatbot
Home Page: https://lrrbot.com/
License: Apache License 2.0
LoadingReadyRun Twitch chatbot
Home Page: https://lrrbot.com/
License: Apache License 2.0
127.0.0.1 - - [14/Sep/2014 18:04:41] "GET /stats?show=gplp HTTP/1.1" 500 -
Traceback (most recent call last):
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1836, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python3.3/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python3.3/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python3.3/site-packages/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/andreas/devel/lrrbot/www/login.py", line 31, in wrapper
return func(*args, **kwargs)
File "/home/andreas/devel/lrrbot/www/stats.py", line 58, in stats
stat['graphdata'] = [("%s (%s)" % (game['display'], game['show']), game['stats'][stat['statkey']]) for game in games if game['stats'][stat['statkey']]]
File "/home/andreas/devel/lrrbot/www/stats.py", line 58, in <listcomp>
stat['graphdata'] = [("%s (%s)" % (game['display'], game['show']), game['stats'][stat['statkey']]) for game in games if game['stats'][stat['statkey']]]
KeyError: 'show'
If it'd be possible having the current game being played at the top of the page would be nice, similar to how it's handled on the votes page.
To make sharing of steam/etc codes possible without bots sniping the codes out of the chat, we should have a page on the website that allows users to post codes, and other users to see them, but only if they're signed in to the site.
Implementation thoughts:
A way to mark individual codes as "claimed" (including a way to say "I got this") would be useful to clear out the older codes. When a new code is added, the bot should announce it in the chat (with throttle, naturally) and it should get SSEd out to anyone who has the page open, so everyone has a chance to see it.
When attempting to vote on games via the /votes page, the ajax call returns 500 Internal Server Error.
Alex changed the show tonight without changing the game (because the game wasn't on Twitch), so now we have some spurious stats for "Dark Souls" on "Let's Nope"
While you're in there, can you also fix the 4 "Unknown" show games?
Cities: Skylines
InsanZ - Retro Survival Horror
Mario Party 10
The World Ends With You
LRRbot should store chat logs from the chat channel, and then have a page which lets you view the VOD archives and show the chat as synced with the video.
Details on how to embed VODs at: https://github.com/justintv/Twitch-API/blob/master/embedding.md#videos (but replace chapter_id
for archive_id
for non-highlight videos)
To get the progress of the video, periodically call:
var player = document.getElementById("clip_embed_player_flash");
var progress = player.getVideoTime()
Can just have the video in a main panel with the full chat for the stream to the side, and just have the chat panel regularly scroll to the correct spot as long as the video is playing.
For accuracy, the chat logs should probably include information like user colour, subscriber status, active emotes, etc. But this can wait for version 2 if necessary.
The new code to poll the Twitch API should be moved to its own thread, and then pass any new subscribers found back to the main thread for processing, so that if the API call hangs (as happened today) it doesn't lock up the bot.
flask-csrf
is already installed on the server for an older project.
What I've been thinking of as the !xboxrecordthat
command, but which we'd probably give a better name... someone sees something awesome in the stream and runs the command, with a description of what happened:
!highlight Tiny helicopter of God
The bot then notes down the time, description, username, and the broadcastid of the current stream from the Twitch channel data.
Then, later, after the stream has finished (say, a cron job at about 3AM Moonbase time when hopefully no-one is streaming), it gets the list of archive videos, finds the one that matches that broadcastid, and gets the archive link, and adds the whole thing to the Highlight reel spreadsheet... noting the time as being "Slightly before x:xx:xx", calculating the time based on when the command was run vs the time the recording started.
Sub-only, maybe? And with a long throttle.
The subscription ran out and the emotes are no longer working. I think that should be addressed.
We should have a command like !next, but which pulls from the fan streamer calendar instead. Call it "!nextfan" or suchlike. Can probably wait until after the !next command gets rewritten (#18).
Can the QDB also be searched by attribution or date on the site?
While a lot of the LRL crew is pretty reliable with sticking with their schedules, sometimes we get surprise streams. And then there's Adam with his marathons. I feel an uptime tracker, callable with !uptime
would be handy in these circumstances.
Currently, the !next command works by downloading the ICAL file for the calendar from Google Calendar, parsing it, and trying to work everything out from there. Occasionally this comes into problems with extra tags for weird things that Google Calendar does that the parsing code doesn't recognise... in particular, when there's exceptions to recurring events, sometimes the bot doesn't recognise the overrides, and reports the normal recurring event even if it's been overridden.
We should instead be pulling from the proper Google Calendar API, rather than doing the work ourselves.
Documentation at: https://developers.google.com/google-apps/calendar/v3/reference/events/list
Example endpoint we care about: https://www.googleapis.com/calendar/v3/calendars/[CALENDAR ID]/events?maxResults=2&orderBy=startTime&singleEvents=true&timeMin=[NOW-1hr]&timeZone=America%2FVancouver&key=[API KEY]
Usually, the first entry in the response to this will be the event we want to respond from !next. If that first event is more than half-over (can happen for short events like Checkpoint) then we want to respond with the second entry.
If there's any astral-plane unicode characters in the chat (ie U+10000 and beyond), MySQL throws a fit when you try to insert them into the chat log table. It truncates the string at the offending character, and throws a warning (which Python turns into an exception).
6976892 stops that exception from tearing down the chat log worker thread (as happened today during GPLP, meaning we have no chat log for most of it, or all of Beej's Backlog), but the problem still happens. Presumably the same issue would affect the other tables too... if someone subscribed with a username containing these symbols (if that's even allowed) or if someone added one of the characters to one of the static responses/explain commands/etc (and it tried to store that in the audit table).
Since the bot and the webserver are running together on the same machine (and have been for some time), we should probably refactor the code so that the two projects live in the same dir (with the bulk of the code in subpackages, naturally, but the __main__
lines in the root).
Currently there's a lot of common code between the two utils.py
modules, and now with #34 having the bot import a www module... it would be nice if we could merge utils
and www.utils
, merge config
and www.secrets
... etc.
Every time a command is being added to the save the bot throws this error:
[2014-08-21 01:40:43,099] ERROR:utils:Exception in on_server_event
Traceback (most recent call last):
File "/srv/lrrbot/lrrbot/utils.py", line 238, in wrapper
return func(_args, *_kwargs)
File "lrrbot.py", line 410, in on_server_event
ret = eventproc(self, data['user'], data['param'])
File "/srv/lrrbot/lrrbot/serverevents.py", line 43, in modify_commands
commands.static.modify_commands(data)
File "/srv/lrrbot/lrrbot/commands/static.py", line 58, in modify_commands
bot.remove_command(generate_expression(storage.data["responses"]))
File "lrrbot.py", line 118, in remove_command
del self.commands[re.compile(pattern.replace(" ", r"\s+"), re.IGNORECASE)]
KeyError: <_sre.SRE_Pattern object at 0x1e600d0>
We should keep a history of all the changes to the static responses (and also the explanations and spam filters, and any future config pages we make), with names of who changed it, and allowing diffs between versions etc. Both for use as an audit log, a backup in case of accidental deletions, and just for keeping a record of old jokes.
The initial history can be populated from the lrrbot logs, everything already should be in there, albeit in a less-useful format.
Per Hosk, there should be a command the mods can use when the channel's getting particularly spammy (eg like when the stream goes down), to lock down the bot into mod-only mode for a certain amount of time, or until explicitly disabled, or something. Say:
!modonly
!modonly [DURATION]
!modonly off
or somesuch. In this mode, it completely ignores commands from non-mods... no error response, nothing. Give the chat a chance to calm down, and re-enable everything when the stream comes back up.
Perhaps a !subonly
for symmetry, but I can't see us using that one very often.
We should have a command in LRRbot for running ad-hoc polls...
I doubt we could build our own polling system that would be better than Straw Poll, so we should probably just hook into that. Have a command that you can pass the options to, and it creates the poll on strawpoll.me and posts the link in the chat. Basically just to make creating a poll more convenient for the host.
Perhaps with an option for a duration to the poll, where it gets the results and posts the winner in the chat after a certain time.
The common syntax I've seen is something like:
!poll <duration-in-seconds> <option-1>, <option-2>, <option-3>
eg:
!poll 60 Go left, Stab it until it dies, Kill him for the katana
Perhaps also a !multipoll
for "pick one or many"
Relevant strawpoll APIs:
POST http://strawpoll.me/ajax/new-poll
Post data: title=Automatic+poll+via+LRRbot&options[]=option-1&options[]=option-2&options[]=option-3&multi=false&permissive=false
Response: {"id":123456}
GET http://strawpoll.me/ajax/votes/123456
Response: {"0":"123","1":"74","2":"8"}
Obviously, this command would have to be mod-only.
The SSE processing for the /notifications
page don't seem to be working... no new notifications ever show up. Has something happened to the event server?
Something's changed with Twitch's video archive, and the archive-with-chat thing is no longer working.
Personal thought, but think another spring-cleaning of commands is needed again. I felt it was best when the lrrbot commands were used purely for informative info (!explain/!next/!codefall) but at the moment it seems that any small joke gets it's own command, which is both unnecessary and simultaneously runs the joke into the ground anyway - especially when chat is going to just say that thing at the time, why use a command? This goes double when the commands make absolutely NO sense unless you've seen the one particular stream it comes from (!officer is a prime example of this.)
Maybe I'm just a grumpy old man but it just feels that !commands should be largely for dispensing useful information without having to copy and paste...and that new ones should be added very selectively instead of just throwing every single joke in there - the best ones are well-established and liked ones that can be used with any stream, like !panic.
For reference, I think ones that really aren't needed just scanning through the command list: !adam, !base, !bawk, !cat, !fudge, !g, !jund, !officer, !powah, !weather and !russell.
While the crew tends to have their funny moments, the chat can have their own witty moments.
So I'm suggesting a sort of Quote Database for the chat. They might be accessible from the LRRBot website, or with certain commands. For example:
!addquote <INSERT FUNNY WORDS HERE> - Add a new quote to the database
!delquote <Quote Number> - (Mod Only?) Remove a quote from the database
!quote <Quote Number> - Reposts the selected quote. If no number is given, LRRBot will reply with a random quote.
!qdb - Links to the quote database.
Any thoughts? I'd love to try and help make it, but I don't really know Python well enough. 😦
Track counts of Adam being tricked into saying "The Dutch, again!"
Currently the only way to remove or modify is to restart the bot.
10:26 qrpth: !card Grapeshot
10:26 LRRbot: Did you mean: Grapeshot; Grapeshot Catapult
Whenever the stream starts, there's some amount of delay between when the bot notices that the game has changed, and when a mod notices that the show has changed and !show override
s... one typically happens before the other, and if the bot does anything to even read the game data when that happens (including !game
, !game refresh
, !deathcount
et al, or just loading a page on the website) then the bot will create an entry for that game in the wrong show. There's currently 4 games in the "Unknown" show, because they were created while the show wasn't set, and probably any number of overlap games (there's a Paper Mario under Things On My Stream, because the game got changed before the show was changed to Beej's Backlog, for instance).
Something should be done to keep these from happening... unless a game/show combination has something actually done in-chat (like a stat change, or a good/bad vote, or !game display
) then it shouldn't show up in the game lists on the webpages (/stats and /votes). Either it shouldn't actually be added to the storage until one of these things happen (ie don't have storage.find_game
actually create the game if it's missing, but rather move that logic to all the places that edit the game... maybe make a storage.save_game
function that adds it to the database if it's missing, call that from commands.stats.stat_update
et al). Alternatively, just have /stats and /votes and any other such pages in the future just not show any games that don't have any stats or votes (and maybe have some way to purge them from the storage for cleanliness). I'm leaning towards the former being the better plan.
This should also handle the case where a mod uses !game override
but has a typo, notices, and immediately !game override
s again to fix it... the game object for the typo'd name should be removed in the same way.
This is because the bot alternates between the IRC and the event socket and currently it takes 200 ms for the IRC framework to determine it has nothing to do.
Options:
Rather than having this built-from-scratch socket implementation that looks messy and from what I hear has some performance issues...
Why not use something like Twisted? Has a stellar IRC library, support for tons of other protocols, and a great built-in plugin system.
(Refer to projects such as DesertBot, the current bot project for the Desertbus chatroom)
A link to bit.ly/letsnopebingo for the Let's Nope Bingo instructions would be awesome.
We currently use tnotifier (tnotifier.net) to run the popup sub notifications. The system works pretty well, but, despite assurance that the feature is in development, it has never worked for resubs. It turns out that our viewers are really loyal (Yay!) which means that most subs are resubs and only a few actually show up on the stream. Other freely available subscriber notification systems I have found have similar problems or don't allow enough customization. I realized that LRRbot is already doing sub and resub tracking with its welcome messages and the notifications page (http://lrrbot.qrpth.eu/notifications). If LRRbot could have an api endpoint that returned all the subs in the last x minutes or maybe the last x subs, along with how many months they have subscribed for, then it should be pretty straight forward for me to make a notifier plugin for xsplit that checks the list every minute, removes duplicates, and then fires a notification for any new subscribers or resubs.
Ideally the endpoint would return a JSON file something like this:
{{"drmcgoodtimes",3},{"aussz",6},{"mrphlip",18},{"newguy",0}}
Does this make sense and do you think this is something that LRRbot could do without too much trouble? As I said, I was thinking that most of the work of actually collecting the notifications has already been done for the welcome and notifications page.
-Paul
The bird glyph on !russell makes Twitch swallow the entire message.
The chat emotes in the archive are applied after the string gets HTMLised, which means that, for example, something like:
Ha ha, he said "poop"!
is turning into:
Ha ha, he said "poop"!
which now contains ;p
, which then becomes .
We need to jump through some hoops so that the emotes are replaced before we HTMLise the string, but with the emotes still coming through as images, and not <img>
... only HTMLise the text between the replaced emotes...
The chat logs on the VODs seem to stop with some regularity. Is there a specific reason they're doing that, or just random things pop up?
If something funny or interesting happens they could type !highlight or !MarkHighlight or something. and it would mark the "uptime" of the stream the date and the stream name.
I get emails a lot saying "this funny thing happened on this show" but the only way for me to highlight it would be to watch the entire stream and try to find what they are talking about.
Not a high priority at all just wondering if its possible.
(wasnt sure how to tag it as a suggestion)
The reasons for using a JSON datafile over a proper DB is getting less and less relevant as more stuff keeps getting stuffed into there, and the data structures keep getting more complicated... and since we actually have a DB to store things into, it probably makes sense to start storing things in there instead.
It doesn't need to be done all in one hit, but it would be good to start moving things out of the JSON and into the DB, make some proper DB structures to store the game stats, etc.
00:25 < lithobraker> Hey glorious moderators: can we set up Lrrbot to include a link to the stream referenced in the result of the nextfan command?
Most of the events in the calendar have a location set to something resembling a Twitch URL.
(╯°Д°)╯︵ɥɔʇıʍʇ: justintv/Twitch-API#338
Should we change the response to accommodate the fact that we now have more information about subs, such as how many months they have been subbed for? Also I need to verify that the change didn't break the bot...
The bot gives the message but there's no mod action that actually happen. Tried restarting the bot to no avail
If we can figure out a good way to determine which show is currently live, we should make it so the game stats are grouped by show. This would allow, for instance, instead of having three different Dark Souls games and two different New Super Mario Bros U games, we'd just have one New Super Mario Bros U game with House of Stark stats and Crossing the Streams stats. We'd still have to have different game entries occasionally, like for all the various Minecraft maps from Crossing the Streams, but it would make the stats a lot cleaner, and would allow more grouping on the /stats page, so we could see just how much more Alex dies than anyone else.
Dynamically assign a throttle to !responses based on the amount of viewers in chat to make high viewer chat manageable with LRRBot on. Also on request an override to simply disable !responses for a certain amount of time.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.