mrphlip / lrrbot Goto Github PK

If it'd be possible having the current game being played at the top of the page would be nice, similar to how it's handled on the votes page.

To make sharing of steam/etc codes possible without bots sniping the codes out of the chat, we should have a page on the website that allows users to post codes, and other users to see them, but only if they're signed in to the site.

Implementation thoughts:
A way to mark individual codes as "claimed" (including a way to say "I got this") would be useful to clear out the older codes. When a new code is added, the bot should announce it in the chat (with throttle, naturally) and it should get SSEd out to anyone who has the page open, so everyone has a chance to see it.

When attempting to vote on games via the /votes page, the ajax call returns 500 Internal Server Error.

Alex changed the show tonight without changing the game (because the game wasn't on Twitch), so now we have some spurious stats for "Dark Souls" on "Let's Nope"

While you're in there, can you also fix the 4 "Unknown" show games?

Cities: Skylines
InsanZ - Retro Survival Horror
Mario Party 10
The World Ends With You

LRRbot should store chat logs from the chat channel, and then have a page which lets you view the VOD archives and show the chat as synced with the video.

Details on how to embed VODs at: https://github.com/justintv/Twitch-API/blob/master/embedding.md#videos (but replace chapter_id for archive_id for non-highlight videos)

To get the progress of the video, periodically call:

var player = document.getElementById("clip_embed_player_flash");
var progress = player.getVideoTime()

Can just have the video in a main panel with the full chat for the stream to the side, and just have the chat panel regularly scroll to the correct spot as long as the video is playing.

For accuracy, the chat logs should probably include information like user colour, subscriber status, active emotes, etc. But this can wait for version 2 if necessary.

The new code to poll the Twitch API should be moved to its own thread, and then pass any new subscribers found back to the main thread for processing, so that if the API call hangs (as happened today) it doesn't lock up the bot.

flask-csrf is already installed on the server for an older project.

What I've been thinking of as the !xboxrecordthat command, but which we'd probably give a better name... someone sees something awesome in the stream and runs the command, with a description of what happened:

!highlight Tiny helicopter of God

The bot then notes down the time, description, username, and the broadcastid of the current stream from the Twitch channel data.

Then, later, after the stream has finished (say, a cron job at about 3AM Moonbase time when hopefully no-one is streaming), it gets the list of archive videos, finds the one that matches that broadcastid, and gets the archive link, and adds the whole thing to the Highlight reel spreadsheet... noting the time as being "Slightly before x:xx:xx", calculating the time based on when the command was run vs the time the recording started.

Sub-only, maybe? And with a long throttle.

The subscription ran out and the emotes are no longer working. I think that should be addressed.

1. MySQL is toilets: 0de144d c606bbd
2. MySQL as it's currently configured on the server can take up to 150M of memory and I have run out of ideas to lower it. On my laptop I have a default PostgreSQL install with a couple of weeks old LRRbot data dump and I can't make it use more than 60M of memory. This is test actually in MySQL's favor as the PostgreSQL server has about 20 active connections each taking about 1.2M of memory.

We should have a command like !next, but which pulls from the fan streamer calendar instead. Call it "!nextfan" or suchlike. Can probably wait until after the !next command gets rewritten (#18).

Can the QDB also be searched by attribution or date on the site?

While a lot of the LRL crew is pretty reliable with sticking with their schedules, sometimes we get surprise streams. And then there's Adam with his marathons. I feel an uptime tracker, callable with !uptime would be handy in these circumstances.

Currently, the !next command works by downloading the ICAL file for the calendar from Google Calendar, parsing it, and trying to work everything out from there. Occasionally this comes into problems with extra tags for weird things that Google Calendar does that the parsing code doesn't recognise... in particular, when there's exceptions to recurring events, sometimes the bot doesn't recognise the overrides, and reports the normal recurring event even if it's been overridden.

We should instead be pulling from the proper Google Calendar API, rather than doing the work ourselves.
Documentation at: https://developers.google.com/google-apps/calendar/v3/reference/events/list
Example endpoint we care about: https://www.googleapis.com/calendar/v3/calendars/[CALENDAR ID]/events?maxResults=2&orderBy=startTime&singleEvents=true&timeMin=[NOW-1hr]&timeZone=America%2FVancouver&key=[API KEY]
Usually, the first entry in the response to this will be the event we want to respond from !next. If that first event is more than half-over (can happen for short events like Checkpoint) then we want to respond with the second entry.

If there's any astral-plane unicode characters in the chat (ie U+10000 and beyond), MySQL throws a fit when you try to insert them into the chat log table. It truncates the string at the offending character, and throws a warning (which Python turns into an exception).

6976892 stops that exception from tearing down the chat log worker thread (as happened today during GPLP, meaning we have no chat log for most of it, or all of Beej's Backlog), but the problem still happens. Presumably the same issue would affect the other tables too... if someone subscribed with a username containing these symbols (if that's even allowed) or if someone added one of the characters to one of the static responses/explain commands/etc (and it tried to store that in the audit table).

Since the bot and the webserver are running together on the same machine (and have been for some time), we should probably refactor the code so that the two projects live in the same dir (with the bulk of the code in subpackages, naturally, but the __main__ lines in the root).

Currently there's a lot of common code between the two utils.py modules, and now with #34 having the bot import a www module... it would be nice if we could merge utils and www.utils, merge config and www.secrets... etc.

Every time a command is being added to the save the bot throws this error:
[2014-08-21 01:40:43,099] ERROR:utils:Exception in on_server_event
Traceback (most recent call last):
File "/srv/lrrbot/lrrbot/utils.py", line 238, in wrapper
return func(_args, *_kwargs)
File "lrrbot.py", line 410, in on_server_event
ret = eventproc(self, data['user'], data['param'])
File "/srv/lrrbot/lrrbot/serverevents.py", line 43, in modify_commands
commands.static.modify_commands(data)
File "/srv/lrrbot/lrrbot/commands/static.py", line 58, in modify_commands
bot.remove_command(generate_expression(storage.data["responses"]))
File "lrrbot.py", line 118, in remove_command
del self.commands[re.compile(pattern.replace(" ", r"\s+"), re.IGNORECASE)]
KeyError: <_sre.SRE_Pattern object at 0x1e600d0>

We should keep a history of all the changes to the static responses (and also the explanations and spam filters, and any future config pages we make), with names of who changed it, and allowing diffs between versions etc. Both for use as an audit log, a backup in case of accidental deletions, and just for keeping a record of old jokes.

The initial history can be populated from the lrrbot logs, everything already should be in there, albeit in a less-useful format.

Per Hosk, there should be a command the mods can use when the channel's getting particularly spammy (eg like when the stream goes down), to lock down the bot into mod-only mode for a certain amount of time, or until explicitly disabled, or something. Say:

!modonly
!modonly [DURATION]
!modonly off

or somesuch. In this mode, it completely ignores commands from non-mods... no error response, nothing. Give the chat a chance to calm down, and re-enable everything when the stream comes back up.

Perhaps a !subonly for symmetry, but I can't see us using that one very often.

We should have a command in LRRbot for running ad-hoc polls...

I doubt we could build our own polling system that would be better than Straw Poll, so we should probably just hook into that. Have a command that you can pass the options to, and it creates the poll on strawpoll.me and posts the link in the chat. Basically just to make creating a poll more convenient for the host.

Perhaps with an option for a duration to the poll, where it gets the results and posts the winner in the chat after a certain time.

The common syntax I've seen is something like:

!poll <duration-in-seconds> <option-1>, <option-2>, <option-3>

eg:

!poll 60 Go left, Stab it until it dies, Kill him for the katana

Perhaps also a !multipoll for "pick one or many"

Relevant strawpoll APIs:

POST http://strawpoll.me/ajax/new-poll
Post data: title=Automatic+poll+via+LRRbot&options[]=option-1&options[]=option-2&options[]=option-3&multi=false&permissive=false
Response: {"id":123456}

GET http://strawpoll.me/ajax/votes/123456
Response: {"0":"123","1":"74","2":"8"}

Obviously, this command would have to be mod-only.

The SSE processing for the /notifications page don't seem to be working... no new notifications ever show up. Has something happened to the event server?

Homepage. Python bindings.

Something's changed with Twitch's video archive, and the archive-with-chat thing is no longer working.

Personal thought, but think another spring-cleaning of commands is needed again. I felt it was best when the lrrbot commands were used purely for informative info (!explain/!next/!codefall) but at the moment it seems that any small joke gets it's own command, which is both unnecessary and simultaneously runs the joke into the ground anyway - especially when chat is going to just say that thing at the time, why use a command? This goes double when the commands make absolutely NO sense unless you've seen the one particular stream it comes from (!officer is a prime example of this.)

Maybe I'm just a grumpy old man but it just feels that !commands should be largely for dispensing useful information without having to copy and paste...and that new ones should be added very selectively instead of just throwing every single joke in there - the best ones are well-established and liked ones that can be used with any stream, like !panic.

For reference, I think ones that really aren't needed just scanning through the command list: !adam, !base, !bawk, !cat, !fudge, !g, !jund, !officer, !powah, !weather and !russell.

While the crew tends to have their funny moments, the chat can have their own witty moments.

So I'm suggesting a sort of Quote Database for the chat. They might be accessible from the LRRBot website, or with certain commands. For example:

!addquote <INSERT FUNNY WORDS HERE> - Add a new quote to the database
!delquote <Quote Number> - (Mod Only?) Remove a quote from the database
!quote <Quote Number> - Reposts the selected quote. If no number is given, LRRBot will reply with a random quote.
!qdb - Links to the quote database.

Any thoughts? I'd love to try and help make it, but I don't really know Python well enough. 😦

Track counts of Adam being tricked into saying "The Dutch, again!"

Currently the only way to remove or modify is to restart the bot.

Whenever the stream starts, there's some amount of delay between when the bot notices that the game has changed, and when a mod notices that the show has changed and !show overrides... one typically happens before the other, and if the bot does anything to even read the game data when that happens (including !game, !game refresh, !deathcount et al, or just loading a page on the website) then the bot will create an entry for that game in the wrong show. There's currently 4 games in the "Unknown" show, because they were created while the show wasn't set, and probably any number of overlap games (there's a Paper Mario under Things On My Stream, because the game got changed before the show was changed to Beej's Backlog, for instance).

Something should be done to keep these from happening... unless a game/show combination has something actually done in-chat (like a stat change, or a good/bad vote, or !game display) then it shouldn't show up in the game lists on the webpages (/stats and /votes). Either it shouldn't actually be added to the storage until one of these things happen (ie don't have storage.find_game actually create the game if it's missing, but rather move that logic to all the places that edit the game... maybe make a storage.save_game function that adds it to the database if it's missing, call that from commands.stats.stat_update et al). Alternatively, just have /stats and /votes and any other such pages in the future just not show any games that don't have any stats or votes (and maybe have some way to purge them from the storage for cleanliness). I'm leaning towards the former being the better plan.

This should also handle the case where a mod uses !game override but has a typo, notices, and immediately !game overrides again to fix it... the game object for the typo'd name should be removed in the same way.

This is because the bot alternates between the IRC and the event socket and currently it takes 200 ms for the IRC framework to determine it has nothing to do.

Options:

• Lower the timeout.
• Take control of the IRC socket and do select on both sockets (like b186e0e)
• Replace irc with irc3 and use asyncio.

Rather than having this built-from-scratch socket implementation that looks messy and from what I hear has some performance issues...

Why not use something like Twisted? Has a stellar IRC library, support for tons of other protocols, and a great built-in plugin system.

(Refer to projects such as DesertBot, the current bot project for the Desertbus chatroom)

A link to bit.ly/letsnopebingo for the Let's Nope Bingo instructions would be awesome.

We currently use tnotifier (tnotifier.net) to run the popup sub notifications. The system works pretty well, but, despite assurance that the feature is in development, it has never worked for resubs. It turns out that our viewers are really loyal (Yay!) which means that most subs are resubs and only a few actually show up on the stream. Other freely available subscriber notification systems I have found have similar problems or don't allow enough customization. I realized that LRRbot is already doing sub and resub tracking with its welcome messages and the notifications page (http://lrrbot.qrpth.eu/notifications). If LRRbot could have an api endpoint that returned all the subs in the last x minutes or maybe the last x subs, along with how many months they have subscribed for, then it should be pretty straight forward for me to make a notifier plugin for xsplit that checks the list every minute, removes duplicates, and then fires a notification for any new subscribers or resubs.

Ideally the endpoint would return a JSON file something like this:
{{"drmcgoodtimes",3},{"aussz",6},{"mrphlip",18},{"newguy",0}}

Does this make sense and do you think this is something that LRRbot could do without too much trouble? As I said, I was thinking that most of the work of actually collecting the notifications has already been done for the welcome and notifications page.

-Paul

The bird glyph on !russell makes Twitch swallow the entire message.

The chat emotes in the archive are applied after the string gets HTMLised, which means that, for example, something like:

Ha ha, he said "poop"!

is turning into:

Ha ha, he said "poop"!

which now contains ;p, which then becomes .

We need to jump through some hoops so that the emotes are replaced before we HTMLise the string, but with the emotes still coming through as images, and not <img>... only HTMLise the text between the replaced emotes...

The chat logs on the VODs seem to stop with some regularity. Is there a specific reason they're doing that, or just random things pop up?

If something funny or interesting happens they could type !highlight or !MarkHighlight or something. and it would mark the "uptime" of the stream the date and the stream name.

I get emails a lot saying "this funny thing happened on this show" but the only way for me to highlight it would be to watch the entire stream and try to find what they are talking about.

Not a high priority at all just wondering if its possible.
(wasnt sure how to tag it as a suggestion)

The reasons for using a JSON datafile over a proper DB is getting less and less relevant as more stuff keeps getting stuffed into there, and the data structures keep getting more complicated... and since we actually have a DB to store things into, it probably makes sense to start storing things in there instead.

It doesn't need to be done all in one hit, but it would be good to start moving things out of the JSON and into the DB, make some proper DB structures to store the game stats, etc.

00:25 < lithobraker> Hey glorious moderators:  can we set up Lrrbot to include a link to the stream referenced in the result of the nextfan command?

Most of the events in the calendar have a location set to something resembling a Twitch URL.

Example: http://qrpth.eu/prism/log?start=1431785340.0&stop=1431785400.0

(╯°Д°）╯︵ɥɔʇıʍʇ: justintv/Twitch-API#338

Should we change the response to accommodate the fact that we now have more information about subs, such as how many months they have been subbed for? Also I need to verify that the change didn't break the bot...

The bot gives the message but there's no mod action that actually happen. Tried restarting the bot to no avail

If we can figure out a good way to determine which show is currently live, we should make it so the game stats are grouped by show. This would allow, for instance, instead of having three different Dark Souls games and two different New Super Mario Bros U games, we'd just have one New Super Mario Bros U game with House of Stark stats and Crossing the Streams stats. We'd still have to have different game entries occasionally, like for all the various Minecraft maps from Crossing the Streams, but it would make the stats a lot cleaner, and would allow more grouping on the /stats page, so we could see just how much more Alex dies than anyone else.

Dynamically assign a throttle to !responses based on the amount of viewers in chat to make high viewer chat manageable with LRRBot on. Also on request an override to simply disable !responses for a certain amount of time.