cloudflare / ghc-errbot

A Google Hangouts Chat Backend for Errbot

Google Hangouts Chat - Errbot Backend

This is a backend for Google Hangouts Chat (https://chat.google.com) for Errbot(https://errbot.io).

It allows you to use errbot to create bots, but as always, it's a work in progress.

Installation

git clone https://github.com/cloudflare/GHC-Errbot

and then

BACKEND = 'Google-Hangouts-Chat'
BOT_EXTRA_BACKEND_DIR = '/path/to/where/you/cloned/the/repo/'

to your config.py

Authentication

1. Create a Google Pub/Sub topic in a GCE project

2. Create a Subscriber on that topic and grant your bot account Subscriber permissions

3. Generate a creds.json for your bot

4. Create an application with errbot init, and then create a BOT_IDENTITY block in your config.py with the following information:

BOT_IDENTITY = {
    'GOOGLE_CREDS_FILE': '/path/to/bot/creds.json',
    'GOOGLE_CLOUD_ENGINE_PROJECT': '<your project name>',
    'GOOGLE_CLOUD_ENGINE_PUBSUB_TOPIC': '<your pub/sub topic>',
    'GOOGLE_CLOUD_ENGINE_PUBSUB_SUBSCRIPTION': '<your pub/sub subscription name>',
}

5. Set BOT_PREFIX to the name of the bot, including the mention(@)

6. (optional) To enable prometheus metrics, set METRICS_PORT to an integer. This will be the port you want to open for metrics.

Examples

Attachments

This backend supports attachments in message events. To download a Google Chat upload attachment, we need to use the GetAttachment API and HTTP GET request with Bearer authentication. Since the backend is already authenticated, we opportunistically provide a ready-to-use downloader object with the message context, so that errbot plugins can use it to directly download the attachments, no extra steps required.

Here's a code example on how to use the downloader helper in a errbot plugin:

from io import BytesIO
from errbot import BotPlugin, botcmd

@botcmd(split_args_with=None)
def upload(self, msg, args):
    attachments = msg._extras.get('attachment', [])
    for attachment in attachments:
        if attachment['source'] == 'UPLOADED_CONTENT':
            url = f"""https://chat.googleapis.com/v1/media/{ attachment['attachmentDataRef']['resourceName'] }?alt=media"""
            downloader = msg._extras.get('downloader')
            content = downloader(url)
            if content != None:
                d = BytesIO()
                d.write(content)
                # jira.add_attachment(issue=issue, attachment=d, filename=attachment['contentName'])

Acknowledgement

The code in markdownconverter.py is from https://github.com/dr-BEat/errbot-backend-hangoutschat. It is MIT licensed.

License

Licensed under the BSD 3 License.

ghc-errbot's Issues

Markdown support should be optional

Google Chat only supports a subset of markdown, and actually formats itself. This leads to weird situations, for e.g. if I format a table in a message:

```
Table ID | Value
-------- |-------
ID1      | Value1 
```

Converting this to markdown in the adapter, and then shipping it off to Google Chat (Where the markdown is processed again) breaks the formatting

Asking the same question of the bot in the same thread never returns

If you ask the bot the same question (message length) in the same thread it will never answer.

I found that it's because the eventTime is being pulled from the wrong data structure.

I will create a PR shortly with a proposed fix

We don't handle too long messages well

If we send a message that is longer than 4096 characters in a single POST to Google Chat, it baulks and sends us back an error:

2018-11-16T14:57:08.332 36s126 2018-11-16 14:57:08,331 ERROR    errbot.backends.hangoutschat status: 400, content: b'{\n  "error": {\n    "code": 400,\n    "message": "Maximum length of the field is 4096. Field: apps.dynamite.CreateMessageInfo.text_body",\n    "status": "INVALID_ARGUMENT",\n    "details": [\n      {\n        "@type": "type.googleapis.com/google.rpc.BadRequest",\n        "fieldViolations": [\n          {\n            "field": "apps.dynamite.CreateMessageInfo.text_body",\n            "description": "Maximum length of the field is 4096. Field: apps.dynamite.CreateMessageInfo.text_body"\n          }\n        ]\n      }\n    ]\n  }\n}\n'
2018-11-16T14:57:08.588 36s126 2018-11-16 14:57:08,587 ERROR    google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager Top-level exception occurred in callback while processing a message

We should cut messages of this size up and make individual POSTs for each chunk

Duplication of messages that have already been handled

We've been seeing an issue when using this as our back-end where the bot performs an action twice in response to a single message. For some functions of the bot this is merely annoying, but for others it can be detrimental (e.g. a function to open a ticket would open two tickets). I've tracked this down as a likely consequence of how the pubsub library works. In the documentation for the pubsub library, they state

Cloud Pub / Sub is designed to provide “at least once” delivery

and in order to not receive a message a second time you have to ack() the message. The code already Acks the messages, but we are still seeing messages multiple times. Digging into the documentation a bit more, the ack() method has this note on it

Acks in Pub/Sub are best effort. You should always ensure that your processing code is idempotent, as you may receive any given message more than once.

As a best guess, this means that the ack() may not actually work 100% of the time (e.g. network issues) and the pubsub library makes no guarantees about this. It may be possible to mitigate this issue by keeping the set of recent messages and the last time we saw them (where recent can be configured, maybe default to last 2 minutes) and checking if we've already received an identical message in the _handle_message method.

Handling cards

See KimNorgaard@9ada23f

This covers being able to send cards through the backend plugin directly like this:

@botcmd
def my_cmd(self, msg, args):
    cards = [{'sections': [{'widgets': []}]}]  # etc...
    self._bot.send_card(cards=cards, space_id=msg.extras['space_id'], thread_id=msg.extras['thread_id'])

I can create a PR for this if you want me to.

The other way around (handling CARD_CLICKED events) is a bit more tricky as it involves somehow making the bot commands aware that the message isn't just plain text. This could be achieved by passing the event data in the Message.extras attribute and reserving commands for handling these events, e.g.

In hangouts_chat.py - GoogleHangoutsChatBackend._handle_message():

    if data['type'] == 'CARD_CLICKED':
        msg.body = 'handle ghc clicked'
        msg.extras['card_event'] = data

In the bot plugin:

@botcmd
def handle_ghc_card_clicked(self, msg, args):
    msg.extras['card_event']  # { "type": "CARD_CLICKED", "message":{ ...

Any thoughts on this?

Support setting thread_key when using send_message

There's currently support for setting thread_id in the message extras used by send_message, but we can't set thread_key. (Thread ID and thread key are different -- The thread key is useful for sending a multiple distinct messages to a single thread over time, without having to keep track of the thread ID. More in the docs.)

The underlying create_message API call does accept a thread_key though -- so let's allow setting it in message extras and pass it on through.

Messages without text cause infinite retries

We assume that every message that we pull off the queue that has a message in it's data is a text message:

message_body = data['message']['text']

This appears to not be the case as of recently (image only messages perhaps?) and causes some noisy logs:

2018-11-16T00:00:06.007 36s126 2018-11-16 00:00:06,008 ERROR    google.cloud.pubsub_v1.subscriber._protocol.streaming_pull_manager Top-level exception occurred in callback while processing a message
2018-11-16T00:00:06.013 36s126   File "/usr/local/lib/python3.5/dist-packages/google/cloud/pubsub_v1/subscriber/_protocol/streaming_pull_manager.py", line 63, in _wrap_callback_errors
2018-11-16T00:00:06.065 36s126     callback(message)
2018-11-16T00:00:06.092 36s126   File "/opt/respect-tables/backends/hangouts-chat/hangouts_chat.py", line 239, in _handle_message
2018-11-16T00:00:06.094 36s126     message_body = data['message']['text']
2018-11-16T00:00:06.095 36s126 KeyError: 'text'

We should fix that and ignore messages that don't have a text field

Unable to load or configure the backend.

15:27:52 ERROR errbot.bootstrap Unable to load or configure the backend.

Traceback (most recent call last):
File "/Library/Python/3.9/site-packages/errbot/bootstrap.py", line 181, in setup_bot
bot = backendpm.load_plugin()
File "/Library/Python/3.9/site-packages/errbot/backend_plugin_manager.py", line 62, in load_plugin
plugin_classes = self.plugin_info.load_plugin_classes(
File "/Library/Python/3.9/site-packages/errbot/plugin_info.py", line 100, in load_plugin_classes
spec.loader.exec_module(modu1e)
File "", line 850, in exec_module
File "", line 228, in _call_with_frames_removed
File "/Users/dhanesh/devops/bot/GHC-Errbot/hangouts_chat.py", line 8, in
from errbot.errBot import ErrBot
ModuleNotFoundError: No module named 'errbot.errBot'

Getting this error, rest of the setup I have completed, how to fix this or any work around is there.
Please update, it will be more useful for google chat integration.