bahmanm / zebel

Zebel: The Intelligent IRC Bot - HAL's Cousin

Introduction

"ZEBEL" (pronounced like "rebel") is an intelligent IRC bot.

Its primary goal is gathering intelligence and secrets through contextual analysis of the conversations on IRC. I almost got you! :-)

Jokes aside, Zebel is a simple IRC bot that whenever you talk to him, tries to reply with a relatively relevant message. If you're curious, drop by in #zebel on irc.freenode.org and have a chat with him.

Design

When someone in an IRC channel, talks to Zebel, e.g. sends out a public message like zebel: greetings, Zebel tries to make sense of your message as below:

1. It first sanitises the message; removing stop words, punctuations and unimportant words.
2. The sanitised text is fed into natural language processing to determine each word's role in the sentence.
3. For each word and its given role, the list of synonyms are extracted from NLP database.
4. The words along with their synonyms are forged into a query.
5. A full-text search on the DB is performed.
6. If anything relevant enough is found, it is returned.
7. If there's no relevant enough answer in the database, a fuzzy search for the original message is performed.
8. If fuzzy search finds a good enough answer, it is returned.
9. Even if fuzzy search can't help, Zebel simply gives up on finding a reasonable answer and instead just picks up a message from the "confused" category.

Implementation

Zebel, developed with Python 3.4+, is composed of 3 main parts:

1. The IRC client (irc library)
2. The database (ElasticSearch)
3. The text and natural language processing unit (nltk library)

How To Run

If you plan to run your own Zebel instance:

1. Install Python 3.4
2. Create a virtual environment for Zebel, e.g. pyvenv-3.4 .venv
3. Activate the venv, e.g. source .venv/bin/activate
4. Clone Zebel.
5. Install required packages, e.g. cd zebel && pip install -r requirements.txt
6. Install nltk data, e.g. python -m nltk.downloader all. Warning: this downloads quite a bit of data.
7. Edit zebel.ini to your needs.
8. Install and run ElasticSearch.
9. Import Zebel's quotes and messages into ElasticSearch, e.g. data/drop-import-all.sh
10. Run Zebel, e.g. cd zebel && python main.py

Notes

• The bot named "vandusen" in #chicken on FreeNode was a great inspiration to me.

zebel's Issues

Zebel can easily get flooded

If set to talk to a bot like himself which cranks out immediate response, both will get stuck in an infinite loop.

It should preferrably stop responding to a nick for a length of annoy-time after responding annoyedly to that nick once.

Stop words and punctuation filtering

Zebel doesn't filter "..." during sanitisation.

Bot never responds, ends up crashing

I followed all of your instructions after spinning up a new VM just for this. When I say hi to the bot after it joins the IRC channel, this is what I see in the console:

(.venv) geeknik@frank:~/zebel$ python main.py
INFO:MSG geeknik > mfm: hi!
INFO:Starting new HTTP connection (1): localhost
INFO:GET http://localhost:9200/zebel_quotes/_search [status:200 request:0.102s]
INFO:curl -XGET 'http://localhost:9200/zebel_quotes/_search?pretty' -d '{
  "filter": {
    "type": {
      "value": "general"
    }
  },
  "min_score": "0.2",
  "query": {
    "query_string": {
      "default_field": "text",
      "query": "text:(hi) OR support:(hi)"
    }
  }
}'
WARNING:GET /zebel_quotes/_search [status:400 request:0.021s]
Traceback (most recent call last):
  File "main.py", line 48, in <module>
    zebel.connect().start()
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 1274, in start
    self.reactor.process_forever()
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 276, in process_forever
    self.process_once(timeout)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 257, in process_once
    self.process_data(i)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 214, in process_data
    c.process_data()
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 583, in process_data
    self._process_line(line)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 654, in _process_line
    self._handle_event(event)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 677, in _handle_event
    self.reactor._handle_event(self, event)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 396, in _handle_event
    result = handler.callback(connection, event)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/irc/client.py", line 1237, in _dispatcher
    method(connection, event)
  File "/home/geeknik/zebel/bot.py", line 67, in on_pubmsg
    conn.privmsg(self.target, text.reply(sender, msg))
  File "/home/geeknik/zebel/text.py", line 264, in reply
    _MIN_SCORE)
  File "/home/geeknik/zebel/db.py", line 86, in fuzzy_fetch_general_quotes
    results = self.client.search(index='zebel_quotes', body=qs)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/elasticsearch/client/utils.py", line 68, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/elasticsearch/client/__init__.py", line 498, in sear                                                                                                ch
    params=params, body=body)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/elasticsearch/transport.py", line 301, in perform_re                                                                                                quest
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=tim                                                                                                eout)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/elasticsearch/connection/http_urllib3.py", line 82,                                                                                                 in perform_request
    self._raise_error(response.status, raw_data)
  File "/home/geeknik/.venv/lib/python3.4/site-packages/elasticsearch/connection/base.py", line 102, in _rai                                                                                                se_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, {'phase': 'query', 'type': 'search_phase_executio                                                                                                n_exception', 'grouped': True, 'root_cause': [{'line': 1, 'index': 'zebel_quotes', 'type': 'query_parsing_ex                                                                                                ception', 'reason': 'No query registered for [fuzzy_like_this]', 'col': 32}], 'failed_shards': [{'shard': 0,                                                                                                 'index': 'zebel_quotes', 'reason': {'line': 1, 'index': 'zebel_quotes', 'type': 'query_parsing_exception',                                                                                                 'reason': 'No query registered for [fuzzy_like_this]', 'col': 32}, 'node': 'ctfhCRYdSE2i-00FZEKhQg'}], 'reas                                                                                                on': 'all shards failed'})

better detection of messages directed towards zebel

Right now only messages of the form zebel: XYZ and zebel, XYZ are detected by zebel. It should understand the following forms as well:

• zebel XYZ
• zebel:XYZ