finos / symphony-bdk-python

Symphony Python Bot Development Kit (BDK)

Symphony BDK for Python

This is the Symphony BDK for Python to help develop bots and interact with the Symphony REST APIs.

Installation and getting started

The reference documentation includes detailed installation instructions as well as a comprehensive getting started guide.

Build from source

The Symphony BDK uses and requires Python 3.8 or higher. Be sure you have it installed before going further.

We use poetry in order to manage dependencies, build, run tests and publish. To install poetry, follow instructions here.

On the first time, run poetry install. Then run poetry build to build the sdist and wheel packages. To run the tests, use poetry run pytest.

It is possible to run pylint scan locally (on a specific file or package) executing the following command: poetry run pylint <module_name>.

To generate locally the Sphinx documentation, run: cd docsrc && make html.

Roadmap

Our next milestone is the 2.5.x one. It will only consist in delivering the next improvements and bug fixes of the BDK.

Contributing

In order to get in touch with the project team, please open a GitHub Issue. Alternatively, you can email/subscribe to [email protected].

1. Fork it
2. Create your feature branch (git checkout -b feature/fooBar)
3. Read our contribution guidelines and Community Code of Conduct
4. Commit your changes (git commit -am 'Add some fooBar')
5. Push to the branch (git push origin feature/fooBar)
6. Create a new Pull Request

NOTE: Commits and pull requests to FINOS repositories will only be accepted from those contributors with an active, executed Individual Contributor License Agreement (ICLA) with FINOS OR who are covered under an existing and active Corporate Contribution License Agreement (CCLA) executed with FINOS. Commits from individuals not covered under an ICLA or CCLA will be flagged and blocked by the FINOS Clabot tool. Please note that some CCLAs require individuals/employees to be explicitly named on the CCLA.

Need an ICLA? Unsure if you are covered under an existing CCLA? Email [email protected]

Update generated code

While contributing to the project, you might need to update the generated code. Python BDK uses OpenAPITools/openapi-generator to generate code. In order to customise the templates, a fork has been created in https://github.com/SymphonyPlatformSolutions/openapi-generator/tree/sym-python-5.5.0.
Here are the steps to follow:

• Checkout the latest branch of the fork (currently sym-python-5.5.0)
• Update the fork source code, review and merge it
• Generate a jar file in openapi-generatormodules/openapi-generator-cli/target/openapi-generator-cli.jar:
  • Using maven: mvn clean install -Dmaven.test.skip=true && mvn clean package -Dmaven.test.skip=true. You can also use IntelliJ's build button to build the project and generate the jar
• Copy the jar in Python BDK repository symphony-api-client-python/api_client_generation/openapi-generator-cli.jar
• Execute the generation script ./generate.sh and commit and push you new code along with the new jar file.

License

Copyright 2021 Symphony LLC

Distributed under the Apache License, Version 2.0.

SPDX-License-Identifier: Apache-2.0

symphony-bdk-python's Issues

Error: Could not find a version that satisfies the requirement symphony-bdk-python>=2.0.0 (from versions: none)

I have created a project with yo @finos/symphony, and then when I run pip3 install -r requirements.txt command, the bellow error occurs. How could it be solved?

$ python3 --version
Python 3.6.9

$ cat requirements.txt
symphony-bdk-python>=2.0.0
Jinja2~=3.0

$ pip3 install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
ERROR: Could not find a version that satisfies the requirement symphony-bdk-python>=2.0.0 (from versions: none)
ERROR: No matching distribution found for symphony-bdk-python>=2.0.0

Forward an attachment without saving it to the disk

How do you retrieve an attachment and forward it to another stream/room without saving its content to the disk?

Closes #120

truststorePath raise a key error

When upgrading the version from 0.1.11 to 0.1.15, I did the following:

1. Reimport the classes(From camel case to snake case)
2. Update the implementation of IMListener abstract method(From camel case to snake case).
3. Consider attribute "truststore" is optional in config.json, I didn't put that in config.json. But I got an error like the following img:
   

I also tried:

1. Adding "truststore" to config.json, same error.
2. Adding "truststorePath" with value empty string "", it gives me an error on the response:
   

To solve the issue:

1. We can either update the "symphony-api-client-python/sym_api_client_python/configure/configure.py", adding "truststore" into self.data.
2. Or, we can consider using "if self.config.data.get('truststorePath')" instead of "self.config.data['truststorePath']" in "symphony-api-client-python/sym_api_client_python/clients/sym_bot_client.py".

Move "Chabot" from under Listener into an "example" folder at the top

https://github.com/SymphonyPlatformSolutions/symphony-api-client-python/tree/master/sym_api_client_python/listeners/chatbot

This example should be better located, in a place where developers can easily find it: in a "sample" directory at the top of the project.

BDK Crashes occasionally, stack trace attached.

Hey guys, we regularly receive exceptions when running the bot. We are using the bot framework 2.0.1. I've just bumped us up to 2.1.0 but curious if anyone had insight what this stack indicates.

Traceback (most recent call last):
  File "/home/dky/.pyenv/versions/3.9.6/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/dky/.pyenv/versions/3.9.6/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/dky/git/bot/src/__main__.py", line 131, in <module>
    asyncio.run(run())
  File "/home/dky/.pyenv/versions/3.9.6/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/home/dky/.pyenv/versions/3.9.6/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/home/dky/git/bot/src/__main__.py", line 119, in run
    await datafeed_loop.start()
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/service/datafeed/abstract_datafeed_loop.py", line 99, in start
    await self._run_loop()
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/service/datafeed/abstract_datafeed_loop.py", line 149, in _run_loop
    await self._run_loop_iteration()
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/service/datafeed/datafeed_loop_v1.py", line 54, in _run_loop_iteration
    events = await self._read_datafeed()
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/retry/_asyncio.py", line 118, in async_wrapped
    return await fn(*args, **kwargs)
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/retry/_asyncio.py", line 80, in __call__
    do = await self.iter(retry_state=retry_state)
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/retry/_asyncio.py", line 41, in iter
    should_retry = await self.retry(retry_state=retry_state)
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/retry/strategy.py", line 119, in read_datafeed_retry
    raise exception
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/retry/_asyncio.py", line 83, in __call__
    result = await fn(*args, **kwargs)
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/service/datafeed/datafeed_loop_v1.py", line 63, in _read_datafeed
    events = await self._datafeed_api.v4_datafeed_id_read_get(id=self._datafeed_id,
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/core/client/trace_id.py", line 46, in add_x_trace_id_header
    return await func(*args, **kwargs)
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/gen/api_client.py", line 195, in __call_api
    response_data = await self.request(
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/gen/rest.py", line 190, in GET
    return await self.request("GET", url,
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/symphony/bdk/gen/rest.py", line 165, in request
    r = await self.pool_manager.request(**args)
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/aiohttp/client.py", line 559, in _request
    await resp.start(conn)
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 913, in start
    self._continue = None
  File "/home/dky/git/bot/env/lib/python3.9/site-packages/aiohttp/helpers.py", line 718, in __exit__
    raise asyncio.TimeoutError from None
asyncio.exceptions.TimeoutError

get_user_streams() has incorrect parameter

The parameter is incorrect:

https://github.com/SymphonyPlatformSolutions/symphony-api-client-python/blob/d8fe63103458ec8109900015814e8de54f4cc9c2/sym_api_client_python/clients/stream_client.py#L167

Per spec it should be includeInactiveStreams
https://developers.symphony.com/restapi/reference#list-user-streams

{
"streamTypes": [
{"type": "IM"},
{"type": "MIM"},
{"type": "ROOM"},
{"type": "POST"}
],
"includeInactiveStreams": true
}

Add Accessible Command Arguments to Slash Activity

Slash activities or command activities should have a way to easily parse argument.
See finos/symphony-bdk-java#511 for the corresponding Java BDK ticket.

Apply FINOS Software Project Blueprint

Steps required by https://odp.finos.org/docs/project-collaboration/#finos-project-blueprint:

Previously missing these 4 components:

• A Code of Conduct template, required by FINOS bylaws
• Contributing guidelines, required by FINOS bylaws
• WhiteSource custom configuration, to setup continuous security scanning for your project
• Issue templates for bugs, feature requests and support questions , defined in the .github folder

Online Presence not working in Python BDK

Bug Report

Online presence not working

Steps to Reproduce:

1. Create a Python Demo Bot using the Python BDK
2. Launch bot, authenticate and login
3. Send/Receive Messages to the bot, but presence does not show as online.

Expected Result:

The Bots online presence should have a green online status.

Actual Result:

Starting up the demo bot, runs fine. It looks good and replies with the @mention followed by /command but the bot's online status at the Symphony App is off line with a grey box.

Environment:

develop2.symphony.com

Broken Links in the Documentation

Bug Report

Steps to Reproduce:

1. Navigate to https://symphony-bdk-python.finos.org/markdown/message_service.html
2. Click on any links
3. ...step 3 description...

Expected Result:

Clicking on the link should take the user to the appropriate page instead of 404.

Actual Result:

Environment:

...version and build of the project, OS and runtime versions, virtualised environment (if any), etc. ...

Additional Context:

...add any other context about the problem here. If applicable, add screenshots to help explain...

Default encryption set to RS256 instead of RS512

"I inspected the python package and it appears that it is using RS256 algorithm to create the JWT token. It appears that the token generated by this algorithm is no longer supported by Symphony. I used the RS512 algorithm and the process worked fine. RS512 algorithm is used in the JWT sample code provide on Symphony’s portal."

Deliver a 2.0.0 version in Pypi

We should deliver a 2.0.0 version in Pypi using the finos user under the new package name symphony-bdk-python.

Update pyproject.toml, README file and documentation following move to FINOS

Goal is to update:

• pyproject.toml to have license set to Apache-2.0 and package name set to symphony-bdk-python (to stay consistent with the java BDK
• README.md file to have links updated and a roadmap added
• documentation in docsrc/ to have all the links updated

Command activity breaks with a message containing "&"

When bots receives a MESSAGESENT event with a presentation ML containing &, a bot having a command activity breaks with following error:

2021-10-05 10:01:25,515 - root - DEBUG - Message received from Alex Nalin (Develop 2): <div data-format="PresentationML" data-version="2.0">&amp;</div>
2021-10-05 10:01:25,515 - root - DEBUG - Exception occurred inside Task-46
Traceback (most recent call last):
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\xml\etree\ElementTree.py", line 1693, in feed
    self.parser.Parse(data, 0)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 54

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\service\message\message_parser.py", line 26, in get_text_content_from_message
    return tostring(fromstring(presentation_ml), method="text").decode().strip()
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\defusedxml\common.py", line 126, in fromstring
    parser.feed(text)
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\xml\etree\ElementTree.py", line 1695, in feed
    self._raiseerror(v)
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\xml\etree\ElementTree.py", line 1602, in _raiseerror
    raise err
  File "<string>", line None
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 1, column 54

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\activity\command.py", line 22, in __init__
    self._text_content = get_text_content_from_message(source_event.message)
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\service\message\message_parser.py", line 28, in get_text_content_from_message
    raise MessageParserError("Unable to parse the PresentationML, it is not in the correct format.") from exc
symphony.bdk.core.service.exception.MessageParserError: Unable to parse the PresentationML, it is not in the correct format.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\service\datafeed\abstract_datafeed_loop.py", line 203, in _dispatch_to_listener_method
    await self._run_listener_method(listener, event)
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\service\datafeed\abstract_datafeed_loop.py", line 218, in _run_listener_method
    await listener_method(event.initiator, event_field)
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\activity\registry.py", line 28, in decorator
    return await func(*args, **kwargs)
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\activity\registry.py", line 83, in on_message_sent
    context = CommandContext(initiator, event, self._bot_display_name)
  File "C:\Users\alex.nalin\AppData\Local\Programs\Python\Python38-32\lib\site-packages\symphony\bdk\core\activity\command.py", line 24, in __init__
    raise FatalActivityExecutionException("Unable to parse presentationML") from exc
symphony.bdk.core.activity.exception.FatalActivityExecutionException: Unable to parse presentationML

Support custom properties in BDK Configuration file

Is there any way for developers to extend or add additional properties to the file for other bot logic usage? Can they still use the BdkConfig class to retrieve these custom properties? This will avoid developers having to load and maintain another config file just for their own bot logic.

How about allowing developers to put their custom properties under a special root in the config, such as “custom” root. Then, they can access them via config.custom.get(“xxx”)?

Improve Support user suspension API

Feature Request

Description of Problem:

We would like to add support for the Suspend User Account v1 public API in the BDK.
Link to the API in the documentation: https://developers.symphony.com/restapi/reference#suspend-user-v1

We decided to ease the usage of this endpoint by providing two different functions to suspend/unsuspend a user account.

• suspendUser(userId, reason, until)
• unsuspendUser(userId)

Gif images as attachments don't move/update

I'm running into an issue where if I attach a gif image as a file using the following method:

with open("image.gif", "rb") as file:
  await bdk.messages().send_message(stream_id, Message(content="Hello", attachments=[file]))

The attachment shows up in a room as an image however the gif doesn't actively update. Am I missing a param that I should add to make the gif be interactive?

I've been reading the docs for this:

https://symphony-bdk-python.finos.org/_autosummary/symphony.bdk.core.service.message.model.Message.html

And it's not clear to me if there's potentially a preview flag that I'm missing. Any help would be appreciated!

Thanks!

Add SessionInfo and HealthCheck endpoints

Exception: V4Message has no attribute 'data'

Bug Report

If there are 2 Python bots in a room when 1 bot sends a message into the room by responding to a end user's command the other bot outputs the following Python exception in the logs.

symphony.bdk.gen.exceptions.ApiAttributeError: V4Message has no attribute 'data' at ['['received_data', 'events', 0, 'payload', 'message_sent', 'message']']['data']

Steps to Reproduce:

1. Create 2 Python BDK bots hualiang_api and luoapisup using Symphony Generator. Both bots have the same code including the following:
   @activities.slash("/hello")
async def hello(context: CommandContext):
name = context.initiator.user.display_name
response = f"<messageML>Hello {name}, hope you are doing well!</messageML>"
await bdk.messages().send_message(context.stream_id, response)
2. Run both bots and add them into a room
3. An end user sends the following message into this room

@hualiang_api /hello

4. The bot hualiang_api responds the following message.

Hello Hualiang Luo (SUP), hope you are doing well!

5. The bot luoapisup outputs the following Python exception in the logs.

2022-07-19 16:40:24,134 - root - DEBUG - Message received from Hualiang Luo (SUP):

@hualiang_api /hello

2022-07-19 16:40:25,188 - root - DEBUG - Message received from hualiang_api:

Hello Hualiang Luo (SUP), hope you are doing well!

2022-07-19 16:40:25,189 - root - DEBUG - Exception occurred inside Task-197
Traceback (most recent call last):
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/core/service/datafeed/abstract_datafeed_loop.py", line 185, in _dispatch_to_listener_method
await self._run_listener_method(listener, event)
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/core/service/datafeed/abstract_datafeed_loop.py", line 200, in _run_listener_method
await listener_method(event.initiator, event_field)
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/core/activity/registry.py", line 28, in decorator
return await func(*args, **kwargs)
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/core/activity/registry.py", line 88, in on_message_sent
if act.matches(context):
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/core/activity/command.py", line 117, in matches
match_result = self._command_pattern.get_match_result(context.source_event.message)
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/core/activity/parsing/slash_command_pattern.py", line 68, in get_match_result
input_tokenizer = InputTokenizer(message)
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/core/activity/parsing/input_tokenizer.py", line 26, in init
json_data = message.data if message.data else "{}"
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/gen/model_utils.py", line 188, in getattr
return self.getitem(attr)
File "/home/hluo/bots/python/zd67221b/env/lib/python3.10/site-packages/symphony/bdk/gen/model_utils.py", line 523, in getitem
raise ApiAttributeError(
symphony.bdk.gen.exceptions.ApiAttributeError: V4Message has no attribute 'data' at ['['received_data', 'events', 0, 'payload', 'message_sent', 'message']']['data']

6. If the end user sends the message "@luoapisup /hello" the bot hualiang_api also outputs the above Python exception in the logs.

Expected Result:

There is NO Python exception in the logs.

Actual Result:

The Python exception occurs.

Environment:

pod: sup.symphony.com
Python BDK: 2.4.0
python3: 3.10.4
Symphony Generator: 2.6.0

Additional Context:

...add any other context about the problem here. If applicable, add screenshots to help explain...

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Pending Approval

These branches will be created by Renovate only once you click their checkbox below.

• Update abatilo/actions-poetry digest to 7b6d33e
• Update dawidd6/action-download-artifact digest to 8a9be73
• Update snyk/actions digest to 8349f90
• Update dependency aiohttp to v3.9.5
• Update dependency python-dateutil to v2.9.0
• Update dependency docutils to v0.21.1
• Update dependency furo to v2022.12.7
• Update dependency liccheck to ^0.9.0
• Update dependency pytest-asyncio to ^0.23.0
• Update actions/cache action to v4
• Update actions/checkout action to v4
• Update actions/setup-python action to v5
• Update actions/upload-artifact action to v4
• Update dependency Sphinx to v7
• Update dependency furo to v2024
• Update dependency pylint to v3
• Update dependency pytest to v8
• Update dependency pytest-cov to v5
• Update dependency safety to v3
• Update dependency ubuntu to v22
• Update dependency urllib3 to v2
• 🔐 Create all pending approval PRs at once 🔐

Detected dependencies

github-actions

.github/workflows/build.yml

• actions/checkout v2
• actions/setup-python v2
• actions/cache v2
• actions/cache v2
• actions/upload-artifact v2
• actions/upload-artifact v2

.github/workflows/cve-scanning-python.yml

• actions/checkout v3
• actions/setup-python v4
• abatilo/actions-poetry 192395c0d10c082a7c62294ab5d9a9de40e48974

.github/workflows/license-scanning-python.yml

• actions/checkout v3
• actions/setup-python v4

.github/workflows/push.yml

• snyk/actions b98d498629f1c368650224d6d212bf7dfa89e4bf
• actions/checkout v2
• actions/setup-python v2
• actions/cache v2
• actions/cache v2

.github/workflows/pylint.yaml

• actions/checkout v2
• actions/setup-python v2
• actions/cache v2
• dawidd6/action-download-artifact 8c96194455d424a24437ec57817e0aa1a1d72adb
• actions/upload-artifact v2

.github/workflows/release.yml

• actions/checkout v2
• actions/setup-python v2

.github/workflows/security.yml

• actions/checkout v2
• actions/setup-python v2
• actions/cache v2
• actions/cache v2

.github/workflows/semgrep.yml

• actions/checkout v3
• ubuntu 20.04

pep621

pyproject.toml

• poetry-core >=1.0.0

poetry

pyproject.toml

• python ^3.8
• nulltype ^2.3.1
• python-dateutil ^2.8.2
• urllib3 ^1.26.9
• aiohttp ^3.9.2
• pyyaml ^6.0
• PyJWT ^2.3.0
• cryptography ^42.0.4
• tenacity ^8.0.1
• defusedxml ^0.7.1
• docutils 0.16
• pytest ^7.1.1
• pylint ^2.6.0
• pytest-cov ^3.0.0
• pytest-asyncio ^0.18.2
• Sphinx ^4.4.0
• recommonmark ^0.7.1
• furo ^2022.3.4
• hazelcast-python-client ^5.0.1
• safety ^2.2.0
• liccheck ^0.6.2

---

• Check this box to trigger a request for Renovate to run again on this repository

Python 3.10 ClientConnectorSSLError

Bug Report

Python BDK (tested on v2.2.0 & 2.3.0) both run into TLS issues when run from Python 3.10:
ClientConnectorSSLError: Cannot connect to host sup.symphony.com:443 ssl:default [Cannot create a client socket with a PROTOCOL_TLS_SERVER context (_ssl.c:801)]

The bot functions fine on Python versions 3.9 and 3.8. As Python 3.10 is required for my project, could a workaround or fix be provided?

Steps to Reproduce:

1. Install Python 3.10 and using that python version, attempt to run your Python BDK

Expected Result:

Expected result would be that the bot would succesfully connect to the Datafeed and start capturing messages being sent to the bot, as shown below for Python 3.9:
2022-06-17 11:27:49,522 - symphony.bdk.core.activity.registry - DEBUG - Registering slash command with command=/hello, mention_bot=True 2022-06-17 11:27:49,523 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <symphony.bdk.core.activity.command.SlashCommandActivity object at 0x104160e80> 2022-06-17 11:27:49,523 - symphony.bdk.core.activity.registry - DEBUG - Registering slash command with command=/connect, mention_bot=True 2022-06-17 11:27:49,523 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <symphony.bdk.core.activity.command.SlashCommandActivity object at 0x104160e20> 2022-06-17 11:27:49,523 - symphony.bdk.core.service.datafeed.abstract_datafeed_loop - DEBUG - Starting datafeed loop 2022-06-17 11:30:47,424 - root - DEBUG - Message received from Nirbhey Jain: <div data-format="PresentationML" data-version="2.0" class="wysiwyg"><p>/notifications</p></div>

Actual Result:

Instead, these TLS issues occur:
Collection(Database(MongoClient(host=['localhost:27017'], document_class=dict, tz_aware=False, connect=True), 'jira_bot'), 'jira_bot') 2022-06-17 11:35:30,233 - root - INFO - Running bot application... 2022-06-17 11:35:30,237 - asyncio - DEBUG - Using selector: KqueueSelector 2022-06-17 11:35:30,488 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <activities.EchoCommandActivity object at 0x10e53dc60> 2022-06-17 11:35:30,488 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <activities.GreetUserJoinedActivity object at 0x10e53dcc0> 2022-06-17 11:35:30,493 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <gif_activities.GifSlashCommand object at 0x10e53dd20> 2022-06-17 11:35:30,494 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <gif_activities.GifFormReplyActivity object at 0x10e53dd50> 2022-06-17 11:35:30,498 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <login_activities.LoginSlashCommand object at 0x10e53e350> 2022-06-17 11:35:30,499 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <login_activities.LoginActivity object at 0x10e53e260> 2022-06-17 11:35:30,503 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <mytickets.MySlashCommand object at 0x10e53ea10> 2022-06-17 11:35:30,506 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <mytickets.TeamSlashCommand object at 0x10e56bfa0> 2022-06-17 11:35:30,509 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <mytickets.WatchedSlashCommand object at 0x10e53fbe0> 2022-06-17 11:35:30,511 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <mytickets.BlockerSlashCommand object at 0x10e568430> 2022-06-17 11:35:30,513 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <notifications_activities.UpdateNotificationsSlashCommand object at 0x10e53ea70> 2022-06-17 11:35:30,514 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <notifications_activities.UpdateNotificationsFormReplyActivity object at 0x10e56bee0> 2022-06-17 11:35:30,514 - symphony.bdk.core.activity.registry - DEBUG - Registering slash command with command=/hello, mention_bot=True 2022-06-17 11:35:30,514 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <symphony.bdk.core.activity.command.SlashCommandActivity object at 0x10e568790> 2022-06-17 11:35:30,514 - symphony.bdk.core.activity.registry - DEBUG - Registering slash command with command=/connect, mention_bot=True 2022-06-17 11:35:30,516 - symphony.bdk.core.activity.registry - DEBUG - Registering new activity <symphony.bdk.core.activity.command.SlashCommandActivity object at 0x10e568670> 2022-06-17 11:35:30,516 - symphony.bdk.core.service.datafeed.abstract_datafeed_loop - DEBUG - Starting datafeed loop 2022-06-17 11:35:31,020 - symphony.bdk.core.auth.bot_authenticator - INFO - Retrying symphony.bdk.core.auth.bot_authenticator.BotAuthenticatorRsa._authenticate_and_get_token in 2.0 seconds as it raised ClientConnectorSSLError: Cannot connect to host sup.symphony.com:443 ssl:default [Cannot create a client socket with a PROTOCOL_TLS_SERVER context (_ssl.c:801)]. 2022-06-17 11:35:33,372 - symphony.bdk.core.auth.bot_authenticator - INFO - Retrying symphony.bdk.core.auth.bot_authenticator.BotAuthenticatorRsa._authenticate_and_get_token in 4.0 seconds as it raised ClientConnectorSSLError: Cannot connect to host sup.symphony.com:443 ssl:default [Cannot create a client socket with a PROTOCOL_TLS_SERVER context (_ssl.c:801)]. 2022-06-17 11:35:37,754 - symphony.bdk.core.auth.bot_authenticator - INFO - Retrying symphony.bdk.core.auth.bot_authenticator.BotAuthenticatorRsa._authenticate_and_get_token in 8.0 seconds as it raised ClientConnectorSSLError: Cannot connect to host sup.symphony.com:443 ssl:default [Cannot create a client socket with a PROTOCOL_TLS_SERVER context (_ssl.c:801)]. 2022-06-17 11:35:38,751 - root - INFO - Ending bot application

Going through the trace, all errors occur within libraries that come with the BDK

Environment:

Python BDK version 2.3.0 on MacOS Catalina 10.15.7, running from a virtual PyEnv installation of 3.10.4. The project also has the following dependencies:
requirements.txt

Messages containing HTML entities such as   cause a MessageParserException

Parsing any message containing HTML entities fails in the function get_text_content_from_message:

https://github.com/SymphonyPlatformSolutions/symphony-api-client-python/blob/a15462a88ea21bc95c5ca28d92f5069c1a0d7369/symphony/bdk/core/service/message/message_parser.py#L24-L28

This issue is that difusedxml won't accept non-XML entities by default.

This is a particular issue because tables copy/pasted from Excel into a symphony chat render empty cells as  

For example:

Arrives as:

<div data-format="PresentationML" data-version="2.0" class="wysiwyg">
    <p>
        <table class="pasted-table">
            <thead>
                <tr>
                    <th>/bilat-barc-dev</th>
                    <th>Bid</th>
                    <th>Mid</th>
                    <th>Ask</th>
                </tr>
            </thead>
            <tbody>
                <tr>
                    <td>&nbsp;</td>
                    <td>1.114</td>
                    <td>1.893</td>
                    <td>0.291</td>
                </tr>
            </tbody>
        </table>
    </p>
</div>

Activity API

Goal is to implement a subclass of real_time_event_listener.py which overrides only on_message_sent method.

Implement:

• a class AbstractActivity with a matcher and on_activity methods.
• a CommandClass which reacts to on_message_sent only and parses the text inside the received PresentationML
• a PatternCommand class (subclass of the above) which filters based on a regex
• a SlashCommand class (subclass of the above) which filters based on a command and optional bot mention.

Take inspiration from the existing Java BDK Activity API.

Module system

The purpose of this issue is to introduce and specify a new module system for the BDK library as in Java: finos/symphony-bdk-java#479

We want to empower contributors to create their own modules or plugins in separate repositories, without necessary having to make their contributions directly in the BDK codebase.

We could think of something like:

async def run():
    async with SymphonyBdk(BdkConfigLoader.load_from_symphony_dir("config.yaml")) as bdk:
        extension = bdk.extension(MyExtension)

with the extension having access to the BDK configuration, the session/km tokens, etc.

Slash command args optional params and params with more than one string?

Support Question

Hello Symphony Developers! Thanks for the work on: https://github.com/finos/symphony-bdk-python/pull/284/files

It was helpful for us to grab @mention users. However we ran into a few issues below.

Borrowing off:

 @activities.slash("/echo {@mention_argument}")
        async def on_echo_mention(context: CommandContext):
            mentioned_user = context.arguments.get_mention("mention_argument")
            message = f"Mentioned user: {mentioned_user.user_display_name}, id: {mentioned_user.user_id}"

We tried:

@activities.slash("/echo {@mention_argument} {a_second_arg")
        async def on_echo_mention(context: CommandContext):
            mentioned_user = context.arguments.get_mention("mention_argument")
            message = f"Mentioned user: {mentioned_user.user_display_name}, id: {mentioned_user.user_id}"

But it appears that if {a_second_arg} is not passed the message won't expand. That's fine and we can get around it by declaring two decorators I guess one with the arg and one without but is there a way to make {a_second_arg} here optional if not passed the message will still trigger?

The second issue we faced was passing an arg/string with spaces in it. That appears to not work at all. In the example above if I were to:

@bot /echo @someone How are you doing? 

This would completely fail, I'm guessing it's thinking that any string with a space is considered another arg it can't process but any way to allow args with spaces? If there's a cleaner workaround to this I'd love to hear it as well!

We need this because we typically take the full string "How are you doing?" and do something with that value and it would be annoying to tell users to have to quote the string.

Thank you for all the awesome work on this, it's made our Symphony experiences much more fun and pleasurable to use.

ModuleNotFoundError: No module named 'symphony.bdk'

I am facing the below error when I try to run the template project. How can I fix this?

Setup :

$ mkdir bdk-bot && cd bdk-bot
$ yo @finos/symphony

$ python3 -m venv env
$ pip3 install -r requirements.txt
$ pip3 install symphony

Error :

$ source env/bin/activate
$ python3 -m src
Traceback (most recent call last):
  File "/Python-3.9.6/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Python-3.9.6/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "src/__main__.py", line 6, in <module>
    from symphony.bdk.core.activity.command import CommandContext
ModuleNotFoundError: No module named 'symphony.bdk'

Implement Datahose loop

Description
Bots are now able to create and read datahose feeds using the same DF2 API. Goal of this issue will be to add support also in Python BDK.

⚠️ BDK currently reuse the first listed feed ID. With datahose, we will need to filter in the fanout types of feeds only.

Acceptance criteria

In DatafeedLoopV2, filter out datahose feeds when listing existing datafeeds
Implement a DatahoseLoop using agent/v5/events/read endpoint to read datahose events

Add update message endpoint

We want to add new endpoint to update messages

list_user_audit_trail crashes when pagination cursors are not null

Description

When the api response contains pagination cursors (before and/or after) set to None/Null, list_user_audit_trail crashes with
TypeError: _from_openapi_data() missing 2 required positional arguments: 'before' and 'after'

This happens when no audit trails are returned for a user, or few than one page size.

Expected result

Optional pagination cursors are correctly mapped from api response when they are None

Datahose config: disabling datafeed in yml causes a NoneType error

Bug Report

Steps to Reproduce:

Use the following in config.yml:

datafeed:
enabled: false

datahose:
enabled: true
tag: fancyTag # optional tag that will be used when creating or reusing a datahose feed
eventTypes: # mandatory field, events you want to receive
- MESSAGESENT

Expected Result:

The datahose should be used going forward

Actual Result:

Logs shows this trace:

Traceback (most recent call last):
File "C:\Users\Alex.Nalin\PycharmProjects\SFDC-FED-NWI\main.py", line 728, in
asyncio.run(run())
File "C:\Users\Alex.Nalin\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Users\Alex.Nalin\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 649, in run_until_complete
return future.result()
File "C:\Users\Alex.Nalin\PycharmProjects\SFDC-FED-NWI\main.py", line 53, in run
async with SymphonyBdk(config) as bdk:
File "C:\Users\Alex.Nalin\AppData\Local\Programs\Python\Python310\lib\site-packages\symphony\bdk\core\symphony_bdk.py", line 102, in init
self._initialize_bot_services()
File "C:\Users\Alex.Nalin\AppData\Local\Programs\Python\Python310\lib\site-packages\symphony\bdk\core\symphony_bdk.py", line 117, in _initialize_bot_services
self._datafeed_loop = self._service_factory.get_datafeed_loop()
File "C:\Users\Alex.Nalin\AppData\Local\Programs\Python\Python310\lib\site-packages\symphony\bdk\core\service_factory.py", line 156, in get_datafeed_loop
if df_version.lower() == DatafeedVersion.V1.value.lower():
AttributeError: 'NoneType' object has no attribute 'lower'

Environment:

This issue occurs on all deployment, locally, via docker, on GKE...

Additional Context:

If removing the datafeed part and leaving the config.yml as below, then no error:

datahose:
enabled: true
tag: fancyTag # optional tag that will be used when creating or reusing a datahose feed
eventTypes: # mandatory field, events you want to receive
- MESSAGESENT

Can jinja2 dependency strict version matching be dropped?

Hi,

I noticed the PyPi package pulls in jinja2 as a transitive dependency with an exact version match. This makes it impossible to use the package in the same project together with others that require more recent versions of jinja2 themselves. Can the strict version requirement be dropped in favor of, say, >=2.x.x?

Best,

Jens

Search messages by Query endpoint

Implement these 2 missing endpoints:

https://developers.symphony.com/restapi/reference#message-search-get
https://developers.symphony.com/restapi/reference#message-search-post

Add Regular Expression Handler to @activities.slash

The Java BDK has supported regular expressions in its @slash handler since the start but the Python BDK did not. This creates a discrepancy when describing how to build command handlers that handle more complex use-cases such as catch-all scenarios between both languages.

// Valid in Java BDK
bdk.activities().register(slash("/Help .*", false, context -> {
    // Logic
}));

# Invalid in Python BDK
@activities.slash("/Help .*")
async def hello(context: CommandContext):
    # Logic

Search rooms function doesn't work in Python 3

https://github.com/SymphonyPlatformSolutions/symphony-api-client-python/blob/3e9a4f66cba05d1f9f1af2907e476641c8bbc6ee/sym_api_client_python/clients/stream_client.py#L150

always sets data to None as this method doesn't return a value

https://docs.python.org/3/library/stdtypes.html#typesmapping
update([other])
Update the dictionary with the key/value pairs from other, overwriting existing keys. Return None.

Datafeed v2 loop calls read datafeed with tag=None

Bug Report

#320 was supposed to remove the use of tags from the Datafeed v2 calls but instead is firing a literal tag=None call to the endpoint.

Steps to Reproduce:

• Start BDK Python project configured with MITM proxy
• Observe HTTP calls

Expected Result:

Assuming an existing datafeed already exists

POST   develop2.symphony.com /login/pubkey/authenticate
GET    develop2.symphony.com /pod/v2/sessioninfo
POST   develop2.symphony.com /relay/pubkey/authenticate
GET    develop2.symphony.com /agent/v5/datafeeds
POST   develop2.symphony.com /agent/v5/datafeeds/e3c5e89455404345b78b18218d7748cc_f_59089/read

Actual Result:

Since tag=None always returns an empty list, a new datafeed is created on every launch

POST   develop2.symphony.com /login/pubkey/authenticate
GET    develop2.symphony.com /pod/v2/sessioninfo
POST   develop2.symphony.com /relay/pubkey/authenticate
GET    develop2.symphony.com /agent/v5/datafeeds?tag=None
POST   develop2.symphony.com /agent/v5/datafeeds
POST   develop2.symphony.com /agent/v5/datafeeds/e3c5e89455404345b78b18218d7748cc_f_59089/read

Environment:

BDK Python 2.6.0