aio-libs / sockjs

SockJS Server

SockJS server based on Asyncio (PEP 3156)

sockjs is a SockJS integration for aiohttp. SockJS interface is implemented as a aiohttp route. Its possible to create any number of different sockjs routes, ie /sockjs/* or /mycustom-sockjs/*. You can provide different session implementation and management for each sockjs route.

Simple aiohttp web server is required:

[server:main]
use = egg:gunicorn#main
host = 0.0.0.0
port = 8080
worker = aiohttp.worker.GunicornWebWorker

Example of sockjs route:

def main(global_settings, **settings):
    app = web.Application()
    app.router.add_route('GET', '/', index)
    sockjs.add_endpoint(app, prefix='/sockjs', handler=chatSession)
    web.run_app(app)

Client side code:

<script src="https://cdn.jsdelivr.net/npm/sockjs-client@1/dist/sockjs.min.js"></script>
<script>
  var sock = new SockJS('http://localhost:8080/sockjs');

  sock.onopen = function() {
    console.log('open');
    sock.send('test');
  };

  sock.onmessage = function(e) {
    console.log('message', e.data);
    sock.close();
  };

  sock.onclose = function() {
    console.log('close');
  };
</script>

Supported transports

• websocket hybi-10
• xhr-streaming
• xhr-polling
• iframe-xhr-polling
• iframe-eventsource (EventSource used from an iframe via postMessage)
• iframe-htmlfile (HtmlFile used from an iframe via postMessage.
• jsonp-polling

Requirements

• Python 3.5.3
• gunicorn 19.2.0
• aiohttp https://github.com/aio-libs/aiohttp

Examples

You can find several examples in the sockjs repository at github.

https://github.com/aio-libs/sockjs/tree/master/examples

License

sockjs is offered under the Apache 2 license.

sockjs's Issues

Is this a SockJS protocol mistake?

We have a message const prefixes one of them is FRAME_MESSAGE_BLOB (https://github.com/aio-libs/sockjs/blob/master/sockjs/protocol.py#L48)

But in SockJS lib we didn't have this prefix: https://github.com/sockjs/sockjs-client/blob/0b4879d5e0b7f44d9f8accb28759fd965f70ea1f/dist/sockjs-0.3.4.js#L1086

I think that this is a typo.

Fix: FRAME_MESSAGE_BLOB = 'm'.

How to use multi worker

I use your demo app, i can't understand running app with multi worker.
Please help me!

Call async function based on example

Hi there,

The example is like "def chat_msg_handler(msg, session):", so my handler cannot be "async def chat_msg_handler". So is there any way to call an async function or do I have to re-do every async function removing the "async"?

About Using Websocket

How to send data from server client, by websocket?

Any simple example code?

Release on PyPI to support latest aiohttp

sockjs 0.5 doesn't work with aiohttp 2.0.x, but the current head version works. Let's release a 0.6 version.

Access to query strings (GET params) in user-defined endpoint handlers

SockJS (the client side) allows adding query strings to the endpoint URL.
Here, currently there is no way to access them (e.g., request.GET) inside user-defined endpoint handler coroutines. I'd like to access them! 👀

In the code, sockjs seems to invoke the user-defined endpoint handler in SessionManager.acquire() and _acquire() methods, which is called by transports' process() coroutines. SockJSRoute and transport subclasses have access to the aiohttp's request object but it is not passed to the user-defined handler via acquire method chains.

Maybe it is not a good idea to pass the whole request object to user-defined handlers due to design principles such as "separation of concern", but still I think it would be nice to have access to GET parameters.

RawWebSocketTransport reason?

Hi! I check the sockjs code and look at RawWebSocketTransport.

Confusion:

We have Session abstraction and _remote_message(msg) coro link which calling on new message from client.

Take a look at websocket transport link
and rawwebsocket transport link
.

As we can see in the first case we pass the dict object and in the secon case we pass the str.

Next. as we can see here the RawWebSocketTransport is not a part of the sockjs transports.
And uses as additional route here

I thik that the RawWebSocketTransport used for tests or something similar. And this is not a part of the sockjs lib.

What is the real purpose of the RawWebSocketTransport?

Is this project now support?

Hi, I see some opend issuse and pull requests.
Master branch is fail.

Can you tall something about state of this project?

Special session implementation with redis as storage?

Rename sockjs package to aiosockjs

I think it's good to have the same name for the library and the package.

Blob

Hi there,

How can I send a blob from sockjs-client to aiohttp-sockjs? I've tryied but there's just [object Blob] result. Right now I'm reading the data with File (window.File, window.FileReader), then encoding to base64, JSON.stringfying it and sending through sockjs, then decoding on server side. But that sometimes corrupts my stream, uses more CPU.. it's not the best way to do it, I guess...
I just need to send a webm file through websockets by chunks, then build it on server side, convert it, whatever.

Thank you, great lib :)

Increase default session timeout longer than default manager heartbeat interval

Currently SessionManager's default heartbeat is 25 seconds and timeout is 5 seconds.
So if there are no messages for a session within 5 seconds, the connection is closed.

If those are intended defaults, I would like to know why and it should be documented explicitly.
If not, session timeout should become longer than heartbeat intervals by default to avoid confusion of new sockjs package users.

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

sockjs client

it is very easy to create websocket client, but it might be possible to create other client transports as well.
would anyone use it?

The default session timeout is less than the heartbeat time.

class SessionManager(dict):
    ...
    def __init__(self, name, app, handler, loop,
                 heartbeat=15.0, timeout=timedelta(seconds=10), debug=False):

As a result, when SessionManager._heartbeat_task() is called first time then a session already is expired. When the session is expired, it closes. I think that it is not good :)

I looked at realisation of SockJS for Twisted (https://github.com/DesertBus/sockjs-twisted). In this package option timeout is only used to identify the problems during the connection. After the connection is established, this option is not used.

PS: sorry for my English

Relicense?

aiosockjs is licensed under MIT, but aiohttp has Apache 2 license.
Maybe change the state in early stage?

An error occurred when trying to install sockjs 0.1

$ easy_install sockjs
...
Writing /tmp/easy_install-7daqeug7/sockjs-0.1/setup.cfg
Running sockjs-0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install...
error: [Errno 2] No such file or directory: 
       '/tmp/easy_install-7daqeug7/sockjs-0.1/CHANGES.txt'

Please, add the file MANIFEST.in with the following contents:

include README.rst CHANGES.txt

And make a new release of sockjs.

Specify CORS settings

It would be nice to be able specify per-site CORS settings for added security, perhaps through integration with aiohttp_cors. Would there be interest in a PR in that direction?

Correct doc (deprecated warning from aiohttp)

The example in readme contains the call of make_handler https://github.com/aio-libs/sockjs/blame/master/README.rst#L30.
But it is deprecated:

Application.make_handler(...) is deprecated, use AppRunner API instead

Please correct example.

Let's put `sockjs` under the `aio-libs` org on PyPI

See the details @ https://github.com/orgs/aio-libs/discussions/26.

For this, we'll need somebody with Owner privileges to either give me access on PyPI (the username there is the same — webknjaz) or be invited to the org by me (for that I'd need the username of such an individual).

As a bonus, this will also allow us to set up secretless publishing from GHA to PyPI and get rid of the in-repo secrets.

cc @pahaz @Cykooz

Incorrect route names

Trying to run a chat example with aiohttp == 18.3

> python chat.py 
Traceback (most recent call last):
  File "chat.py", line 33, in <module>
    sockjs.add_endpoint(app, chatSession, prefix='/sockjs/')
  File "/home/gleb/.python-virtualenvs/sockjs/lib/python3.5/site-packages/sockjs/route.py", line 59, in add_endpoint
    hdrs.METH_GET, prefix, route.greeting, name=route_name)
  File "/home/gleb/.python-virtualenvs/sockjs/lib/python3.5/site-packages/aiohttp/web_urldispatcher.py", line 481, in add_route
    self.register_route(route)
  File "/home/gleb/.python-virtualenvs/sockjs/lib/python3.5/site-packages/aiohttp/web_urldispatcher.py", line 452, in register_route
    raise ValueError('Incorrect route name value, '
ValueError: Incorrect route name value, Route name should be a sequence of python identifiers separated by dot or column

PyPI release?

I see there's a placeholder on pypi. It would be great to see a preliminary release. (I'd like to use this for testing.)

Closing session in the server leads to 100% cpu usage and make server hang

Server:

import asyncio
import sockjs

from aiohttp import web


loop = asyncio.get_event_loop()
app = web.Application(loop=loop)


from sockjs import MSG_OPEN, MSG_MESSAGE, MSG_CLOSE, MSG_CLOSED 

def handler(msg, session):
    tp = msg.tp
    print("tp", tp, msg, session)
    if tp == MSG_MESSAGE: # onmessage
        print("session {} Got message: {}".format(session.id, msg.data))
        session.close()
        pass
    elif tp == MSG_OPEN: # onopen
        print("session {} Open".format(session.id))
        pass
    elif tp == MSG_CLOSE or tp == MSG_CLOSED: # onclose
        print("session {} {}".format(session.id, "Closing" if tp == MSG_CLOSE else "Closed"))
        pass

sockjs.add_endpoint(
    app,
    handler,
    name='skjs',
    prefix="/chat_sockjs",
    sockjs_cdn='https://cdn.jsdelivr.net/npm/sockjs-client@1/dist/sockjs.min.js',
)

if __name__ == '__main__':
    web.run_app(app, host='0.0.0.0', port=9321)

Client:

<!doctype html>
<html>
  <head>
    <meta charset='utf-8'>
  </head>
  <body>
    <div id="connmsg" style="display: none">Connected!</div>
    <pre id="last_delivered"></pre>
    <textarea id="sender" rows=4 style='width:100%'></textarea>
    <button onclick='return send()'>Submit</button>
    <script src='https://cdn.jsdelivr.net/npm/sockjs-client@1/dist/sockjs.min.js'></script>
    <script>
var SOCKJS_URL = 'http://localhost:9321/chat_sockjs';
var sock = new SockJS(SOCKJS_URL, null, {transports: ['websocket']});
sock.onopen = function () {
  document.getElementById("connmsg").style.display = 'block';
};

sock.onmessage = function (msg) {
  var node = document.createTextNode(msg.data);
  var el = document.getElementById("last_delivered");
  el.innerHTML = '';
  el.appendChild(node);
};

sock.onclose = function () {
  document.getElementById("connmsg").style.display = 'none';
};

function send() {
  var el = document.getElementById("sender");
  sock.send(el.value);
  el.value = '';
  return false;
}
    </script>
  </body>
<div>

You can reproduce the issue with the above codes. Make server run on port 9321 and open the html file and send something. You could easily see cpu usage soaring up to 100% and server is unresponsive to any message in further.

In some cases, you may want server to close session (e.g. when it receives a frame of unrecognized format). However if you do so, something goes wrong in the internal code of sockjs, it gets trapped to an infinite loop and server hangs. This is surely unacceptable in any asynchronous server settings.

I followed the code flow with pdb and found out:

1. When underlying websocket connection is asked to be closed, the socket state turns into MsgType.Closing.
2. Then, ws.receive() is made to always return ClosingMessage. This happens inside aiohttp code base.
3. As a result, ws.receive() in coroutine client() gets a closing message. But because nonthing handles the case, it reaches ws.receives() once again without breaking the loop or awaiting event, and the same thing happens again and again. Take a look at sockjs/websocket.py:35-61.

I guess the easiest way to fix is just adding handling case for MsgType.closing condition. But, I couldn't work it out as I don't understand the code base currently. But, the thing is, until it gets done right, sockjs server is almost unusable (or maybe I should stick to xhr-streaming). I'm willing to do PR for this issue if you could help me :)

Can it run on windows?

The example states dependency on gunicorn, which does not run on MS Windows.

Is there any alternative to run on MS Windows? aiohttp seems to allow that.

GZIP compression

Is it possible to enable per-message compression with sockjs?

https://www.igvita.com/2013/11/27/configuring-and-optimizing-websocket-compression/

Use msg.type instead of msg.tp

msg.tp is deprecated and will be removed in next aiohttp release

Increase test coverage

81% is a shame

cdn.sockjs.org is gone

The hardcoded sockjs_cdn URL http://cdn.sockjs.org/sockjs-0.3.3.min.js is no more valid:

$ curl http://cdn.sockjs.org/sockjs-0.3.3.min.js
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>3BB0BB487751DE22</RequestId><HostId>j8A9HZa9CXG2MqNWOX4ROjCuaCaDyrakWCQIYavU4o4lm6JXjpB0j7Epj81bkisFUu2lrki+dxo=</HostId></Error>

According to sockjs.org, the new CDN URL should be https://cdn.jsdelivr.net/sockjs/0.3/sockjs.min.js (or https://cdn.jsdelivr.net/sockjs/1.1/sockjs.min.js if 0.3 is not specifically required.)

Race condition during iteration over sessions

I am using a coroutine to track events generated by DB and push them to SockJS clients.
That coroutine is scheduled by aiohttp.web.Application().on_startup.append() method.

Recently, I have found this exception in my logs:

Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "app.py", line 142, in subscribe_to_changes
    await watch_changes(app=app)
  File "app.py", line 104, in watch_changes
    for active_session in sockjs_session_manager.active_sessions():
  File "/usr/local/lib/python3.5/dist-packages/sockjs/session.py", line 381, in active_sessions
    for session in self.values():
RuntimeError: dictionary changed size during iteration

See #217

Accessing HTTP request from session

SockJS sessions are initiated with a HTTP handshake.
Is there any way to access this request from the Session object?

It would be useful to be able to access the initial request object from Session to do things like identify and authenticate the client with the HTTP headers.

iframe transports not working

Communication via iframe transports is broken because of SyntaxError in the embedded html, which is supposed to call SockJS.bootstrap_iframe().

With sockjs server running on port 23299, you can reproduce it with the following stub (Tested in python 3.6.1, Chrome 61 and Firefox 55):

SockJS("http://127.0.0.1:23299/sockjs_path", null, {transports: 'iframe-htmlfile'})
>> Uncaught SyntaxError: Invalid or unexpected token iframe.html:9 

At line 62 in protocol.py, there are leading backslashes before a pair of parentheses where SyntaxError occurs. To make it work, you can just remove these backslashes.
SockJS.bootstrap_iframe\(\);

Publish docs

Now the project has docs folder -- but shame on us, the content describes pyramid_sockjs instead of sockjs.

We need rewrite documentation and publish it on rtfd.io

Broadcast question

Hi there,

I have a dict like

companies = {
{
"company": "test",
"viewers": []
},
{
"company": "test_2",
"viewers": []
},
}

and I append the ws session to the viewers. But when I do like
for viewer in companies[1]['viewers']:
test = { 'msg': 'test' }
viewer.manager.broadcast(resp)

everyone connected to that ws endpoint receives the msg, despite of which company it is.
How can I control this better?

Update all `secret: ` values in `.travis.yml`

Do like this:

REPO_NAME={{ YOUR_REPO_NAME }}
travis encrypt -r "aio-libs/${REPO_NAME}" --api-endpoint 'https://api.travis-ci.com/'

Ref: https://github.com/orgs/aio-libs/teams/admins/discussions/9

The library maintainer is wanted

I don't use the sockjs for 5 years.
I have no time/intention to maintain the library.

We need a champion to pick the flag up.
The biggest problem that I see is the library test suite: it is entirely built on top of aiohttp mocking.
aiohttp envolves, mocks become obsolete, tests fail.
Reliable tests should be rewritten in the functional style, using aiohttp_client / aiohttp_sever pytest fixtures.
The change is obvious and even relatively easy but tedious: all supported transports are affected.
I have no capacity to do it, but happy to help with review and other support.

If somebody wants to be the project maintainer -- please let me know by a comment for this issue.

Error when use middleware with sockjs

In your examples chat.py you don`t use middleware. But I try to create simple chat with authorization and it fails in middleware. When commented middleware functions, everything works fine

TypeError: object generator can't be used in 'await' expression

Could you write example with middleware and for example with aiohttp_session

Reuse session.id to persist connections across refreshes

Hi all,

I don't know if what I've encountered is a bug or the expected behaviour. I'll need to hear opinions on this.

Essentially, a web app has been created that connects to the sockjs server a fixed URL and port. Additionally, we have a case where we would like to reconnect to the same socket session (id) after a refresh or disconnect (based on a unique user id).

When we leave the sockjs server to generate the UUID for each refresh we get a connection and we can send and receive messages correctly. However, if we provide a fixed session.id the first connection to the sockjs server works perfectly. A refresh on that page produces the following:

1. Multiple repeated 101 upgrades but no connection for approximately 20s.
2. Eventually when the socket connection is established no messages are sent back to the server. It's as if the connection is there but broken.

Please excuse my ignorance if I don't understand a concept here, but, seeing as the session.id can be specified why would this approach not work?

websocket transport

websocket transport doesnt work with browser, while it passes sockjs-protocol-dev.py tests

Avoid "except Exception" or re-raise them when possible

I have run into some nasty corner cases where my code is "slightly" broken but cannot see what's wrong.
Currently many places (example 1 and 2) catches "Exception" and wraps the exception information into CLOSE frames with exception logging.
If this is combined with a missing argument in onclose(), which may happen quite frequently due to the lack of documentation/example in the client-side Javascript, it is hard to debug what is the real problem. (In fact, client-side onclose() handler gets a pair of error code and message. But nowhere this is described in the client docs!)
In my case, it was even harder because my application code was running under a custom logger setting which only exposed specific logging namespaces but "sockjs".

I think it is always a better practice to leave handling "any kind of exception" for application writers instead of library writers. Libraries should catch only specific exceptions and re-raise others after doing their own clean-ups, whenever possible. Otherwise, such behavior should be explicitly documented.

I need to access my custom match_info?

e.g:
sockjs.add_endpoint(app,chatSession,name='chat',prefix='/chat/{uid:\d+}')

How do i get "uid"

What about graceful shutdown?

SessionManager.clear could be done on cleanup. But as I tested, current code breaks cleanup process (if active websocket connections opened ) and hangs server (app.on_cleanup never called) and shutdown never happens. I need some clue what is happening and how this can bee fixed.

python3.7 support

This lib doesn't work at python3.7, because 'async' became keyword (https://docs.python.org/3/whatsnew/3.7.html). The problem is in the session.py (line 10):

    ensure_future = asyncio.async

The python3.7 says SyntaxError: invalid syntax

Need GC for expired sessions

Build FAILED!

Hi! I submit new pull requiest. After ci tests pass I see many errors:

Traceback (most recent call last):
  File "/home/travis/build/aio-libs/sockjs/tests/test_route.py", line 213, in test_fail_transport
    'transport': 'test', 'session': 'session', 'server': '000'})
  File "/home/travis/build/aio-libs/sockjs/tests/test_route.py", line 52, in make_request
    False, False)
TypeError: __new__() missing 1 required positional argument: 'compression'
-------------------- >> begin captured logging << --------------------
asyncio: DEBUG: Using selector: EpollSelector
--------------------- >> end captured logging << ---------------------

This is not a my pull request problem.

Plese rebuild CI build.

SessionManager.clear() was never yielded from

Research:
__del__ at line call self.clear() coroutine line.

I think this is quite normal, but I have still some internal question.
What do you think about this? Is this a normal behavior or bug?