using this with an NAF project? about dialog

[vincentfretin]

I followed the different Hubs PRs, so I can reply to some of your questions.
(1) The adapter lives there https://github.com/mozilla/hubs/blob/master/src/naf-dialog-adapter.js and replace naf-janus-adapter
(2) The networked-aframe that Hubs uses is https://github.com/MozillaReality/networked-aframe.git
(3) As my understanding, I guess this node program replace completely janus and janus-sfu plugin
(4) I don't know

[vincentfretin]

Just looked into the code. I found the following.
janus-plugin-sfu allowed the connection to a room if you didn't configure an auth_key in the plugin (so it didn't verify the JWT token).
https://github.com/mozilla/janus-plugin-sfu/blob/598359f4339e08f43c2781488f457582113c8aab/src/lib.rs#L407
In dialog defining the authKey is mandatory.
So you need a valid JWT token containing {join_hub: bool, kick_users: bool}
see the structure in rust https://github.com/mozilla/janus-plugin-sfu/blob/598359f4339e08f43c2781488f457582113c8aab/src/auth.rs#L12-L15 so I think those two booleans are the only thing you need in the JWT token for dialog too.

kick_users permission defined in reticulum: https://github.com/mozilla/reticulum/blob/0cde188f38f6ac4677ae261da178f65e6ea5f72d/lib/ret/hub.ex#L559
kick_users check in janus: https://github.com/mozilla/janus-plugin-sfu/blob/598359f4339e08f43c2781488f457582113c8aab/src/lib.rs#L494
kick_users check in dialog: https://github.com/mozilla/dialog/blob/007a5c8f02f73524cefcd1b7701f06087e083a04/lib/Room.js#L1421

[vincentfretin]

Obviously you need your own backend where the user authenticate, and return a JWT token. Then on the naf adapter, you need to set a client id and jwt token, on hubs this is done here: https://github.com/mozilla/hubs/blob/bf78a09218109ede68a41c7fa504168011983d1a/src/hub.js#L1455-L1456
If you didn't read it already, look at Hubs-Foundation/hubs#2534 for some pointers where is the code in hubs that connect to the webrtc server.

[vincentfretin]

I have a naf project where I use Janus 0.9.5 with janus-plugin-sfu master (commit mozilla/janus-plugin-sfu@3694c36), and js client naf-janus-adapter 3.0.20 (not master with version 4.0.x that introduced new syncOccupants behavior)
For chat and component updates, this uses by default "datachannel" transport, see
https://github.com/mozilla/naf-janus-adapter/blob/v3.0.20/src/index.js#L81-L82
If you want to send private message described in the api doc with the whom parameter https://github.com/mozilla/janus-plugin-sfu/blob/master/docs/api.md#data (the whom parameter is set automatically by naf-janus-adapter https://github.com/mozilla/naf-janus-adapter/blob/v3.0.20/src/index.js#L953 when you use NAF.connection.sendDataGuaranteed(toClientId, 'chatbox', data) for example) you need the "websocket" transport. NAF.connection.adapter.reliableTransport = "websocket". If you use transport "datachannel" the private message will be send to everyone in the room. :-)

Some years ago, I guess Hubs did use this.
Hubs now uses a Phoenix app (reticulum), chat and component updates go through the Phoenix websocket, this is configured here https://github.com/mozilla/hubs/blob/d7b33f24f8a76b27462bb3f6f42051a55a913dc2/src/hub.js#L637-L638

With janus-sfu-plugin it created two datachannels to support chat and components updates https://github.com/mozilla/naf-janus-adapter/blob/v3.0.20/src/index.js#L429-L438
With the new naf-dialog-adapter it doesn't create them, you may need to create them yourself or use an external app like phoenix to communicate the chat and components updates via websocket like Hubs do. I'm not sure if the new dialog sfu supports creating datachannels or if it can only handle the audio/video via WebRTC.

[vincentfretin]

Btw, I don't know if you found the example at https://github.com/mozilla/naf-janus-adapter/blob/master/examples/index.html
this is not linked in the README and both the example index.html and the code snippet in the README use really old versions of aframe and networked-aframe. I completely missed this example at first, I found the relevant element to set the audio with setLocalMediaStream directly in hubs.js. Ah ah.
For networked-aframe at https://github.com/MozillaReality/networked-aframe you need to include it with npm or build your own dist version to have the latest version. The dist bundle in the repo wasn't updated with the latest commits.

[vincentfretin]

About horizontal scaling, you may find some answers here Hubs-Foundation/hubs#2887
As far I know, if you use the "websocket" or "datamanager" transport of janus-plugin-sfu, you are limited to one node. If you want to scale horizontally you need a phoenix app that use distributed elixir or a nodejs app with redis which direct all participants of a room to the same janus instance (or this new dialog sfu).

A note about version 4.0.x of naf-janus-adapter, this version changed the syncOccupants behavior to fix some issues when using another app for user presence (phoenix app like hubs reticulum, see how it uses it here https://github.com/mozilla/hubs/blob/d7b33f24f8a76b27462bb3f6f42051a55a913dc2/src/hub.js#L1288-L1298), so you may find the latest version useful, see description of mozilla/naf-janus-adapter#100

[vincentfretin]

As far as I understand, Hubs server reticulum uses the guardian elixir module https://github.com/ueberauth/guardian to generate the jwt perms token here https://github.com/mozilla/reticulum/blob/e50b16244a3ef6b41617fd4c81c0537dc2830306/lib/ret/perms_token.ex#L27 that works for 5 minutes. This perms token is different that the one for authentication that lives longer and is set in localStorage.
It uses the private RSA 2048 key to sign the token with the RS512 algorithm and I think perms variable is here { join_hub: true, kick_users: true} you can see in the Rust struct.
The Rust code want a JWT token encoded with the RS512 algorithm as well https://github.com/mozilla/janus-plugin-sfu/blob/14a33464726166fa0d3a20bd452ad05d2f7c53a6/src/auth.rs#L19

In the janus conf auth_key should point to the public RSA key in DER format
https://github.com/mozilla/hubs-ops/blob/0c27ac2bf5b29a3988ce0a5e85f4903fb66177d9/plans/janus-gateway/habitat/config/janus.plugin.sfu.cfg#L3
You can generate a 2048 RSA key with ssh-keygen -t rsa -b 2048 and transform to DER format with openssl commands, see instructions here: mozilla/janus-plugin-sfu#29

If you already have a node app on the server, you can try creating a JWT token with https://github.com/auth0/node-jsonwebtoken
You can see an example with node and express here https://stormpath.com/blog/nodejs-jwt-create-verify
The following should be enough to create the token:

var jwt = require('jsonwebtoken');
var privateKey = fs.readFileSync('private.key');
var token = jwt.sign({ join_hub: true, kick_users: true}, privateKey, { algorithm: 'RS512' }, expiresIn: 300 });

When your user is logged in with your node app, return this token and set it with adapter.setJoinToken(token).

[kylebakerio]

Excellent, thanks. That's enough to make getting started and going in the dark a bit possible.

(Any other responses are welcome as well, of course.)

[kylebakerio]

I really am not familiar with this project in detail; what you've shared is extremely helpful, but I'm still lacking a lot of context about how using JWT tokens with this project works. If someone behind this project could spend 30 minutes writing a quick readme update that gives context to an outsider, it would go a long way to making this project more accessible.

Thanks so much already for all you've given, though, I'll keep trying in the meantime.

[kylebakerio]

Thanks, that's the most helpful summary of a collection of tidpits I'd pieced together about all of that over the last weeks that I've seen in one place, along with some stuff that I hadn't seen clearly anywhere.

I'm very close to having this whole janus setup working, but something isn't quite right. I see the adapter is correctly pinging my janus server, but it gets an error back (I think from the janus sfu plugin), which reads data did not match any variant of untagged enum OptionalField, which is not really very readable. :)

Any ideas?

Also, I do need to use NAF.connection.broadcastData() (or an equivalent), but I haven't understood everything you said about the specifics of the datachannel transport. It seems it's a detail about how to properly replicate sendData and broadcastData properly? Are you saying the default behavior is such that everyone's client gets the sendData() calls even though they aren't addressed to them, and that this is why 'whom' has to be added?

This is super helpful, if there's a better way to have this conversation than here, let me know, picking your brain for a few minutes would save me a lot of headache. I'd like to document this process somewhere when I'm done so it's available and out there for once all together. (I see you're on the aframe slack, I'll ping you there, though I don't know if you're still active or not.)

[vincentfretin]

I'm glad you're making progress. I remember having the data did not match any variant of untagged enum OptionalField error, doing a "OptionalField" search in the rust code https://github.com/mozilla/janus-plugin-sfu/search?q=OptionalField you see this is related to decoding a json message. I think I got the error when I tried to set the JWT token with adapter.setJoinToken. At one point I gave up to understand what it wanted, I didn't even know what was JWT at that time so I tried just to send like a json string with adapter.setJoinToken(JSON.stringify({join_hub: true})) I think but a JWT is more complicated than that. At the end I didn't use adapter.setJoinToken and left auth_key commented in janus.plugin.sfu.cfg as I don't currently have a way to authenticate user and I have no backend.

For the transport, you're right. For sendData or sendDataGuaranteed defined in https://github.com/mozilla/naf-janus-adapter/blob/v3.0.20/src/index.js#L933-L939 the default "datachannel" will send the message { clientId, dataType, data } to janus and janus will broadcast it to everyone in the room, so be careful. I saw this behavior when adding a feature to send a text message to a specific user in the room and saw that everyone received it. :-)
The api documented at https://github.com/mozilla/janus-plugin-sfu/blob/master/docs/api.md#data with the whom to send a private message apply only to the "websocket" adapter, like you see in the sendData that's the only one using whom.
You're fine with the "datamanager" transport if you don't plan to add the feature to send a private message to your app.

I'm not connected anymore on slack aframe, but I will open it again.

[vincentfretin]

You will need the JWT to work if you want the kick a user feature though. I currently don't have it. :-)
And of course without JWT it's like your janus rooms are open to everyone to listen (if someone knows the room id), so use it without JWT only for public conversations :-)

[kylebakerio]

I see. I'm tempted to put together a repo explaining all this clearly and giving a functional readme on the state of things after all of this. That's all invaluable information--I had heard pieces here and there, but it's nice to see it all put together as a single coherent narrative.

[vincentfretin]

The data did not match any variant of untagged enum OptionalField error can show up if you didn't call setClientId.
Here is a simple code:

function genClientId() {
  let num = '';
  for (let i = 0; i < 16; i++) {
    num += Math.floor(Math.random() * 10).toString();
  }
  return num;
}

AFRAME.scenes[0].addEventListener("adapter-ready", ({ detail: adapter }) => {
   const clientId = genClientId();  // generate a random 16 characters string, but you can use a uuid4 for example
   adapter.setClientId(clientId);
   //adapter.setJoinToken(...);
   const constraints = {audio: true};
   navigator.mediaDevices.getUserMedia(constraints).then((mediaStream) => {
     NAF.connection.adapter.setLocalMediaStream(stream).then(() => {
        // Note that networked-scene audio:true option has no effect with the janus adapter
        NAF.connection.adapter.enableMicrophone(true); // set it to false if you want to be muted initially.
      });
   });
 });
});