OpenTok / TokBox support about bowser

[superdump]

1. navigator.getUserMedia should be present and it seems we also overwrite navigator.webkitGetUserMedia: https://github.com/EricssonResearch/openwebrtc/blob/master/bridge/client/webrtc.js#L1224 (note there and just below, also note the FIXME)

2. Not yet but that is a good idea. I think we may be able to put that together and it would help in general.

3. See above. bridge/client/webrtc.js it seems.

Thanks for the feedback, we're definitely interested in either fixing bugs to allow interoperability with higher-level WebRTC application frameworks or to investigate them to get them to fix mistakes in WebRTC API usage.

[stefanalund]

If you are comfortable, and can, share a URL for your app we can also take it for a spin.

[noodlemichaelmartin]

Unfortunately, I cannot - it's not public.

I can say that we're using the standard implementation of TokBox, which is basically a paid version of the "OpenTok" standards (they host the TURN/STUN services and all the streams are routed through their Media server to enable things like recording and changes in frame rates or stream resolution).

If you want to test the libraries themselves, they do let you setup a free 30-day trial developer account prior to setting up billing - https://tokbox.com/

The only thing that I do in my implementation that's not present in the out of the box TokBox implementation, is the above JS detection to put a custom message into the DOM for the user. When I remove the JS detection code, it causes a JS error that prevents the rest of the JS on the page from executing. I can't seem to see what the error is, though. It should be noted that while the navigator object is defined, neither the navigator.mozGetUserMedia nor the navigator.webkitGetUserMedia are not defined, when tested via JavaScript in Bowser.

Is it possible that this is just a compilation issue of some kind or is it just that it's throwing exceptions / errors that I just can't see within Bowser?

[stefanalund]

You can use the Developer Tools in Safari to debug errors in Bowser (UIWebViews):

https://github.com/EricssonResearch/bowser#debugging-webrtc-scripts

[noodlemichaelmartin]

I'm developing this on an iPhone, which does have Inspection enabled, but I don't see any additional options or tools that seem to affect Bowser. Also, Bowser does not run in the simulator, so I can't use any debugging tools there either..

When running the JS in any platform that currently does not support WebRTC (IE, Safari, etc), it does not produce any actual JavaScript errors - it's accurately detected either by my JS (the IF statement above) or, even if I remove that IF statement, it's still cleanly detected by the TokBox JS libraries and informs the user properly.

In Bowser, when I omit my JS detection (the IF statement above) code, it instead produces a JS error or exception that I cannot see, which breaks the page. This implies that the library is likely incompatible with Bowser, but more importantly both navigator.mozGetUserMedia and navigator.webkitGetUserMedia are still "undefined" when I check for them in my detection code. So, if those two methods are "undefined", then it stands to reason that it's still not going to work anyways.

I compiled Bowser from the Git repository and did not modify it, with the exception of setting the developer account in the settings. Is there maybe something missing?

[noodlemichaelmartin]

Actually, additional information - that error was due to something unrelated. Now that it's suppressed, the only thing it's telling me is that "This browser is not compatible with WebRTC" from the TokBox platform.

[superdump]

So the getUserMedia API is found now? But TokBox says that the browser is not compatible with WebRTC? Do you know if they do any user agent sniffing?

[noodlemichaelmartin]

No, that is not correct - GetUserMedia is not found (it's undefined) - TokBox is displaying that message because it cannot find GetUserMedia.

The error that I mentioned above ("In Bowser, when I omit my JS detection (the IF statement above) code, it instead produces a JS error or exception that I cannot see, which breaks the page.") was unrelated TokBox, it was a bug in my code that I had never seen, because my JS detection code was always in the way of it. When I fixed the error in my JS, the TokBox platform was able to load normally and it agreed with my JS detection code and said that Bowser was not WebRTC compatible (because GetUserMedia was not found).

[superdump]

Does this getUserMedia test app or our point-to-point audio/video test app work?

[stefanalund]

Looks like the Google getUserMedia test app has moved to http://googlechrome.github.io/webrtc/samples/web/content/getusermedia/gum/

[noodlemichaelmartin]

Tested with that sample GUM page above and it does work, however, when I load my page, none of that seems to be defined.

In fact, I added this to the top of all of my other JS includes in to see if maybe one of them was overriding the function directly, but instead, each of the following alerts "undefine" for each of the alert lines below):

<!doctype html>
<html>
    <head>
    <script type="text/javascript">
        alert('before ANY other scripts IN theme.tpl...');
        alert(navigator.getUserMedia);
        alert(navigator.mozGetUserMedia);
        alert(navigator.webkitGetUserMedia);
        alert(window.webkitRTCPeerConnection);
        alert(window.mozRTCPeerConnection);
    </script>
   ...

Is there something I'm missing here or does it just look like none of this is defined?

[noodlemichaelmartin]

Ok, so now that I've ripped apart a non-minified version of opentok.js from TokBox, I can see where it's truly failing:

1. The first failure is on Browser version. It's detecting webkit version 27, but a minimum of webkit 34 is required.

2. It fails when checking the capabilities of webkitRTCPeerConnection, as both of these conditions of this if statement return false:

if (typeof(window.webkitRTCPeerConnection) === 'function' && !!window.webkitRTCPeerConnection.prototype.addStream)

If I had to guess, I'd say that the reason is entirely due to the webkit version.

An chance of upgrading to webkit 34 or higher?

[stefanalund]

That's easy actually: The UserAgent is set in BowserAppDelegate. It's actually set to 'not Chrome' but I guess that is ignored :-)

[noodlemichaelmartin]

Yes, I see that #define KUserAgent @"xxxx" line, but that's not the only issue.

In opentok.js, I actually removed the "return" part of the if statement referenced in point # 1 above so that the function just ignores the version requirement (temporary hack for debugging). However, even ignoring the version deficiency, as it continues with the requirement checks, it then fails on the very next requirement (listed as point # 2 above).

I was only speculating that if the webkit version is actually, physically 27, then perhaps the functionality it's trying to use is not present?

[Rugvip]

if (typeof(window.webkitRTCPeerConnection) === 'function' && !!window.webkitRTCPeerConnection.prototype.addStream)

This check fails because addStream is not a property of the prototype in our implementation, but instead assigned directly to the peer connection object.

Regarding the webkit version it will be whatever the iOS WebView is using. We are building a part of WebKit as a dependency (JavaScriptCore), but that's only being built for Linux and Android to be used as JavaScript engine.

[noodlemichaelmartin]

So then the nature of the check is technically invalid? As in they're assuming it's defined in the prototype, but you're just attaching it to the object directly? If that's the case, I can hack that...

[noodlemichaelmartin]

I can see where it's referencing the .addStream() method, but I can't seem to locate where you've attached that method. I'll just manually assign window.webkitRTCPeerConnection.prototype.addStream = [the function], if I can locate said function. How can I reference it?

[Rugvip]

window.webkitRTCPeerConnection.prototype.addStream = function() {} should be enough, it'll use the correct function anyway

[noodlemichaelmartin]

Adding window.webkitRTCPeerConnection.prototype.addStream = function() {} to the JS before it checks the requirements, basically only allows you to get past the requirement checks. When it tries to load the video chat box, it never actually connects or does anything. I get the normal loader box that it displays before you grant permissions to use the camera/microphone, but it never asks for permissions and it never loads the video stream. Attached a screen shot of what I see in Bowser and also what I see in FireFox or Chrome at the same URL.

[noodlemichaelmartin]

I should point out that it IS detecting the number of streams being published and will show multiple video boxes for each browser that's got the page loaded into it, but it's never able to render the video streams inside the boxes.

[Rugvip]

are you rendering a self-view or only remote streams?

[noodlemichaelmartin]

Both - It dynamically detects if new streams are published. In fact, right now, I was testing - I closed all browsers that had it open, opened the test page in Bowser and it had only 1 box (the larger, main box) and then a waited a few seconds. Then, I opened the page on my laptop and started streaming. On my phone, it rendered another (smaller) video box below the main one, as expected. Problem is that neither of them are physically rendering the stream - locally or remotely. It appears that the normal data exchanges and normal logging events are being fired back and forth, as designed, but the rendering of the video streams is just not happening. I'm guessing it's still related to the addStream issue that we've been discussing, but i could be wrong - it could be related to something else that I'm not seeing.

[stefanalund]

Are you using the latest OpenWebRTC? A couple of important interoperability bugs were fixed about 2 days ago: https://github.com/EricssonResearch/openwebrtc/wiki/Building-OpenWebRTC#update-openwebrtc

[noodlemichaelmartin]

Attempting a re-build now, will continue to test/debug. After trapping some errors with try/catch, I can get it to publish a video stream locally, but none of the peer-to-peer or routed connections are working, nor remote streams rendering locally. I'll test after the rebuild and continue to debug. Thanks for the update.

[noodlemichaelmartin]

Going back to our previous point of discussion, it appears that none of these have a function called AddStream, either natively or in the prototype, so where is your addStream function?

These are all undefined:

webkitRTCPeerConnection.prototype.addStream
RTCPeerConnection.prototype.addStream
webkitRTCPeerConnection.addStream
RTCPeerConnection.addStream

[Rugvip]

It's assigned once the object is created, try new webkitRTCPeerConnection({iceServers:[]}).addStream

[noodlemichaelmartin]

This is difficult to put in a "small" post, but I'll try.

First, @Rugvip, I hacked the RTCPeerConnection objects to have the addStream function you referenced above:

            if(window.webkitRTCPeerConnection) {
                var tempHack = new webkitRTCPeerConnection({iceServers:[]});
                window.webkitRTCPeerConnection.prototype.addStream = tempHack.addStream;
                window.RTCPeerConnection.prototype.addStream = tempHack.addStream;
            }

I'm not 100% sure if it worked yet, because I'm iterating through the TokBox/OpenTok code and finding other issues, so this might have solved a problem, but I can't be sure yet.

Prior to debugging further issues, the video and audio stream was rendering locally in Bowser on my iPhone, but it still wasn't publishing to the stream or TokBox. It was notifying the other participants that I connected, but no stream ever came through. So, I started troubleshooting / stepping through some JS errors I was catching via try/catch. I was able to work my through or hack my way through those, but I've come to a point where one particular issue has me stuck.

I started troubleshooting a MediaStreamTrack object issue that was throwing JS errors. MediaStreamTrack objects have an "enabled" property (boolean) that controls whether or not the individual track is enabled. TokBox had the process of updating the enabled property wrapped in a function that they attached to the prototype:

if (window.MediaStreamTrack && typeof(window.MediaStreamTrack) !== 'undefined') {
        if (!window.MediaStreamTrack.prototype.setEnabled) {
          window.MediaStreamTrack.prototype.setEnabled = function (enabled) {
            this.enabled = OT.$.castToBoolean(enabled);
          };
        }
      }

In FireFox and Chrome, window.MediaStreamTrack and window.MediaStreamTrack.prototype exist and this code works, attaching the nearly superfluous function to the prototype.

In Bowser, window.MediaStreamTrack and window.MediaStreamTrack.prototype are not defined in this context, so it does not attach setEnabled to the prototype and it was breaking the JS code.

At some point in their code, it separately calls functions to get the streams of both video and audio and it loops through them and tries to toggle the "enabled" property based on whether or not a provided "muted" variable is set to true or false. Here's the pseudo code:

var videoTracks = webRTCStream.getVideoTracks();
...
var audioTracks = webRTCStream.getAudioTracks();

Then it looped like this:

for (var i=0, num=audioTracks.length; i<num; ++i) {
    audioTracks[i].setEnabled(!_muted); }

This thew a JS error when it hit the .setEnabled line, so I wrapped it in a try/catch. This was never setting audioTracks[i].enabled or videoTracks[i].enabled to boolean true, so this left the property as "undefined", but catching the error let the JavaScript was letting me continue past the problem. At that point, it was rendering my audio and video in Bowser, but it was not even attempting to publish to the stream, so other participants could not see or hear me, nor I them. With more troubleshooting, I added some additional code to ensure that it wasn't throwing the exception in the first place:

for (var i=0, num=audioTracks.length; i<num; ++i) {
    try {
        if(audioTracks[i].setEnabled) {
            audioTracks[i].setEnabled(!_muted);
        } else { 
            audioTracks[i].enabled = !_muted;
        }
    } catch(error) {
            alert(error.message);
    }
}

So at this point, it's setting the enabled property to true, which it wasn't before, but now the audio and video streams aren't rendering in Bowser anymore (which really doesn't make any sense, logically), but it IS attempting to publish the feed to the stream and throws the following error:

"Publisher PeerConnection Error: The Media resource indicated by the src attribute was not suitable," which is an error code 4 from the window.MediaError.MEDIA_ERR_SRC_NOT_SUPPORTED list.

With the window.MediaStreamTrack and window.MediaStreamTrack.prototype not being defined in Bowser, I'm not sure if there's a problem with my build of if this was done on purpose. This also raises the concern of "what else is not defined the same way between Bowser and Chrome?" and whether or not this difference has anything to do with the issue that I'm working on.

I've done a bunch of reading on various WebRTC implementations recently and I haven't been able to find an answer to the MediaStreamTrack issue.

[noodlemichaelmartin]

Also, I've been given permission to throw a copy of our testing/debugging page and scripts on my bluehost instance, so if anyone has a build of Bowser and want's to take a stab at it, maybe that will help. I can upload that tomorrow morning my time (eastern).

[adam-be]

As @Rugvip said earlier, we don't add anything to the constructor function prototypes in the JS API, all functions and attributes are added to the object when created.

webkitRTCPeerConnection.prototype.addStream is undefined, but
new webkitRTCPeerConnection(config).addStream is defined.

MediaStreamTrack doesn't have public constructor (in the standard) so I'm not sure why Chrome and Firefox expose one. You can't use it since it throws "Illegal constructor", but I guess it can be used for monkey patching (like setEnabled() above).

Anyhow, there's nothing wrong with your build.

[noodlemichaelmartin]

Alright, good, so the troubleshooting continues and thanks for the verification. With MediaStreamTrack.enabled = true, it's attempting to publish but failing, which is likely related to something in RTCPeerConnection. It could be a similar situation where they're monkey patching and it's failing because of it, but they don't need to or some other type Chrome specific expectation that's in Chrome, but not exactly the same in Bowser.. I'll keep plugging through it.

[stefanalund]

We really appreciate your efforts, please let us know how we can support further.

[noodlemichaelmartin]

One additional request that could help me debug this - do you have a sample page that you know does 2-way video chatting and works in Bowser so that I can test and examine the implementation? In the marketing video, it shows 2 people video chatting, so I'm hoping that maybe you've got that little app hosted or available somewhere. The OpenTok/TokBox platform is massive and it'll be easier to isolate the differences if I had something to which I could compare.

[stefanalund]

Absolutely: http://demo.openwebrtc.io

We have confirmed it works between all combinations of Bowser, Chrome and Firefox.

[noodlemichaelmartin]

When loaded in Bowser, it doesn't let me click Join or Call. When using the "Send link to a friend" url, it loads the page and doesn't load the video chat.

[stefanalund]

This is really strange, it works for me on a variety of devices. Are you starting Bowser "fresh" before you load a page? I.e. killing it or running from Xcode. There are known issues with releasing resources: EricssonResearch/openwebrtc#38

Would be interesting to know if you can run our self-view example app:
https://github.com/EricssonResearch/openwebrtc-examples/tree/master/ios/Selfie

[noodlemichaelmartin]

At this point, with a couple of minor tweaks to the opentok.js file, it's displaying the local video again, but it's still not connecting to the opentok servers. I'm getting a specific timeout with their Raptor/Rumor messaging service which sends data back and forth between the browser and their media server, but I've just begun troubleshooting that. This error could be anything at this point.

I can get any page that just simply displays the audio and/or video feed from the current camera/mic in a video tag in the DOM to work fine. Any basic HTML5 sample pages that do that work in my current build of Bowser.

Also, that's correct - I always clear history and kill the app from running before attempting to open it again and load the page. It's cumbersome, but it's the only way it actually reloads the .js and .html content from the server.

As to the demo.openwebrtc.io page, I get the following error text spewing on the page in Bowser now (not sure why the video didn't render locally yesterday when testing it, but it is now):

InvalidStateError: addIceCandidate: no remote description set
InvalidSessionSessionDescriptionError: setRemoteDescription: description type "offer" invalid for the current state "have-local-offer"
InvalidStateError: addIceCandidate: no remote description set
InvalidStateError: addIceCandidate: no remote description set
InvalidStateError: addIceCandidate: no remote description set
InvalidStateError: addIceCandidate: no remote description set
InvalidStateError: addIceCandidate: no remote description set
InvalidStateError: addIceCandidate: no remote description set
InvalidStateError: addIceCandidate: no remote description set
...
(this continues down into the bottom of the background image, where I can no longer read the text)

In my browser on my computer, Chrome in this case, I attempt to "join" and "call", which leaves me with the following error:

Error processing ICE candidate
Error processing ICE candidate
Error processing ICE candidate
Error processing ICE candidate
Error processing ICE candidate
Error processing ICE candidate
...
(this continues down into the bottom of the background image, where I can no longer read the text)

I then tested the page between two different computers using chrome and the page certainly works fine. I'm guessing that the errors I'm getting in both browsers when testing with a Bowser user are Bowser originating. Any thoughts?

[noodlemichaelmartin]

Update: I was able to get the two-way video chatting on demo.openwebrtc.io to work on an iPad with iOS 8.1 on it. Are there iOS version limitations?

[stefanalund]

Nice! The build target is 7.1 currently but I think there is no reason not to only support the latest iOS version. I'm guessing people that would be interested in Bowser will be upgrading their devices.

Did you do any changes? Was this using our demo app or also the one you're working on?

[noodlemichaelmartin]

Sorry, I realize now that I'm kind of shifting back and forth talking about what happened with demo.openwebrtc.io and my test page.

The errors that I posted containing "InvalidStateError: addIceCandidate: no remote description set" were the demo.openwebrtc.io page in my 7.1 build on my iPhone.

I tested demo.openwebrtc.io on this iPad with 8.1 and it works properly (so that's good overall, but bad that demo.openwebrtc.io isn't working in Bowser on my iPhone with 7.1).

Right now, I'm testing my test page on the 8.1 iPad and it's not work and throwing errors when attempting to bind the video stream to the video element.

I'm actively troubleshooting my test page on both the 7.1 iPhone and the 8.1 iPad in Bowser, trying to work through the various errors they're throwing (video element stream bind errors in 8.1 and peer connection errors in 7.1).

Currently, I'm unable to confirm whether these issues solely the nature of the JavaScript code, because the demo.openwebrtc.io page did not work on my 7.1 iPhone and my test page with TokBox isn't working in either version. I did post my test page to a publicly accessible place, so at any point, if someone has a build of Bowser and they could possible easily spot the JS issues, I can privately provide the URL.

[adam-be]

InvalidStateError: addIceCandidate: no remote description set
InvalidSessionSessionDescriptionError: setRemoteDescription: description type "offer" invalid for the current state "have-local-offer"

This (mainly the second error) happens if both clients initiate a call to each other at the same time. The client has generated an offer and installed it locally. But then it gets an offer from the other side. But what it really expected was an answer.

[superdump]

You should only push the call button on one side.

[noodlemichaelmartin]

On my 7.1 iPhone and the latest version of Chrome on my laptop:

• Both joined the same session.
• Audio/Video was being displayed locally in each browser, as expected, "call" button was available in both browsers.
• I clicked call ONLY on my 7.1 iPhone.

After around 25 seconds, an empty video panel is rendered in Chrome on my laptop and a few seconds later refreshes with the feed from my iPhone (painfully slow, one frame every 5-8 seconds and both my machines are on WiFi, same network). The initial panel with the feed from my laptop renders in Bowser on my phone, but never refreshes after the first frame. Errors are produced:

1.7 iPhone, same procedure:
InvalidStateError: addIceCandidate: no remote description set
InvalidStateError: addIceCandidate: no remote description set

Chrome on Laptop:
Error processing ICE candidate x 29 (I counted)

Tried between my iPhone and FireFox (latest version) and regardless of whether I initiate the call in FireFox or I initiate the call in Bowser, the other browser see's that a call has been initiated, but no connection is made, no errors are thrown, but the video panels rendered no less than 2 minutes after the call was initiated.

I've tested it between two computers (as a control test) here in the office in varying combinations of FireFox and Chrome (all the latest versions) with complete successand speedy performance.

At this point, demo.opentok.io DOES work in Bowser (even if the performance is unbearable at times), so I'm going to focus solely on troubleshooting my test page that employs TokBox, where I'm still getting video element stream bind errors in 8.1 and peer connection errors in 7.1.

[superdump]

Which iPhone? And are you using current git master of openwebrtc?

[noodlemichaelmartin]

I'm not rebuilding the openwebrtc daily, because it takes hours to download and recompile. Last build was a few days ago. When rebuilding openwebrtc, do you have to boostrap, rebuild dependencies and build openwebrtc every time, or can I skip some of that?

iPhone 4 iOS 7.1.2

[superdump]

An iPhone 4 is going to struggle. It can never have hardware codec support and software encoding is rather taxing. However, the settings could be tweaked to make it work better for you (lower resolution and framerate.) I'll put up a patch soon.

As for updating, most of the time normally you would just update openwebrtc itself and rebuild but sometimes and recently we updated dependencies. Check the wiki build page for update instructions.

[stefanalund]

Yeah, those older iOS devices are going to struggle. The iPhone 4 is ~4.5 year old and I seem to remember that the iPhone 4s was roughly a 2x speed bump. I know that the iPhone 5 works pretty good even at these higher (VGA) resolutions. Perhaps we should put a disclaimer on the Wiki about this?

[noodlemichaelmartin]

What's the expected output of the following line in Bowser?

window.URL.createObjectURL(webRTCStreamObj);

Right now, it's returning a "mediastream:xxxxxxxxxxx" formatted url, but the problem is that I've noticed that there's strange looking characters in the string, such as this example:

mediastream:cBb5FS+AhcWq7ty6yT70xaSsFy5kH+fMsNdT

I ran the test and this format is pretty much the same random string of chars. I tested some other versions of FireFox and Chrome, some of which giving me a blob:xxxxx formatted ObjectURL and some giving me the mediastream formatted ObjectURL.

The problem is that after the window.URL.createObjectURL(webRTCStreamObj) output is assigned to the src attribute of the DOM video element, the video element fires the onError function that's been attached to it via videoElement.addEventListener('error', onError, false) code. This fires reporting that the src url is not valid by throwing "MEDIA_ERR_SRC_NOT_SUPPORTED" or code 4.

interface MediaError {
  const unsigned short MEDIA_ERR_ABORTED = 1;
  const unsigned short MEDIA_ERR_NETWORK = 2;
  const unsigned short MEDIA_ERR_DECODE = 3;
  const unsigned short MEDIA_ERR_SRC_NOT_SUPPORTED = 4;
  readonly attribute unsigned short code;
};

So, the question is - - Why would a videoElement.src = window.URL.createObjectURL(webRtcStream); call throw that error?

[noodlemichaelmartin]

Additional notes:

webRTCStreamObj is of type MediaStream (appears to be native)
window.URL.createObjectURL is native
videoElement.src the output of a var videoElement = document.createElement('video'); call (this is standard/native).

This LOOKS exactly the same as the way demo.openwebrtc.io does it, but that page does not throw this error. I'm using the most recently available code for both bower and openwebrtc.

[stefanalund]

I'd be thrilled if you took the time to document your findings on this page:

https://github.com/EricssonResearch/bowser/wiki/Developing-for-Bowser

It will help other developers to get started with Bowser. The wiki is currently a bit thin, we plan to add more ourselves. Thanks.

[noodlemichaelmartin]

Honestly, I've changed very little thus far. In Bowser, I've only changed the version that's being reported in the header of the Bowser source. In openwebrtc, I've changed nothing. For developers writing apps/web pages that are meant to run in Bowser, there's a few things to document in regards to the differences between the prototypes, method names, and any other little quirks that might deviate slightly from the other two main WebRTC enabled browsers. I'd be happy to circle back and write up some notes or things to consider when building for Bowser once I'm finished getting the TokBox/OpenTok platform to work on it.

On that note, have you considered setting the actual user-agent to be unique, such as "Bowser", rather than "not Chrome" as it is now? I know this basically makes it compatible with chrome-specific code (which it should be by default), but since there are significant differences in the ostensible implementation of the common WebRTC JS functions, perhaps it would be nice to be able to more definitively identify it as Bowser? Just a thought.

Also, does anyone have any suggestions or thoughts on my previous two posts (they're really a single post, I just posted some additional notes after my original posting)? I've been stuck for several days at that point and have been forced to jump over to some other little projects temporarily due to the roadblock.

[stefanalund]

@noodlemichaelmartin "Bowser/0.2.1" is included in the UserAgent so you could just check for that. I leave the other question to someone else (ping @adam-be @pererikb).

[webvinita]

Dear Noodlemichaelmartin

Are you able to make opentok working on iPhone Bowser. I have been also waiting for a browser which can be used in iPhone to run my opentok application.
Any suggestion on how to make openok application working on iPhone Bowser?

[EvilTrev]

Did anyone make any further progress on this? I've been stabbing at trying to solve the same issue, so far:

1. I've had to include http://localhost:10717/owr.js myself in the head area of my page, because otherwise other loading code cannot find getUserMedia etc... it's all undefined.

2. Include this code immediately after the above header to patch in the missing prototype. It would seem Bowser acts like webkit, but not like Chrome/Opera... I've seen others using this same prototype for testing the presence of webrtc... so not just OpenTok. I'm sure this is old/invalid or not to standard... but it exists in the most recent version of OpenTok and Chrome/Opera seems to allow and pass on the test for it.

   if(window.webkitRTCPeerConnection) {
       var tempHack = new webkitRTCPeerConnection({iceServers:[]});
       window.webkitRTCPeerConnection.prototype.addStream = tempHack.addStream;
   }
   

3. I replaced self.javascriptCode with an empty @"" string in BowserViewController.m to avoid causing issues with including it manually.

4. For an experiment, I also hacked opentok.js to include the code below when they create the peerconnection (this was just in case there is some magic lost in the above stub... it didn't seem to make any difference):

   pc = new NativeRTCPeerConnection(config, options);
   if (window.webkitRTCPeerConnection) window.webkitRTCPeerConnection.prototype.addStream = pc.addStream;
   

5. Also tried playing with the useragent string (which wasn't required, but thought it might prevent some monkeying)... no change.

   #define kUserAgent @"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.76 Safari/537.36"
   
   #define kUserAgent @"Mozilla/5.0 (iOS; like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) not Chrome/36.0.1500.95 Mobile/12B411 Safari/600.1.4 Bowser/%@"
   

6. Found this code explaining the setEnabled nonsense (basically it's to support OpenTok IE plugin):

   // The setEnabled method on MediaStreamTracks is a OTPlugin
   // construct. In this particular instance it's easier to bring
   // all the good browsers down to IE's level than bootstrap it up.
   if (typeof window.MediaStreamTrack !== 'undefined') {
       if (!window.MediaStreamTrack.prototype.setEnabled) {
           window.MediaStreamTrack.prototype.setEnabled = function (enabled) {
               this.enabled = OT.$.castToBoolean(enabled);
           };
       }
   }
   

7. Wrapped the opentok.js setEnabled in try...catch, found that got local video preview to show, remote doesn't, seems to be stopped dead by these exceptions:

   [Error] Failed to load resource: unsupported URL (mediastream:tOtTiip/yEO0ZWmtk9XMhrfxXeWujnElFMf2, line 0)
   [Error] OT.exception :: title: Connection Failed (1013) msg: Publisher PeerConnection Error: There was an unexpected problem with the Video Stream: The media resource indicated by the src attribute was not suitable.
   [Error] 1013 "Publisher PeerConnection Error: There was an unexpected problem with the Video Stream: The media resource indicated by the src attribute was not suitable."
   [Error] OT.exception :: title: Connection Failed (1013) msg: Publisher PeerConnection Error: There was an unexpected problem with the Video Stream: The media resource indicated by the src attribute was not suitable.
   [Error] 1500 "Session.publish :: Publisher PeerConnection Error: There was an unexpected problem with the Video Stream: The media resource indicated by the src attribute was not suitable."
   [Error] OT.exception :: title: Unable to Publish (1500) msg: Session.publish :: Publisher PeerConnection Error: There was an unexpected problem with the Video Stream: The media resource indicated by the src attribute was not suitable.
   

8. Found possible clue to the above spew (haven't had chance to investigate):

   In Chrome or Opera, the URL.createObjectURL() method converts a MediaStream to a Blob URL which can be set as the src of a video element. (In Firefox and Opera, the src of the video can be set from the stream itself.)
   

9. Plenty of this going on in Xcode:

   (<unknown>:12691): GStreamer-CRITICAL **: guint64 _gst_util_uint64_scale(guint64, guint64, guint64, guint64): assertion `denom != 0' failed
   

All pretty hacktastic so far... I'm just trying to prove that it might even work. Which it doesn't currently :) I don't see the iPhone on my desktop, I don't see the desktop on the iPhone, the preview on the iPhone is super laggy, the iPhone seems to hear itself (not feedback screams, just echos). When the session is terminated, the phone still hears itself. I'll report back if I make any further progress.

[EvilTrev]

So no? ;)

[superdump]

@adam-be?

@EvilTrev - we will get to testing more thoroughly such WebRTC apps and figuring out problems but our priority so far has been on improving the core of the framework and project infrastructure (new build system coming very soon, automated builds, releases, testing).

We plan to compile a list of popular WebRTC libraries commonly used by apps/services, test compatibility with those and fix/report issues.

[RyanPetrucci]

Hey!

Any updates on this?

As far as I can currently see this is by far the nearest solution to OpenTok support within a hybrid iOS app - what's the outlook on adding OpenTok support any time soon?

@noodlemichaelmartin - as you never returned does that mean you found a solution or also are currently awaiting an update on this?

Cheers!! :)

[noodlemichaelmartin]

We have not found a solution. Another developer here attempted to build a more recent version of Bowser last month and was met with device support issues that render the browser unusable for us and the majority of our potential users.

We've been forced to abandon this issue until such time when either Bowser runs on a more expansive list of "legacy" iOS devices or appropriate solution presents itself.

[RyanPetrucci]

Hey!

Thanks a lot for the update, I've been searching and searching for a hybrid iOS WebRTC solution and was sure I'd found one until now.

It's a shame Apple force vendors to use only their rendering engine else we'd have a solution with Chromium.

On that front I wonder if it would be viable to create a fork of Chromium and add WebRTC support using Google's WebRTC library in a similar to manner that EricssonResearch added WebRTC support using the OpenWebRTC library or if it's already in the works - if so it's hiding well!

If I do manage to find/create a solution I'll let you know and leave an update on here anyway as I know this is widely sought.

Thanks for the fast response! :)

[superdump]

The device support issue is a problem with language bindings and the changing hardware landscape - arm64 entered the picture, libffi was updated to support arm64 on iOS. This simultaneously broke armv7 support on iOS and it is yet to be fixed. Unfortunately as the iOS toolchain has also been updated by Apple, it is not a case of just using an old version of libffi for armv7 iOS until it is fixed as they no longer work either with newer Xcode.

[RyanPetrucci]

Basically we got an inch away from WebRTC on native iOS apps and Apple killed it - good to know!

If a solution crops up in the medium-terms I'll report back here unless somebody else does - thanks for the clarification and quick reply! :)

[noodlemichaelmartin]

@superdump - Yes, that's what he ran into when trying to get Bowser to run on several of our test devices.

@ArtisanCodesmith - Yes, we were excessively close during my initial attempts. Got down to some JS issues that we never quite got resolved. However, good luck to you on the Chromium+OpenWebRTC idea; hope you manage to get it working. We opted not to pursue a custom solution, as we would need to develop and support the app ourselves. TokBox has an iOS SDK, but it's not a browser and is rather myopic - you can't easily have web content + video chat in one place without, essentially, building your own browser within your custom app. Using a normal browser on iOS, we'd basically have needed to embed links to the SDK-based external, custom app in the web page and force the user to switch between apps to make the whole thing work. That's a terrible user experience. Only a browser with some form of webRTC technology embedded into it provides the appropriate user experience.

[RyanPetrucci]

They're currently requesting audio and data support in Chromium for iOS here which the Chromium team has said would be achieved easily which would be great as it would be half-way there at least. Will have to wait on that (and perhaps nag a little as then there would be something to perhaps work with).

They apparently will then opensource the web shell which will open a few doors also.

I'll keep looking into solutions and report back anyway - thanks and good luck with everything! :)

[superdump]

We're actively making patches for a project to add WebRTC support into WebKit and some have landed already. http://www.webrtcinwebkit.org/

Maybe in the not too distant future Apple will decide to jump onto the mobile WebRTC train.

[RyanPetrucci]

That's great and most definitely the way to go - having WebRTC support built into WebKit would be outstanding! Thanks for actively contributing to such a feat!

[RyanPetrucci]

Hey!

I've successfully made an iPad app using OpenTok that allows for group video chats (the videos can be dragged around and all sorts!). You have to update the videos' position when dragged manually but it's incredibly smooth!

It has the same codebase for Android and the web also (the Android version is wrapped in CrossWalk which has full WebRTC support).

Hopefully you can get your app finished now @noodlemichaelmartin !

To achieve this I used PhoneGap and this Cordova plugin!

The video/voice chat now works seamlessly across iOS, Android and browsers!

This does not work with PhoneGap Build I must add and you must also (in order to build the app) first prepare it via the terminal by doing:
"phonegap prepare ios"

Then open the created Xcode project in Xcode and finish the build yourself and tadaa!

A fully working WebRTC app on the iOS!

Really hope this helps you get back to the project you're working on - it you need any help give us a shout and I'll be more than happy to oblige!

See you soon!! :)

@webvinita @EvilTrev - you two also mentioned you have projects that require fully functional OpenTok on the iOS - I hope this helps!