Stream video have big delay, it's not like a chat realtime about libstreaming

Hi dominivis,
As you have mentioned in point-2, ( Using the highest resolution possible (1920x1080) with 4 Mbps bitrate and 15 fps we had 3 seconds of delay.) which this high resolution is working for you. Could you please describe here, how exactly you made it working?
I am pulling my hairs since weeks while setting up high quality resolution.
If anyone has done this, plz help.
Thanks

[raghavrao1]

Get rid of frame rate / sleep / high priority thread logic to increase throughput. if u can make the libstreaming an rtsp client instead of a server.

[dominivis]

I'm having the same problem. Can anyone be more specific about this issue?

Is there anyway of reducing this delay by adjusting the code or is it by design?

Thank you in advance.

[raghavrao1]

Can u describe your setup hardware and software? Are you streaming from libstreaming to a media server like wowza or is your client connecting to libstreaming directly. The more info you can provide the more i can be helpful.

[dominivis]

I am using libstreaming on a Galaxy S4 with Android 4.4.2

I use VLC as the player on a Windows 8 machine and I connect directly to libstreaming. We are using it for real time streaming from the phone's camera.

The delay is about 3 seconds for any resolution I use. If the network gets unstable we notice the delay gets higher if we stream the video for longer times. (I'm using 300 Mbps Wifi router, but the environment I work have a lot of routers and that seems to be of some trouble at times)

Looking at the code we found that this delay seems to be the time between recording and sending the first packet. It looks like that the when the packetizer starts to send the packets it doesn't get the "correct" time, it just starts from the beginning of the "movie" not the real time.

I hope this helps, please let me know if I couldn't make myself clear.

Thanks for the attention!

[raghavrao1]

Ok, so you are using peer to peer mode.

Some questions and pointers

1. Write a general program that runs between your windows 8 computer and
   S4. Send UDP packets from S4 and capture them on Windows 8 to gauge the
   time delay. If it is a networking issue it would be evident from this
   experiment( don't use ping since ICMP adds a delay automatically).
2. Just to identify the delay use the least possible resolution for the
   camera. 320*240 or even less, the less the better since we want to see
   where is the delay coming from.
3. Use 15 frames per second(fps) for your camera i.e when the camera
   setParameters should set fps to 15.
4. libstreaming supportsmedia capture via MediaRecorder and MediaCodec.
   Media Codec also have 2 flavors, in api 19 it uses Camera surface directly
   instead of getting the buffer via preview frame.
   Check which one is getting used. If you are on kitkat use MediaCodec and
   MediaRecorder might be a little slower.
5. Inside libstreaming , search for setPriority. Android does not like
   higher priority threads , comment all calls to setPrioirty.
6. libstreaming is trying to maintain framerate, comment out the logic. you
   need to send out a packet as soon as it ready.

On Thu, Nov 20, 2014 at 12:43 PM, dominivis [email protected]
wrote:

I am using libstreaming on a Galaxy S4 with Android 4.4.2

I use VLC as the player on a Windows 8 machine and I connect directly to
libstreaming. We are using it for real time streaming from the phone's
camera.

The delay is about 3 seconds for any resolution I use. If the network gets
unstable we notice the delay gets higher if we stream the video for longer
times. (I'm using 300 Mbps Wifi router, but the environment I work have a
lot of routers and that seems to be of some trouble at times)

Looking at the code we found that this delay seems to be the time between
recording and sending the first packet. It looks like that the when the
packetizer starts to send the packets it doesn't get the "correct" time, it
just starts from the beginning of the "movie" not the real time.

I hope this helps, please let me know if I couldn't make myself clear.

Thanks for the attention!

—
Reply to this email directly or view it on GitHub
#57 (comment).

[dominivis]

1. We sent 10 packages of 1024 Btyes using UDP from the S4 to the PC, the total time for this 10Mb test was 0.150 seconds.

2. With MediaRecorder we have delays even in the lowest resolution (176x144), with approximately 2 seconds of delay. Using 500 kbps bitrate and 15 fps. Using the highest resolution possible (1920x1080) with 4 Mbps bitrate and 15 fps we had 3 seconds of delay.

3. We used 15 fps on all the tests.

4. Using MediaCodec with the lowest resolution (176x144) we had a reduction on the delay, it is still noticeable, but it is roughtly 1 second. We also found a comment inside the file H264Stream.java that says that MediaCodec shouldn't be used for higher resolutions though. And we need Full HD. If we try Full HD with MediaCodec the application crashes.

5. We couldn't find any setPriority

6. Is this on the RTPSocket class? We foung a FIFO queue there but we didn't find it simple to just "comment out". We dindn't find a way to get rid of this queue and sent the packaged immediately. Could you be more specific about this?

Thank you so much for all this help and sorry we took so long to respond.

We will be glad to answer any more questions you have.

[raghavrao1]

I thought you had a delay of 10 secs. 2 seconds delay is not bad and can be improved.

500 Kbps is very high bit rate. Use the following link to calculate bit rate. Go to step 2, enter resolution and enter fps and then adjust the resolution and fps to get 0.25 bits per pixel. Use the bitrate.

http://www.silverjuke.net/forum/topic-213.html

Use the lowest possible resolution for testing for now.

1. In RtpSocket there is a call to setPriority, comment this out as wells as any other call that sets the thread priority
   mThread.setPriority(Thread.MAX_PRIORITY);
2. In RtpSocket comment out all calls to Thread.sleep

Thread.sleep(mCacheSize);

if (mCacheSize>0) Thread.sleep(d);

[dominivis]

Hi!

We managed to get a very acceptable delay for low resolution with MediaCodec, thank you.

Unfortunately, our application requires high resolutions.

Do you think we count maintain this low delay using MediaCodec and higher resolutions? (The current version just crashes)

I think we can assume we won't get lower delays with MediaRecorder, right?

If this is the case I'm afraid we won't be able to use libstreaming for our solution. What do you think?

Anyway, thank you very much for your patience and time.

Eduardo

[raghavrao1]

What is the crash? is it at codec level or in libstreaming.

The problem with MediaRecorder is that you cannot set IFrame rate and other parameters, since Media recorder records in MP4 format.

Can you check with codec is crashing. I am sure that there is more than one H264 codec available on s4.
You can print out all the codecs available via MediaCodecList methods. If there is more than H264 codec available then you can change libstreaming code to use a different codec. MediaCodec.createByCodecName

Besides libstreaming which library you are looking at?

[dominivis]

The error we are reffering with MediaCodec was the same on this thread #43

At some point in the other thread it was suggested to use a large heap size on the android manifest, we used that, but we then got another error: "The encoder is too slow." from the method "enconde" in the class EncoderDebugger

Unfortunately, we couldn't find other libraries that met our needs. We even thought about changing the protocol to HLS, but there is nothing out there stable enough at this point.

If we can't get this right, maybe our research will end at this point.

Do you know any other libraries that work as a H264 stream server on Android?

[raghavrao1]

Don't use HLS or Mpeg Dash, they are segment based protocols and they introduce an inherent delay not suitable for live streaming. I think the choice for live streaming is RTSP/RTP or RTMP.

I have not encountered the buffer issue since i am using low resolution 324 * 240 , and both Media Recorder and Media Codec work fine with it. But it looks like there are issues with higher resolution, so i might have to look for different implementation.

There is a google open source project called java cv which is a wrapper over open cv which is wrapper over ffmpeg and other libraries. It works on Android, but i have not played with it.

https://github.com/bytedeco/javacv

If you come across any other library that provides live streaming, then please let me know. I will update you if i am able to fix this issue or come across a better solution.

[dominivis]

Thank you, I will notify this thread if we have any progress.

[gabrielrolfsen]

I am facing the same problem as dominivis, does anyone have a solution? I would like to stream videos with a higher resolutions (doesn't need top be full HD, but at least 800x600) and I cannot.

Thank you.

[raghavrao1]

use http://www.webrtc.org/
they have opensource code that works on android, ios and web.

another webrtc opensource

http://www.openwebrtc.io/

On Tue, Jun 16, 2015 at 11:32 AM, Gabriel R. [email protected]
wrote:

I am facing the same problem as dominivis, does anyone have a solution? I
would like to stream videos with a higher resolutions (doesn't need top be
full HD, but at least 800x600) and I cannot.

Thank you.

—
Reply to this email directly or view it on GitHub
#57 (comment)
.

[qdk0901]

I test the newest version, and the video has 3~4 seconds delay on android 4.4 with resolution of 176x144 and framerate of 12fps

Well, change mBufferCount in RtpSocket from 300 to 5, I get a delay reduce to less than 1 second,
that good enough for me temporary