Regular receipt of undersized packet. about libfreenect

[zarvox]

Looks to me like the streams are switched - the RGB stream is 1908 data bytes/packet, and the depth stream is 1748 data bytes/packet. Perhaps stream 0x70 isn't always the depth stream?

[marcan]

Maybe the USB stack is somehow padding them? We ask for 1920 bytes for both streams (because that is the iso packet size).

[korostelevm]

I'm having the same problem, frame rate goes back up and there are no more errors when I don't pull a video feed with freenect_sync_get_video

[korostelevm]

Also, really weird. The problem with dropping frames goes away when I plug in a usb mouse, comes back when unplugging

[madgadjt]

Having the same problem, has anyone had any luck correcting this issue?
I have compiled and run glview fine on my desktop, however when i try the same on my laptop, I have similar output and my framerate is unusable.

[tnbozman]

I agree that there is certainly a problem with libfreenect, however the library is usable. I can use libfreenect on both ubuntu 11.10 laptop or pandaboard. However I have written code that results in minmal magic numbers and dropped frames but it requires 90% cpu utilisation. I do this by threading the process_events libfreenect function. The high cpu utilisation is due to the main thread having my image processing in a tight while loop. If I use any type of blocking call in the main thread, then libfreenect become unusable and spewing magic number errors. This is rather unacceptable as 90% cpu utilisation over a long period of time is rather undesirable. I to am interested in diving it libfreenects usb code and fixing the issue, but time is the problem.

[lancelet]

I'm also still having this problem with the current git master (is this an issue over 3 years old?!). If I only use a depth stream then the frames come back cleanly. If I use both depth and RGB streams simultaneously, I see choppy capture and the console messages

[Stream 70] Expected max 1748 data bytes, but got 1908. Dropping...

The dual depth+video capture is choppy on a 2013 Haswell MacBook Pro (2.6 GHz), so I doubt that it will be smooth for anyone's machine.

[piedar]

This may be due to problems in the way libusb works with the USB stack on OSX. I would try booting Linux on a USB and running it there.

[anshulvj]

I'm facing a similar problem on Pcduino (ARM). I get a constant stream of errors:

[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 7 packets
[Stream 70] Lost 3353 total packets in 383 frames (8.754569 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 80] Invalid magic 0515
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 7 packets
[Stream 70] Lost 3360 total packets in 384 frames (8.750000 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 80] Invalid magic 7255
[Stream 80] Lost 1 packets
[Stream 80] Lost 3047 total packets in 401 frames (7.598504 lppf)
[Stream 80] Lost 4 packets
[Stream 80] Lost 3051 total packets in 401 frames (7.608479 lppf)
[Stream 80] Inconsistent flag 85 with 66 packets in buf (162 total), resyncing...
[Stream 80] Invalid magic 101b
[Stream 80] Lost 1 packets
[Stream 80] Lost 3052 total packets in 401 frames (7.610972 lppf)
[Stream 80] Lost 3 packets
[Stream 80] Lost 3055 total packets in 401 frames (7.618454 lppf)
[Stream 70] Invalid magic ffff

And although glview shows Depth and RGB, the framerate is too slow, probably because of the drops. Can anyone suggest a solution to this?

[piedar]

Small boards like that often lack USB bandwidth and/or CPU power to process Kinect data. There are Linux kernel drivers gspca_kinect (video) and librekinect (depth). Kernel-space USB has less overhead and higher throughput at the cost of platform independence.

[J-G-R]

I get the same as the original OP. I'm running libfreenect on a Beaglebone Black rev c running Ubuntu 14.04. If I work with one stream at a time it works well - 21fps with rgb and 6-7fps with the depth stream. However if i try to work with both RGB and depth streams at the same time, I get the same error as the original OP and i receive nothing from either stream. Has any progress been made with this issue?

[piedar]

@J-G-R This is a tricky issue because I've only seen it happen briefly so I can't really say what's causing it. Which example did you run to see the problem?

[J-G-R]

I'm running a custom application written in C. It simply gets RGB and depth frames from the kinect, compresses them and then sends them back to my laptop via TCP socket. The method i'm using can be seen below:

int kinectUpdate(void) {
    uint32_t ts;
    // Pull a (distorted) RGB frame
    int ret = freenect_sync_get_video_with_res((void **) &rgb_raw, &ts, 0,
            FREENECT_RESOLUTION_MEDIUM, FREENECT_VIDEO_RGB);
    if (ret != 0) {
        printf("Error: unable to acquire RGB stream\n");
        return ret;
    }

    // Pull a depth frame registered to the above image
    int retD = freenect_sync_get_depth_with_res((void **) &depth_registered, &ts, 0,
            FREENECT_RESOLUTION_MEDIUM, FREENECT_DEPTH_REGISTERED);

    if (retD != 0) {
        printf("Error: unable to acquire registered depth stream\n");
        return retD;
    }
    return 0;
}

If I delete either of the freenect_sync methods and run it it works perfectly and I receive the frames on my laptop. I've also tried separating this method into two individual methods but that doesn't work either.

[piedar]

See if you get the same messages when running the freenect-camtest example (Ctrl-C to quit).

[J-G-R]

I ran freenect-camtest for about 5 seconds and this is the result:
freenect-camtest
[Stream 70] Negotiated packet size 1920
write_register: 0x0105 <= 0x00
write_register: 0x0006 <= 0x00
write_register: 0x0012 <= 0x03
write_register: 0x0013 <= 0x01
write_register: 0x0014 <= 0x1e
write_register: 0x0006 <= 0x02
write_register: 0x0017 <= 0x00
[Stream 80] Negotiated packet size 1920
Failed to submit isochronous transfer 0: -1
Failed to submit isochronous transfer 1: -1
Failed to submit isochronous transfer 2: -1
Failed to submit isochronous transfer 3: -1
Failed to submit isochronous transfer 4: -1
Failed to submit isochronous transfer 5: -1
Failed to submit isochronous transfer 6: -1
Failed to submit isochronous transfer 7: -1
Failed to submit isochronous transfer 8: -1
Failed to submit isochronous transfer 9: -1
Failed to submit isochronous transfer 10: -1
Failed to submit isochronous transfer 11: -1
Failed to submit isochronous transfer 12: -1
Failed to submit isochronous transfer 13: -1
Failed to submit isochronous transfer 14: -1
Failed to submit isochronous transfer 15: -1
write_register: 0x000c <= 0x00
[Stream 70] Lost 12 total packets in 0 frames (inf lppf)
write_register: 0x000d <= 0x01
write_register: 0x000e <= 0x1e
write_register: 0x0005 <= 0x01
Received depth frame at -1846614715
[Stream 70] Lost 24 total packets in 1 frames (24.000000 lppf)
write_register: 0x0047 <= 0x00
Received depth frame at -1842610405
[Stream 70] Expected max 1748 data bytes, but got 1908. Dropping...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 8772
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 84b0
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Lost 12 packets
[Stream 70] Lost 36 total packets in 2 frames (18.000000 lppf)
[Stream 70] Lost too many packets, resyncing...
[Stream 70] Invalid magic 9b33
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 1d83
[Stream 70] Invalid magic c5f8
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 6cc9
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic f75e
[Stream 70] Invalid magic f27e
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
Received depth frame at -1838606095
Received depth frame at -1836603940
[Stream 70] Expected max 1748 data bytes, but got 1908. Dropping...
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 83b0
[Stream 70] Invalid magic f8bf
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic c197
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic eee1
[Stream 70] Invalid magic 5ceb
[Stream 70] Invalid magic 7e4f
[Stream 70] Lost 12 packets
[Stream 70] Lost 48 total packets in 4 frames (12.000000 lppf)
[Stream 70] Lost too many packets, resyncing...
Received depth frame at -1832599630
[Stream 70] Expected max 1748 data bytes, but got 1908. Dropping...
[Stream 70] Invalid magic fffb
[Stream 70] Invalid magic 3586
[Stream 70] Invalid magic cd7f
[Stream 70] Invalid magic ffff
[Stream 70] Invalid magic 6aed
[Stream 70] Lost 6 packets
[Stream 70] Lost 54 total packets in 5 frames (10.800000 lppf)
[Stream 70] Lost too many packets, resyncing...
^CShutting down
write_register: 0x0006 <= 0x00
write_register: 0x0005 <= 0x00
Done!

[piedar]

So it's dropping packets even without doing work (other than printf) in the freenect event thread. My best guess is that the host can't keep up with all the transfers coming in. There are a few things you can try.

First is to compile for your particular CPU and with heavy optimizations.

cmake .. -DCMAKE_C_FLAGS="-march=native -O3"

If that doesn't help, try fiddling with the PKTS_PER_XFER and NUM_XFERS constants in usb_libusb.h. The settings for Linux are in the last block, at line 47.

If you get really desperate, you can also have a look at the chunk processing feature introduced in #351, which lets you downsample to a lower resolution and skip a lot of decoding.

Sorry I can't be of more help. I'm 85% sure it's a hardware or USB stack issue, but it's really hard to tell.

[J-G-R]

Hmm, I'll give your suggestions a try. I'm also thinking it's a hardware issue due to the camtest failed almost instantly. I knew the RPi wasn't able to handle libfreenect so I wen't for the beaglebone which I honestly thought would be able to keep up - 1GHz processor, 512Mb RAM - but now it looks like I'll have to go for a more powerful board. I'm thinking along the lines of an odroid which has a 1.7GHz quad core processor and 2GB of RAM. Do you think that will be enough to handle libfreenect?

[piedar]

It's more an issue of USB bandwidth than CPU or RAM. According to my rough calculation, depth and video together push about 20 MB/s over USB.

The ODROID is better in that regard because it has separate hosts for ethernet and USB. The U3 should be up to the job. But if you can spend $30 extra, go for XU3 Lite; I've seen a report that it even works with two Kinects, one on the 2.0 port and one on the 3.0.

[J-G-R]

Thanks for the insight. I've decided to bite the bullet and go for the Odroid xu3 lite like you suggested. I'll post here if it's successful.

[J-G-R]

Update: The kinect used with libfreenect works like a dream on the odroid. I'm even receiving 30 fps on my laptop over wifi, with no dropped packets. Thanks for the advice @piedar 👍

[Cherden]

@J-G-R is there a way to get in contact with you?