Question: Can you load up two Networks at once? about jetson-inference

[S4WRXTTCS]

@raaka1

The dataset for the object detection consist of 271 images, and the dataset for the image classification consists of a little over 1000 images.

I'll likely add more images to the object detection to better detect cards at different distances. Right now it's pretty locked to a certain distance away from the cards, and with using a USB camera (to mount it facing down).

[S4WRXTTCS]

The DetectNet training data is here
https://drive.google.com/file/d/0B8dR1eAmu3fTR3l4WkNtR0dqS0E/view?usp=sharing

The ImageNet training data is here
https://drive.google.com/file/d/0B8dR1eAmu3fTcG1mZVN4OHFNTU0/view?usp=sharing

[dusty-nv]

Hi Jason, you can create an imageNet object in addition to detectNet and run them separately. In normal circumstances there should be enough memory for both. Imagenet is faster than detectNet. Best regards, Dustin On Feb 27, 2017 8:27 PM, Jason Mecham <[email protected]> wrote: Currently I use detectnet-camera to detect a position of objects within the live camera frame, and then I write out the image crops for the objects found. Now I want to run the image crops through a trained imagenet classification network to identify exactly what the object is (like a ford pinto instead of simply being a vehicle). Do I have to stop the DetectNet network in order to do this? Or can I run an imagenet based classification while the detectnet is still loaded? I want to be able to go between the two with as little reduction on the frame rate as possible. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#45>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AOpDK2g8kuF_E0lh5dGaQYb0Rh-_ZWL5ks5rg3gSgaJpZM4MN2iB>.

…

----------------------------------------------------------------------------------- This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -----------------------------------------------------------------------------------

[S4WRXTTCS]

Thanks. It seems to load fine.

Now I just have to figure out how to get the image data within the boundary box (from detectnet) into another memory buffer for the classify function. Or expand the classify function to work on a ROI.

I probably should have started with learning about how to use CUDA on some basic level before playing around with this. :-)

[dusty-nv]

Feel free to use the CUDA code from https://github.com/dusty-nv/jetson-inference/tree/master/cuda as a template, for example cudaResize is pretty simple to get started with. From: Jason Mecham [mailto:[email protected]] Sent: Tuesday, March 07, 2017 7:38 PM To: dusty-nv/jetson-inference Cc: Dustin Franklin; Comment Subject: Re: [dusty-nv/jetson-inference] Question: Can you load up two Networks at once? (#45) Thanks. It seems to load fine. Now I just have to figure out how to get the image data within the boundary box (from detectnet) into another memory buffer for the classify function. Or expand the classify function to work on a ROI. I probably should have started with learning about how to use CUDA on some basic level before playing around with this. :-) — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#45 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AOpDK3Dl2hdC_endt4NaBtDKRMjRaxtUks5rjfh7gaJpZM4MN2iB>.

…

----------------------------------------------------------------------------------- This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -----------------------------------------------------------------------------------

[S4WRXTTCS]

Thanks.

I copied the gpuPreImageNetMean to new function that works on a region of interest of the live camera image coming in. Where the ROI is from the boundary box of the DetectNet.

It's pretty neat seeing detectnet detect the objects, and then imagenet classify them all while running live.

Here is the function I added to do deal with a ROI of interest on the 1280x720 incoming image. I believe it works correctly from the limited testing I've done.

// gpuPreImageNetMeanROI
global void gpuPreImageNetMeanROI( float2 scale, float4* input, int iWidth, float* output, int oWidth, int oHeight, int roix_start, int roiy_start, float3 mean_value )
{
const int x = blockIdx.x * blockDim.x + threadIdx.x;
const int y = blockIdx.y * blockDim.y + threadIdx.y;
const int n = oWidth * oHeight;

if( x >= oWidth || y >= oHeight )
	return;

const int dx = ((float)x * scale.x) + (float)roix_start;
const int dy = ((float)y * scale.y) + (float)roiy_start;
   
    const float4 px  = input[ dy * 1280 + dx ];
const float3 bgr = make_float3(px.z - mean_value.x, px.y - mean_value.y, px.x - mean_value.z);

output[n * 0 + y * oWidth + x] = bgr.x;
output[n * 1 + y * oWidth + x] = bgr.y;
output[n * 2 + y * oWidth + x] = bgr.z;

}

[S4WRXTTCS]

Here is what I did with this project.

[dusty-nv]

That's really cool! I appreciate the novel use of pipelining detectNet and imageNet. I would like to highlight that approach in the tutorial, maybe we can collaborate to use your application as an example if you are willing?

In practice, do you have any plans for the card detection system? Nice work!

[S4WRXTTCS]

Yeah, I can share the Datasets and the Models along with the code. Now keep in mind the code is really just a bunch of hack of your code, and I probably didn't do things the right way. So it's going to need some fixing. The collaboration would benefit me as I'm new to this CUDA stuff, and hopefully other people can use it to do some cool things with it.

I also hacked in an image/label saving tool. So it saves not just the image, but the label file as well. That enabled me to quickly add more data to the dataset. Where I could easily fix the boundary boxes (for detectnet) or fix the classification (for imagenet).

I picked playingcards so I could share the code/data/model without revealing anything about what I'm actually working on for work. It's using the same type of pipeline, but different data.

As a hobby project I might finish it up to play a game of 21 with the user. Where the user has to be the dealer, and the computer card counts.

[S4WRXTTCS]

I uploaded my AlexNet based model for the playing card identification, and the DetectNet model for detecting the playing card. But, for some reason wget doesn't download them correctly.

I edited the link in the same way that the CMakePreBuild.sh has them, but it ends up saving a very small file that isn't correct.

Here are the links with the original formatting.

AlexNet
https://drive.google.com/file/d/0B8dR1eAmu3fTWWpkSEVLV0Z5TVE/view?usp=sharing

CardNet (DetectNet Cased)
https://drive.google.com/file/d/0B8dR1eAmu3fTMU9Vb3p1S05Bclk/view?usp=sharing

Here is the modified wget line for AlexNet
wget --no-check-certificate 'https://drive.google.com/uc?export=download&id=0B8dR1eAmu3fTWWpkSEVLV0Z5TVE' -O alexnet_54cards.tar.gz

Is there something I'm missing? I uploaded the files simply using a chrome client to google drive and then shared them via a public link.

[S4WRXTTCS]

Apparently this is because of a "Google cannot provide a virus scan" error message.

http://unix.stackexchange.com/questions/136371/how-to-download-a-folder-from-google-drive-using-terminal

In the above link there is a solution, and the solution works. But, I don't know how to return the code it sends. For now I just reuse the same code that it gave me the first time, but I'm sure that will break.

[dusty-nv]

Hey, thanks for sharing Jason. For the reason you mentioned with gdrive, I've changed over to Box.com for the most part (see master CMakePreBuild.sh). Box doesn't seem to care about serving up large files and also can provide the direct link. Best regards, Dustin On Mar 30, 2017 8:22 PM, Jason Mecham <[email protected]> wrote: Apparently this is because of a "Google cannot provide a virus scan" error message. http://unix.stackexchange.com/questions/136371/how-to-download-a-folder-from-google-drive-using-terminal In the above link there is a solution, and the solution works. But, I don't know how to return the code it sends. For now I just reuse the same code that it gave me the first time, but I'm sure that will break. — You are receiving this because you commented. Reply to this email directly, view it on GitHub<#45 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AOpDKz3ImriclscUMnt814vOsHOQ1EZ3ks5rrEdWgaJpZM4MN2iB>.

…

----------------------------------------------------------------------------------- This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. -----------------------------------------------------------------------------------

[S4WRXTTCS]

Yeah, I might try Box and another option is this utility which seems to work great.

https://github.com/circulosmeos/gdown.pl

So I can download that in the script file and then use it.

I can't use dropbox because my work internet blocks them all the time (but, not always which is weird).

[raaka1]

How many images did you use for training ?

Regards

[S4WRXTTCS]

I uploaded a very primitive blackjack type game using this dual method detection/classification.

https://github.com/S4WRXTTCS/jetson-inference

It's not really suited for the onboard camera (what it defaults to) so it's best to switch to the USB camera.

There are some bugs like crashing when the card is half off the screen, and stuff like that.

I haven't updated the readme yet, but I'll get around to it once I've added some of my other code.

[raaka1]

Can you share the training data ? :)

[raaka1]

Thanks , i will let you know if there are issues.

[S4WRXTTCS]

I fixed an issue with the cropping of the boundary boxes causing it to crash, and I also edited the Readme to add some information as to what it is.

[S4WRXTTCS]

I added a Dual Net example. Basically it's detectnet-camera that crops the boundary boxes, and sends the image data from each one to ImageNet for classification.

That's likely the last thing I'll add to it for now.