microsoft / cognitive-customvision-windows

Custom Vision Client Library and Sample for Windows

Microsoft Custom Vision Service: Windows Client Library & Sample

This repo is out of date and will be retired on Oct 1, 2019

This repo exists to hold source for the older non Azure integrated version of the Cognitive Services Custom Vision SDK. The older versions will be deprecated on Oct 1, 2019. See our March 26 release notes for more information.

The latest versions can be found via NuGet:

• Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training
• Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction

Source for those packages can be found in the Azure SDK For .NET repo and the latest samples are available with Azure Samples for Custom Vision.

This repo contains the Windows client library & sample for the Microsoft Custom Vision Service, an offering within Microsoft Cognitive Services

The Prediction Client Library

The prediction client library is an automatically generated C# wrapper that allows you to make predictions against trained endpoints.

The easiest way to get the prediction client library is to get the Microsoft.Cognitive.CustomVision.Prediction package from nuget.

The Training Client Library

The training client library is automatically generated C# wrapper that allows you to create, manage and train Custom Vision projects programatically. All operations on the website are exposed through this library, allowing you to automate all aspects of the Custom Vision Service.

The easiest way to get the training client library is to get the Microsoft.Cognitive.CustomVision.Training package from nuget.

The Sample

The sample is a Windows Console application that shows how to create and train a project, then make a prediction against the newly trained endpoint.

Build the sample

1. Start Microsoft Visual Studio 2015 and select File > Open > Project/Solution.

2. Starting in the folder where you clone the repository, go to Samples > CustomVision.Sample

3. Double-click the Visual Studio 2015 Solution (.sln) file CustomVision.Sample.

4. Press Ctrl+Shift+B, or select Build > Build Solution.

Run the sample

After the build is complete, press F5 to run the sample.

The sample will prompt for your training key. You can obtain your training key from the website:

1. Open a project, and go to the project settings (the gear icon)

2. Copy the value listed underneath 'Training Key'.

After entering your training key, hit enter, and the program will run.

The sample uses the Training Client Library to create a new project, add two tags and associated images, and train it. Once trained, the program will use the Prediction Client Library to predict a test image.

The results of the test will be printed out to the screen, and should look like the below image

The sample does not delete the project after it has run. If you visit the website you will be able to see and explore the newly created project.

Contributing

We welcome contributions. Feel free to file issues and pull requests on the repo and we'll address them as we can. Learn more about how you can help on our Contribution Rules & Guidelines.

You can reach out to us anytime with questions and suggestions using our communities below:

• Support questions: StackOverflow
• Feedback & feature requests: Cognitive Services UserVoice Forum

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

License

All Microsoft Cognitive Services SDKs and samples are licensed with the MIT License. For more details, see LICENSE.

Sample images are licensed separately, please refer to LICENSE-IMAGE.

Developer Code of Conduct

Developers using Cognitive Services, including this client library & sample & tools, are expected to follow the “Developer Code of Conduct for Microsoft Cognitive Services”, found at http://go.microsoft.com/fwlink/?LinkId=698895.

cognitive-customvision-windows's Issues

Operation returned an invalid status code: 'NotFound'

I'm using the NuGet package Microsoft.Cognitive.CustomVision.Prediction version 1.2.0. I created 1 trial project and trained it with a few images. Now when I try to call the API for a prediction using the PredicionEndpoint, the system throws an exception: Microsoft.Rest.HttpOperationException.

When I debug the code and inspect the exception, it says:

{"Operation returned an invalid status code 'NotFound'"}

This is my code:

var attachmentStream = await httpClient.GetStreamAsync(imageUrl);

PredictionEndpoint endpoint = new PredictionEndpoint() {
 ApiKey = "MY_CORRECT_PREDICTION_KEY"
};

var predictions = new ImagePredictionResultModel();

try {
 predictions = endpoint.PredictImage(new Guid("MY_CORRECT_PROJECT_ID"), attachmentStream);
} catch (Microsoft.Rest.HttpOperationException exception) {
 await context.PostAsync(exception.Body.ToString());
 await context.PostAsync(exception.Data.ToString());
}

foreach(var c in predictions.Predictions) {
  Console.WriteLine($ "{c.Tag}: {c.Probability}");
}

attachmentStream is a correct stream of an image, I am able to output it to a project-specific view.

The exception gets thrown when calling endpoint.PredictImage(...).

KEYs

Where can i get the training and prediction key

Add support for .NET Standard 2.0

Hi,
Any plans to add support for .NET Standard 2.0 ?
Thanks!

Status code 200 but still cannot load images

Hello, I'm trying to use Training Api. I 'm facing with some issues hard to debug.

My data: I have some images and some regions (zero or more) for each image
My goal: I want to load images on a project I just created
My issue: I follow the steps described in python samples code here. After lots of Bad Request, I understood lots of stuffs (max size for each file, max batch size) and now I get a 200 Response Code, but after that if I check project remote page I cannot see my images loaded. So I used my Pycharm debugger to read thee body of the responde code, and:

{"isBatchSuccessful":false,"images":[{"sourceUrl":"scissors_8","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_6","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_17","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_20","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_9","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_5","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_15","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_2","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_13","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_9","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_18","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_2","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_3","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_4","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_1","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_8","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_12","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_16","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_6","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_15","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_5","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_11","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_13","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_11","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_3","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_18","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_19","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_1","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_12","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_14","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_16","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_19","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_7","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_10","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_7","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_10","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_14","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_4","status":"ErrorUnknown","image":null},{"sourceUrl":"fork_17","status":"ErrorUnknown","image":null},{"sourceUrl":"scissors_20","status":"ErrorUnknown","image":null}]}

Fields "isBatchSuccessful":false and "status":"ErrorUnknown" caught my attention, but I don't know how to debug.

The first time I ran the samples.py script everything worked. Now I'm trying to re-open the project with the api trainer.get_project(<project_id>) and add more images (without creating again classes), but I don't know how to fix my problem.

Can you help me?
Thank you very much.

Getting "Operation returned an invalid status code 'Unauthorized'" when creating a custom vision project for which I have a valid training key

Getting Operation returned an invalid status code 'Unauthorized' when creating a custom vision project for which I have a valid training key.

static void Main(string[] args)
{
// https://customvision.ai/projects#/settings
// Add your training key from the settings page of the portal

        string trainingKey = "<redacted>";

        // Create the Api, passing in the training key
        TrainingApi trainingApi = new TrainingApi() { ApiKey = trainingKey };

        // Create a new project

        Console.WriteLine("Creating new project:");

        var project = trainingApi.CreateProject("My New Project");<== This returns Operation returned an invalid status code 'Unauthorized'

TrainingApi.CreateImagesFromFilesAsync with folder containing more than 55Mb of images throws RequestTimeout

Getting RequestTimeout error thrown when calling TrainingApi.CreateImagesFromFilesAsync - when using folders with large image repositories. Below currentFolder contains 50 images, totalling approx. 60mb. The below code will throw RequestTimeout

var images = Directory.GetFiles(currentFolder).ToList();
var imageFiles = images.Select(img => new ImageFileCreateEntry(Path.GetFileName(img), File.ReadAllBytes(img))).ToList();
var imageBatch = new ImageFileCreateBatch(imageFiles);
var summary = await trainingApi.CreateImagesFromFilesAsync(project.Id, new ImageFileCreateBatch(imageFiles));

Error: RequestTimeout

{Microsoft.Rest.HttpOperationException: Operation returned an invalid status code 'RequestTimeout'
at Microsoft.Cognitive.CustomVision.Training.TrainingApi.d__29.MoveNext()

However, deleting some files to get the total below 55Mb works.

TrainProject returns BadRequest when there are not enough images or tags

The TrainProject method fails with the exception message "BadRequest" when there are not enough images (<5) or tags (<2). The exception should provide a meaningful message like the one you get on the portal.

Source:

"Microsoft.Cognitive.CustomVision.Training"

StackTrace:

at Microsoft.Cognitive.CustomVision.TrainingApi.d__46.MoveNext() in C:\projects\IRIS\Iris.SDK\CustomVisionSDK\Cognitive-CustomVision-Windows\ClientLibrary\Microsoft.Cognitive.CustomVision.Training\Generated\TrainingApi.cs:line 3363
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Cognitive.CustomVision.TrainingApiExtensions.d__61.MoveNext() in C:\projects\IRIS\Iris.SDK\CustomVisionSDK\Cognitive-CustomVision-Windows\ClientLibrary\Microsoft.Cognitive.CustomVision.Training\Generated\TrainingApiExtensions.cs:line 1366
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Cognitive.CustomVision.TrainingApiExtensions.TrainProject(ITrainingApi operations, Guid projectId) in C:\projects\IRIS\Iris.SDK\CustomVisionSDK\Cognitive-CustomVision-Windows\ClientLibrary\Microsoft.Cognitive.CustomVision.Training\Generated\TrainingApiExtensions.cs:line 1349
at CustomVision.Sample.Program.Main(String[] args) in C:\Users\xxxxxx\Source\Repos\Cognitive-CustomVision-Windows\Samples\CustomerVision.Samples\Program.cs:line 75

Newly created project doesn't show up on custom vision portal

**

**
Created yet not visible -

Please advice.

Issue in Predicting Images

Hi

I'm having an issue in predicting the images;

Please let me know how can I resolve them?

Thanks
Arafat

Pull images from directory & Predict not just test_image.jpg

hi @areddish

I believe that would make it far more useful. Also to write the results in a text file as well. Thanks

API key

Where can o get the training key and prediction key

How do I get the project id without using training module?

Hi

I have created the project online on customvision.ai and trained there as well.

I am not sure how can I get the project ID without creating the project from codebehind. Please let me know.

I'm only using the Predictions API, not the training one.

Thanks

API updated but sample not

I have tested CustomVision.Sample and it works well. But when I create a new project and download packages(Microsoft.Cognitive.CustomVision.Prediction,Microsoft.Cognitive.CustomVision.Training)from nuget，there's something wrong in the sample program:

        TrainingApiCredentials trainingCredentials = new TrainingApiCredentials(trainingKey);
        TrainingApi trainingApi = new TrainingApi(trainingCredentials);

        PredictionEndpointCredentials predictionEndpointCredentials = new 
       PredictionEndpointCredentials(predictionKey);
        PredictionEndpoint endpoint = new PredictionEndpoint(predictionEndpointCredentials);

the right way may be this:
TrainingApi trainingApi = new TrainingApi() { ApiKey = trainingKey };
PredictionEndpoint endpoint = new PredictionEndpoint() { ApiKey = predictionKey };

reference:
https://docs.microsoft.com/fi-fi/azure/cognitive-services/custom-vision-service/csharp-tutorial

Related to issue #1: The SDK does not seem to build out of the box

Related to issue #1 - I wanted to include a local build of the SDK with the sample, however, it does not build.

I tried to build the SDK from two different machines - a Windows PC and a Windows Server, both with the newest Visual Studio 2017 installed. They both returned the following errors:

It looks like it is trying to use some modules within the SDK that is neither self-contained nor referenced in the list of references.

Could you provide the config parameters used by the CV trainer?

I'd like to use the exported model w/OpenCV DNN. On edge devices, the OpenCV DNN is much faster and less processor intensive than the Tensorflow lib. The OpenCV DNN works with TensorFlow models from other trainers, and I think it'll work with the Azure Custom Vision models, but they need some tweaks. This repo: https://github.com/opencv/opencv/wiki/TensorFlow-Object-Detection-API has a converter tool but requires the config parameters used for training. The repo has a bunch of sample config files the converter script can use (https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs), but it's not clear which config settings the Custom Vision trainer uses. Could you provide the config file/settings in your github repo or as part of the ZIP file generated by the Export process? Or point to a config file in the OpenCV repo that you think will work? Thanks.

Custome vision exported Onnx not working in windows ML.

I`m currently working on a project where we use the onnx model trained with costume vision on azure in order to perform offline object detection using the Windows.ML runtime. Since this Monday, our models exported from costume vision do not run in WindowsML. When bind method is called "winrt::Windows::AI::MachineLearning::LearningModelBinding" with a winrt::Windows::AI::MachineLearning::ImageFeatureValue created from the SoftwareBitmap of the camera. The bind method returns an exception with the following message: "Model variable data expects Float16[-1, 3, 416, 416]". This happens with all onnx models generated after Monday. Models generated before that work fine. Have Custom Vision changed the onnx format that it generates?

Cannot retrieve Custom Vision SDK from NuGet to build sample

Hi,
I believe the sample generally does not build at the moment, because the Microsoft.Cognitive.CustomVision.Prediction and Microsoft.Cognitive.CustomVision.Training do not install when I try to click restore from NuGet.

I tried on two different machines, a Windows PC and a Windows Server - both with the newest version of Microsoft Visual Studio 2017 - yielding two different errors it seems, however, causing the same problem.

Windows PC:

Windows Server:

Regards, Jens Emil

Update API to 2.0

The Custom Vision API has been updated to 2.0, but the code here and on Nuget is still 1.2.