Error in conversion of Inception Model from Keras to CNTK about mmdnn

[kitstar]

Hi @coolrishi2005 ,

The input layer of keras InceptionV3 doesn't contain shapes info, which is placed in model config batch_input_shape. So it can't be parsed correctly.

To fix it, please change the code

BackEndModel = applications.InceptionV3(include_top=False, weights='imagenet')

to

BackEndModel = applications.InceptionV3(input_shape=(299, 299, 3), include_top=False, weights='imagenet')

And I tested it with follow scripts

$ python -m mmdnn.conversion._script.convertToIR -f keras -d ./kit_imagenet -n incpetionTopModel.json -w incpetionTopModel.h5
$ python -m mmdnn.conversion._script.IRToCode --dstModelFormat cntk --IRModelPath kit_imagenet.pb --dstModelPath kit_imagenet.py --IRWeightPath kit_imagenet.npy
$ python -m mmdnn.conversion.examples.cntk.imagenet_test -n kit_imagenet.py -w kit_imagenet.npy --dump cntkmodel.dnn

CNTK model file is saved as [cntkmodel.dnn], generated by [kit_imagenet.py] and [kit_imagenet.npy].

Hope it can help you.

[coolrishi2005]

Hi kitstar,

Thanks for the help. The model has been converted to *.dnn, but with the following warning:

(C:\Program Files\Anaconda3\envs\py35) D:\mmdnn\conversion\cntk>python -m mmdnn.conversion.examples.cntk.imagenet_test
-n cntkInceptionTopModel.py -w IRInceptionTopModel.npy --dump cntkInceptionTopMo
del.dnn
Selected CPU as the process wide default device.
C:\Program Files\Anaconda3\envs\py35\lib\site-packages\cntk\core.py:82: RuntimeW
arning: data is not C contiguous; rearrange your data/computation to avoid costl
y data conversions
RuntimeWarning)
CNTK model file is saved as [cntkInceptionTopModel.dnn], generated by [cntkIncep
tionTopModel.py] and [IRInceptionTopModel.npy].

Is it fine?

[kitstar]

Supposed to be fine.

[coolrishi2005]

Great!! Thanks.

[coolrishi2005]

Hi kitstar,

I have saved the created and saved the Inception model with following parameters:
BackEndModel = applications.InceptionV3(input_shape=(150, 150, 3), include_top=False, weights='imagenet')

I am trying to load the above saved model (cntkInceptionTopModel.dnn) using cntk CPP Evaluation Example provided by Microsoft (https://github.com/Microsoft/CNTK/blob/release/2.3/Examples/Evaluation/CNTKLibraryCPPEvalCPUOnlyExamples/CNTKLibraryCPPEvalCPUOnlyExamples.cpp)

Now, for evaluation, I am providing binary RGB data as input (which is of size 150x150x3, i.e. 67500 bytes data). Following is the code for the same:

void EvaluationSingleSampleUsingDense(const wchar_t* modelFile, const DeviceDescriptor& device)
{
printf("\n===== Evaluate single sample using dense format.\n");

// Load the model.
// The model is trained by <CNTK>/Examples/Image/Classification/ResNet/Python/TrainResNet_CIFAR10.py
// Please see README.md in <CNTK>/Examples/Image/Classification/ResNet about how to train the model.
FunctionPtr modelFunc = Function::Load(modelFile, device);

// Get input variable. The model has only one single input.
Variable inputVar = modelFunc->Arguments()[0];

// The model has only one output.
// If the model has more than one output, use modelFunc->Outputs to get the list of output variables.
Variable outputVar = modelFunc->Output();
NDShape outputShape = outputVar.Shape();
std::vector<size_t> outputShapeDim = outputShape.Dimensions();

// Prepare input data.
// For evaluating an image, you first need to perform some image preprocessing to make sure that the input image has the correct size and layout
// that match the model inputs.
// Please note that the model used by this example expects the CHW image layout.
// inputVar.Shape[0] is image width, inputVar.Shape[1] is image height, and inputVar.Shape[2] is channels.
// For simplicity and avoiding external dependencies, we skip the preprocessing step here, and just use some artificially created data as input.
NDShape inputShape = inputVar.Shape();
std::vector<size_t> inputShapeDim = inputShape.Dimensions();

unsigned char inputDataChar[67500];
std:string filepath = "D:\\Rishi\\CNTK Evaluation\\TestCNTK\\TestCNTK\\videoRGBData.rgb";
std::basic_ifstream<unsigned char> infile(filepath, std::ios::binary | std::ios::in);
infile.read(inputDataChar, 67500);
infile.close();

std::vector<float> inputData(inputVar.Shape().TotalSize());
/*for (size_t i = 0; i < inputData.size(); ++i)
{
	inputData[i] = static_cast<float>(i % 255);
}*/

for (int i = 0; i < 67500; i++)
	inputData[i] = (float)inputDataChar[i];

// Create input value and input data map
cout << inputVar.Shape();
ValuePtr inputVal = Value::CreateBatch(inputVar.Shape(), inputData, device);
std::unordered_map<Variable, ValuePtr> inputDataMap = { { inputVar, inputVal } };

// Create output data map. Using null as Value to indicate using system allocated memory.
// Alternatively, create a Value object and add it to the data map.
std::unordered_map<Variable, ValuePtr> outputDataMap = { { outputVar, nullptr } };

// Start evaluation on the device
modelFunc->Evaluate(inputDataMap, outputDataMap, device);

// Get evaluate result as dense output
ValuePtr outputVal = outputDataMap[outputVar];
std::vector<std::vector<float>> outputData;
outputVal->CopyVariableValueTo(outputVar, outputData);

PrintOutput<float>(outputVar.Shape().TotalSize(), outputData);

}

But, during the call of modelFunc->Evaluate(inputDataMap, outputDataMap, device), I am getting following runtime error:
Unhandled exception at 0x00007FFE1527FEDE (Cntk.Core-2.3d.dll) in CNTKLibraryCPPEvalCPUOnlyExamples.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x000000474F603FD0). occurred

Can you please help?

[kitstar]

Hi,
Does the converted model with original input shape (299, 299, 3) work?
If it works, maybe resizing your image in your application is a workaround.

[coolrishi2005]

Hey, no issue. I am working on it.
Thanks :)