Training doesn't work about tf2-mobile-2d-single-pose-estimation

[grin]

So if anyone lands here: this repo is broken. I managed to get it working with the ^^^ above changes plus I had to replace the whole hourglass model. I used this implementation as a reference: https://github.com/yuanyuanli85/Stacked_Hourglass_Network_Keras/blob/eddf0ae15715a88d7859847cfff5f5092b260ae1/src/net/hg_blocks.py . It's implemented in Keras so it's easy to convert to TF2.

[tucan9389]

@grin

Thank you for answering.

Currently, the master branch is not properly working. I'm trying to solve some trouble by making a tmp branch. The code of the tmp branch worked properly, and the conversion to the coreml (or tflite) model is being carried out now.
I will merge the tmp branch to the master soon.

[grin]

I'm seeing very similar broken prediction heatmaps after 70 epochs of training with loss 0.0046

[grin]

@tucan9389 I think I found some errors in the code. Here: https://github.com/tucan9389/tf2-mobile-pose-estimation/blob/5248196af945ec81ae1935fd7fbb482e0e3ccf72/hourglass_model.py#L106-L113

Based on my understanding of how the Hourglass network works, it shouldn't merge and average the intermediate heatmaps. Instead, it should just return:

return hg_out, intermediate_heatmap_layers

And later, where you create the model, here: https://github.com/tucan9389/tf2-mobile-pose-estimation/blob/5248196af945ec81ae1935fd7fbb482e0e3ccf72/hourglass_model.py#L43-L45

The intermediate heatmaps should be set as outputs of the model, so it should look something like this:

        hg_next_stage, l2s = self.build_network(inputs, trainable=trainable) 
       
        self.model = tf.keras.Model(inputs=inputs, outputs=l2s)

But for this to work, the data loader should also repeat the label set for each image STAGE_NUM times. So that each intermediate output would get its own labels.

Or, am I missing something?

[aeyoa]

Thank you, @grin! I've already solved my problem using different approach. But it's interesting to get the clarification!