Pre-trained Models about prettytensor

[eiderman]

When you set up the tf.Saver you can pass a dict of names -> variables. You can use this to map existing weights with names to variables from TF checkpoints. A good practice is to assign names to every layer so that this mapping either becomes unnecessary because it lines up or the mapping is trivial.

The Pretty Tensor objects expose the generated weights and bias parameters using layer_parameters (e.g. my_tensor.layer_paramerters['weights']).

If your weights are not in checkpoints, then you can create assign nodes for each variable (assigner = tf.assign(var, tf.placeholder(NAME)) and then run it with a placeholder (sess.run(assigner, {NAME: numpy_array})

I can see that for a very large model, either of these would be quite tedious and a mechanism to scope the mapping names would be nice, but I haven't started working on this. If you are interested in contributing, the hooks would be in pretty_tensor_class.py / VarStoreMethod and bookkeeper.py

[nishithbsk]

Hi,

Thanks a lot for the detailed response! I had a follow up question more specific to my implementation.

My network resembles the following:

    return (pt.wrap(input_tensor).
            reshape([batch_size, image_size[0], image_size[1], 3]).
            conv2d(5, 32, stride=2, name='conv1').
            conv2d(5, 64, stride=2, name='conv2').
            conv2d(3, 128, stride=2, name='conv3').
            conv2d(3, 256, stride=2, name='conv4').
            conv2d(3, 512, stride=2, name='conv5').
            flatten().
            fully_connected(hidden_size, activation_fn=None, name='fc1')).tensor

Is there a way to obtain the variable associated with conv1 prettytensor layer? I would need it for var in (assigner = tf.assign(var, tf.placeholder(NAME)). I tried using tf.get_variable('conv1') within the scope of the model in my train function but tensorflow couldn't find any such variable.

Thank you so much!

[eiderman]

The weights for conv1 would be 'conv1/weights' and the bias would be 'conv1/bias' in the appropriate scope.

To use tf.get_variable to fetch the variable, you will need be in a reusing scope.

[nishithbsk]

Thank you!