cleverhans-lab / entangled-watermark Goto Github PK

View Code? Open in Web Editor NEW

40.0 40.0 9.0 30 KB

Python 100.00%

entangled-watermark's People

Contributors

Stargazers

Watchers

Forkers

zeta1999 mooler0410 wh-forker woodencross hongshenghu qinmianer opmusic jeiks grabermtw

entangled-watermark's Issues

Watermark not successful in extracted model

Hi,

Really nice work. but when I ran the code using MNIST dataset, it prints out the following:

Victim Model || validation accuracy: 0.9820713140070438, watermark success: 1.0 Extracted Model || validation accuracy: 0.981770833954215, watermark success: 0.03013390302658081， baseline: 0.006696428571428571

Does this means the watermark fail to transfer to the victim model? If so, what could have gone wrong? I ran the code with all default settings and seed.

Unknowns parameters for BatchNormalization

Hello,
When migrating your code from TF1 to TF2, I cannot figure out what some parameters for batch_normalization (in resnet.py, line 153) are : decay=0.9, updates_collections=None, scope=scope

My research gave me the following results :

momentum = 1 - decay (according to keras-team/keras#6839)
updates_collections : is it related to parameter 'Trainable' of BatchNormalization in TF2 ?
scope : i understand the objective of scope, but cannot understand why it is used here.

here is my (working) code without those parameters using TF2 :

def batch_norm(x, is_training=True, scope='batch_norm'):
    return tf_keras.layers.BatchNormalization(epsilon=1e-05,
                                              center=True, scale=True)(x, training=is_training)

Thx a lot,

Low watermark success on extracted model

Hello,
I tried to reproduce the results reported in the article using tensorflow==1.14.0 as recommended. In particular I ran your original code, training on MNIST and default hyperparameters (python train.py --dataset mnist --default 1) and got the following results:

Victim Model || validation accuracy: 0.9899259870871902, watermark success: 1.0
Extracted Model || validation accuracy: 0.9869449017569423, watermark success: 0.004720052083333333

Running a few more times the results continue to oscillate around these values.

I have done the same for Fashion MNIST (train.py --dataset fashion --default 1) obtaining analogous results:

Victim Model || validation accuracy: 0.9912860579788685, watermark success: 1.0 
Extracted Model || validation accuracy: 0.9904847759753466, watermark success: 0.0015285326086956522

Do you have any idea what the problem might be? Thanks.

Unable to replicate default training on CIFAR10

Hello,
I tried python3 train.py --dataset cifar10 **--default 1** with tensorflow 1.x, on the unmodified code cloned from this repo.
Training took ~3h30, but results are : Victim Model || validation accuracy: 0.8653846085071564, watermark success: 0.10416666666666667 which seems very poor performance.

Have you recently been able to run this code successfully ? Thank you for your attention.

The logs are :

WARNING:tensorflow:Entity <function _get_dataset_from_filename at 0x7f84171da950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Index'
WARNING:tensorflow:Entity <function _get_dataset_from_filename at 0x7f84171da950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Index'
WARNING:tensorflow:Entity <bound method TopLevelFeature.decode_example of FeaturesDict({
    'id': Text(shape=(), dtype=tf.string),
    'image': Image(shape=(32, 32, 3), dtype=tf.uint8),
    'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10),
})> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:Entity <bound method TopLevelFeature.decode_example of FeaturesDict({
    'id': Text(shape=(), dtype=tf.string),
    'image': Image(shape=(32, 32, 3), dtype=tf.uint8),
    'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10),
})> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
2021-06-29 09:27:23.750566: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2021-06-29 09:27:23.788338: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:23.789144: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: 
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
2021-06-29 09:27:23.789449: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2021-06-29 09:27:23.791271: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2021-06-29 09:27:23.798880: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2021-06-29 09:27:23.799260: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2021-06-29 09:27:23.801274: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2021-06-29 09:27:23.809596: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2021-06-29 09:27:23.825124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-06-29 09:27:23.825318: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:23.826240: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:23.827010: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0
WARNING:tensorflow:Entity <function _get_dataset_from_filename at 0x7f84171da950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Index'
WARNING:tensorflow:Entity <function _get_dataset_from_filename at 0x7f84171da950> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: module 'gast' has no attribute 'Index'
WARNING:tensorflow:Entity <bound method TopLevelFeature.decode_example of FeaturesDict({
    'id': Text(shape=(), dtype=tf.string),
    'image': Image(shape=(32, 32, 3), dtype=tf.uint8),
    'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10),
})> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
WARNING:tensorflow:Entity <bound method TopLevelFeature.decode_example of FeaturesDict({
    'id': Text(shape=(), dtype=tf.string),
    'image': Image(shape=(32, 32, 3), dtype=tf.uint8),
    'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=10),
})> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: Bad argument number for Name: 3, expecting 4
2021-06-29 09:27:23.979333: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2299995000 Hz
2021-06-29 09:27:23.979606: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55bfb4472d80 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2021-06-29 09:27:23.979644: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2021-06-29 09:27:24.089919: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:24.090751: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55bfb4472680 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2021-06-29 09:27:24.090789: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Tesla K80, Compute Capability 3.7
2021-06-29 09:27:24.090898: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-06-29 09:27:24.090920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      
2021-06-29 09:27:24.093346: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-06-29 09:27:24.093387: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      
tcmalloc: large alloc 1228800000 bytes == 0x55bfd00da000 @  0x7f8489c661e7 0x7f8486bd746e 0x7f8486c27c7b 0x7f8486c27d18 0x7f8486ce3d79 0x7f8486ce6e4c 0x7f8486e05e7f 0x7f8486e0bfb5 0x7f8486e0de3d 0x7f8486e0f516 0x55bfb339bf30 0x55bfb339bb09 0x7f8486cee4d8 0x55bfb337e303 0x55bfb3484646 0x55bfb340c785 0x55bfb34094ae 0x55bfb34091b3 0x55bfb34d3182 0x55bfb34d34fd 0x55bfb34d33a6 0x55bfb34aa723 0x55bfb34aa3cc 0x7f8488a50bf7 0x55bfb34aa2aa
2021-06-29 09:27:35.067325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-06-29 09:27:35.067420: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      
2021-06-29 09:27:35.069059: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-06-29 09:27:35.069099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      
tcmalloc: large alloc 1800364032 bytes == 0x55c037306000 @  0x7f8489c661e7 0x7f8486bd746e 0x7f8486c27c7b 0x7f8486c27d18 0x7f8486ce3d79 0x7f8486ce6e4c 0x7f8486e05e7f 0x7f8486e0bfb5 0x7f8486e0de3d 0x7f8486e0f516 0x55bfb339bf30 0x55bfb339bb09 0x7f8486cee4d8 0x55bfb337e303 0x55bfb3484646 0x55bfb340c785 0x55bfb34097ad 0x55bfb339c3ea 0x55bfb340a3b5 0x55bfb34094ae 0x55bfb34091b3 0x55bfb34d3182 0x55bfb34d34fd 0x55bfb34d33a6 0x55bfb34aa723 0x55bfb34aa3cc 0x7f8488a50bf7 0x55bfb34aa2aa
WARNING:tensorflow:From train.py:111: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From train.py:111: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.

WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/image_ops_impl.py:1518: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/image_ops_impl.py:1518: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Deprecated in favor of operator or tf.math.divide.
WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/models.py:71: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/models.py:71: The name tf.variable_scope is deprecated. Please use tf.compat.v1.variable_scope instead.

WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/resnet.py:26: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.keras.layers.Conv2D` instead.
WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/resnet.py:26: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.keras.layers.Conv2D` instead.
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/layers/convolutional.py:424: Layer.apply (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `layer.__call__` method instead.
WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/resnet.py:126: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/resnet.py:126: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/resnet.py:33: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/resnet.py:33: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.Dense instead.
WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/models.py:176: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From /content/PAF/entangled_watermark_v1/models.py:176: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From /tensorflow-1.15.2/python3.7/tensorflow_core/python/ops/math_grad.py:1424: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From train.py:129: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From train.py:129: The name tf.train.Saver is deprecated. Please use tf.compat.v1.train.Saver instead.

WARNING:tensorflow:From train.py:130: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

WARNING:tensorflow:From train.py:130: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2021-06-29 09:27:46.283988: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:46.284790: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Found device 0 with properties: 
name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235
pciBusID: 0000:00:04.0
2021-06-29 09:27:46.284891: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2021-06-29 09:27:46.284942: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2021-06-29 09:27:46.284991: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2021-06-29 09:27:46.285038: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2021-06-29 09:27:46.285081: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2021-06-29 09:27:46.285124: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2021-06-29 09:27:46.285169: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2021-06-29 09:27:46.285265: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:46.286123: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:46.286943: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1767] Adding visible gpu devices: 0
2021-06-29 09:27:46.291237: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2021-06-29 09:27:46.292958: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1180] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-06-29 09:27:46.292992: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1186]      0 
2021-06-29 09:27:46.293011: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1199] 0:   N 
2021-06-29 09:27:46.293289: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:46.294116: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-06-29 09:27:46.294914: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.
2021-06-29 09:27:46.294960: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1325] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10813 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)
WARNING:tensorflow:From train.py:131: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

WARNING:tensorflow:From train.py:131: The name tf.global_variables_initializer is deprecated. Please use tf.compat.v1.global_variables_initializer instead.

Victim Model || validation accuracy: 0.8653846085071564, watermark success: 0.10416666666666667

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.