deeplite / deeplite-profiler Goto Github PK
View Code? Open in Web Editor NEWA collection of metrics to profile a single deep learning model or compare two different deep learning models
License: Apache License 2.0
A collection of metrics to profile a single deep learning model or compare two different deep learning models
License: Apache License 2.0
The volatile memory (usually on-chip SRAM) hosts the activations and the buffer is used as a scratchpad for auxiliary procedures (e.g., the im2col buffer for GEMM-based convolutions).
So for each Conv Layer, the standard approach requires to allocate INPUT + OUTPUT activation buffer in SRAM, releasing the INPUT just after the whole layer computation is finished.
During the feed-forward pass of a ConvNet, a sequence of operators (layers), read/store the tensors, from/to a contiguous region of the volatile memory (memory pool). Each tensor is allocated with an offset address defining the portion of the memory pool where it will be hosted during its lifetime. Since tensors are accessed by a limited number of layers, and layers are scheduled sequentially, the memory previously assigned to a tensor can be reused over time to store other non-conflicting tensors.
Peak Memory: highest memory value across all the layer computation.
For standard models (with a regular chain-like graph topology) it is easier: Ram(layer) ~= InputAct + OutAct. However, for more complex graph topologies the situation change: complex layers connectivity and tied tensors dependencies make the memory allocation problem much harder to solve efficiently in practice.
Refs:
profiler.compute_network_status(batch_size=1, device=Device.CPU, short_print=False,
File "deeplite/profiler/profiler.py", line 200, in deeplite.profiler.profiler.Profiler.compute_network_status
File "deeplite/profiler/profiler.py", line 158, in deeplite.profiler.profiler.Profiler.compute_status
File "deeplite/profiler/profiler.py", line 362, in deeplite.profiler.profiler.ProfilerFunction.pipe_kwargs_to_call
File "deeplite/torch_profiler/torch_profiler.py", line 52, in deeplite.torch_profiler.torch_profiler.ComputeComplexity.call
File "deeplite/torch_profiler/torch_profiler.py", line 60, in deeplite.torch_profiler.torch_profiler.ComputeComplexity._compute_complexity
File "/home/ubuntu/anaconda3/envs/cerberus/lib/python3.8/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/home/ubuntu/anaconda3/envs/cerberus/lib/python3.8/copy.py", line 270, in _reconstruct
state = deepcopy(state, memo)
File "/home/ubuntu/anaconda3/envs/cerberus/lib/python3.8/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/home/ubuntu/anaconda3/envs/cerberus/lib/python3.8/copy.py", line 230, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/home/ubuntu/anaconda3/envs/cerberus/lib/python3.8/copy.py", line 153, in deepcopy
y = copier(memo)
TypeError: 'dict' object is not callable
Sources:
Try OpenVino Benchmark
Limitations: Supports only .xml
and .onnx
format:
-m PATH_TO_MODEL, --path_to_model PATH_TO_MODEL
Required. Path to an .xml/.onnx/.prototxt file with a
trained model or to a .blob file with a trained
compiled model.
torchvision.hub
tf.keras.applications
In order to support torchvision models like mask-rcnn
, faster-rcnn
and keypoints-rcnn
for our engine we need to support
bboxes
and labels
Support AP by obj size for object detection. Will need implementation in zoo as well
For both python3.6
and python3.71
when running the examples/tf-example.py
the following error happens:
Traceback (most recent call last):
File "examples/tf_example.py", line 60, in <module>
include_weights=True, print_mode='info')
File "/home/ahmed/projects/deeplite-profiler/deeplite/profiler/profiler.py", line 204, in compute_network_status
self.compute_status(sk, recompute=recompute, **kwargs)
File "/home/ahmed/projects/deeplite-profiler/deeplite/profiler/profiler.py", line 162, in compute_status
rval = pf_func.pipe_kwargs_to_call(self.model, self.data_splits, kwargs)
File "/home/ahmed/projects/deeplite-profiler/deeplite/profiler/profiler.py", line 382, in pipe_kwargs_to_call
return self(**bounded_args.arguments)
File "/home/ahmed/projects/deeplite-profiler/deeplite/tf_profiler/tf_profiler.py", line 192, in __call__
return self._compute_exectime(temp_model, dataloader, batch_size=batch_size)
File "/home/ahmed/projects/deeplite-profiler/deeplite/tf_profiler/tf_profiler.py", line 203, in _compute_exectime
_ = model(tnum, training=False)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 547, in __call__
inputs = nest.map_structure(_convert_non_tensor, inputs)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 515, in map_structure
structure[0], [func(*x) for x in entries],
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/util/nest.py", line 515, in <listcomp>
structure[0], [func(*x) for x in entries],
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 544, in _convert_non_tensor
return ops.convert_to_tensor(x)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1087, in convert_to_tensor
return convert_to_tensor_v2(value, dtype, preferred_dtype, name)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1145, in convert_to_tensor_v2
as_ref=False)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1224, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 305, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 246, in constant
allow_broadcast=True)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 254, in _constant_impl
t = convert_to_eager_tensor(value, ctx, dtype)
File "/home/ahmed/projects/deeplite-profiler/_env3.6/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 115, in convert_to_eager_tensor
return ops.EagerTensor(value, handle, device, dtype)
RuntimeError: Error copying tensor to device: /job:localhost/replica:0/task:0/device:GPU:0. /job:localhost/replica:0/task:0/device:GPU:0 unknown device.
They could use some love
Once the evaluation function is registered to a profiler's instance, any secondary metric added to it will silently be ignored. This happens because the binding of the profiler to a pfunction is fixed after registry. The profiler then fails at catching up with the dynamic status key.
I would either refactor the profiler to handle dynamic status keys all across the board or block adding a secondary metric once the function is registered.
Also, I don't really like the global REGISTER at the class level of DynamicEvalMetric
... if two profiler coexists with their respective set of eval metrics it looks like a recipe for errors.
The current table is not copy pastable in .md
format. It would require small change to do so
Note: For computing max_batch_size metric, when overloading the CPU RAM with data, the following error could occur.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.