rapidsai / cucim Goto Github PK
View Code? Open in Web Editor NEWcuCIM - RAPIDS GPU-accelerated image processing library
Home Page: https://docs.rapids.ai/api/cucim/stable/
License: Apache License 2.0
cuCIM - RAPIDS GPU-accelerated image processing library
Home Page: https://docs.rapids.ai/api/cucim/stable/
License: Apache License 2.0
HI cuCIM staff
My company is doing image processing stuff, and the image format provided is '.svs'.
Since we are interesting in this tool kit, we convert the .svs to .tif format by tifffile package.
I got this error message when I ran code below.
"
input_file = "1M09_1.tif"
slide = cucim.CuImage(input_file)
"
OS: Ubuntu 18.04
The point is I really don't know how to fix this error.
BTW, I did try with another image(get from online) in this step. And it works as normal. So, I guess my cuCIm installed correctly.
Is your feature request related to a problem? Please describe.
I recently proposed a PR upstream in scikit-image that decomposes a binary footprint into a series of smaller footprints: scikit-image/scikit-image#5482. On the CPU this gives large performance improvements when working with larger footprints.
Describe the solution you'd like
This would be good to adapt here as morphology operations on the GPU are currently much faster on small sized elements than large ones, likely due in part to much more favorable memory access patterns.
Describe alternatives you've considered
There are a number of alternative ways to do the decomposition, but this one is straightforward to implement and wouldn't require any changes upstream to CuPy itself.
Additional context
I have not yet benchmarked this approach on the GPU, but am confident it will generally be faster here as well.
Is your feature request related to a problem? Please describe.
We would like to start thinking about supporting BioFormats.
Bioformats is a pathway to access so many formats that we'll never want to implement ourselves.
Even if it's in Java, may as well find a way to hook into it as an option. There are way too many formats that we will have a hard time supporting. Bioformats handles lots of them, including proprietary ones.
Describe the solution you'd like
TBD
Describe alternatives you've considered
We can make it easy to load various microscopy image formats while we support the formats in an optimized way.
One concern is that for such adaptation layer (plugin) without optimization, there is already a solution in Python so doing it in cuCIM could be a duplicated effort:
Do we also want to provide such integration without optimization?
Additional context
N/A
What is your question?
When we originally ran isort
on the skimage
module we used the setting multi_line_output = 0
which results in a fairly compact import style. For example:
cucim/python/cucim/src/cucim/skimage/morphology/__init__.py
Lines 1 to 8 in 78ca078
but if I rerun isort
on that file now, it seems there has been a force_single_line = True
setting added to setup.cfg
which result in the following more verbose style:
from ._skeletonize import thin
from .binary import binary_closing
from .binary import binary_dilation
from .binary import binary_erosion
from .binary import binary_opening
from .grey import black_tophat
from .grey import closing
from .grey import dilation
from .grey import erosion
from .grey import opening
from .grey import white_tophat
from .greyreconstruct import reconstruction
from .misc import remove_small_holes
from .misc import remove_small_objects
from .selem import ball
from .selem import cube
from .selem import diamond
from .selem import disk
from .selem import octagon
from .selem import octahedron
from .selem import rectangle
from .selem import square
from .selem import star
Can we switch back to force_single_line = False
or is there a reason to prefer the single line style? I looked at a few other RAPIDS projects and the style is not uniform across them. cuml
does seem to have a single line style, but cudf
and cusignal
do not.
The clara
package here is using the single line style, but the skimage
package is not. Whichever one we choose we should make the two consistent!
Is your feature request related to a problem? Please describe.
Current implementation is not handling error cases well -- memory allocated is manually freed by calling free()
or cucim_free()
method.
We need to improve the whole code to use RAII.
Describe the solution you'd like
Create a utility class to handle resources.
This is a question migrated over from cupyimg
: will there be multi-GPU support for GPU-enabled scikit-image
functions?
phase_cross_correlation
and affine_transform
are two functions I have in mind. These are probably ubiquitously used functions in biomedical imaging for image registration.
Problem is, for certain problems where image resolution is key and down-sampling is not possible (ie. single-molecule spatial transcriptomics), there's a need to register large 3D volumes that cannot be loaded into one GPU alone. Being able to "chunk" the image not unlike dask
would be extremely helpful. I realize this is easier said than done!
I'm actually very keen on learning how to implement this, for personal growth and for to benefit the community - but need some guidance. Would the authors be able to comment on this utility?
Design and Implement image patch generator by loading raw compressed image at once using GDS and provide a series of image patches by decoding multiple tile images in batch using nvJPEG
We may want to revisit at some point and look at using std::filesystem::path
. Though that requires C++17 and we were already using std::string
. So just a future note
Originally posted by @jakirkham in #128 (comment)
Hello!
I very often use function Non-local means denoising - it really good denoising method for preserving textures
I used to: from skimage.restoration import denoise_nl_means"
I tried "from cucim.skimage.restoration import denoise_nl_means" but it doesnt work. :(
Here is link: https://scikit-image.org/docs/dev/auto_examples/filters/plot_nonlocal_means.html
Is your feature request related to a problem? Please describe.
Currently, we don't use Carbonite's dependent plugin-loading mechanism.
Instead, we manager plugins separately, making some plugins built-in plugins:
cucim/cpp/include/cucim/plugin/plugin_config.h
Lines 35 to 36 in 95e4483
And commented on some methods from Carbonite code that are not ported yet:
cucim/cpp/include/cucim/core/framework.h
Line 66 in 95e4483
cucim/cpp/src/core/framework.cpp
Lines 40 to 44 in 95e4483
cucim/cpp/src/core/cucim_framework.cpp
Lines 107 to 110 in 95e4483
We need to revisit current Carbonite integration someday.
Describe the solution you'd like
Revisit Carbonite SDK and use (minimal) SDK as it is, once it is available:
For medical imaging inference, DICOM loading, resampling, morphology, window location and width adjustment are commonly used in both training and inference. However most of them are in GPU packages like ITK/VTK, numPy etc. There is either a toolkit set nor an example pipeline/tutorial to support these tasks on GPU.
Following is expected features:
1. A Toolkit package for pre- and post processing including but not limited to DICOM Loading, Volume resampling, Windows location/width adjusting, batching, morphology filtering, connect component extraction etc.
2. A demo/example code that could use above features to make a inference pipeline on GPU, together with AI inference. So the whole inference pipeline is on GPU.
3. Compling tools that could deploy the pipeline as a compiled engine.
Is your feature request related to a problem? Please describe.
It would be great if we have a foundation for malloc and memory pool implementation.
cuCIM C++ implementation currently tries to use a separate call (cucim_malloc()
) to abstract malloc functionality.
it currently does not use memory pool for read_region()
method and cache.
We can make use of memory pool implementation for that.
Describe the solution you'd like
According to https://lwn.net/SubscriberLink/872869/0e62bba2db51ec7a/, mimalloc
(https://github.com/microsoft/mimalloc) seems to be a good candidate.
We can also consider other implementations like tcmalloc
(https://github.com/google/tcmalloc).
For memory pool, we can exploit PMR(std::pmr::monotonic_buffer_resource) and rmm.
With #100 in, how should we document these new functions ? Should we simply add cucim.skimage.operations
? Or broken out for each subsection: cucim.core.operations.color
/ cucim.core.operations.expose.transform
/ etc
Is your feature request related to a problem? Please describe.
It's be great if cucim
implements skimage
's watershed
algorithm. watershed
is really useful for segmentation and it'd be great if we had a GPU implementation of it.
Describe the solution you'd like
Implement skimage
's watershed algorithm.
Describe alternatives you've considered
None
Additional context
None
This issue is related to #49 and would like to discuss and track the issue here.
(@grlee77 Please feel free to edit/update this description).
Watershed algorithm in scikit-image (https://scikit-image.org/docs/dev/api/skimage.segmentation.html?highlight=watershed#skimage.segmentation.watershed) is a popular algorithm for segmentation. However, cuCIM doesn't support it so would like to provide the algorithm through cucim.skimage.segmentation.watershed
method.
It turns out that using the same algorithm used in scikit-image with CuPy is not feasible or tricky to implement:
There are some GPU implementations:
Describe the bug
For Aperio svs files, I run into a cryptic error:
/slide.svs: Compression scheme 33003 tile decoding is not implemented.
However, cuCMI still loads a tile and doesn't raise but the color are wrong or it is entirely black. See the following grid of tiles:
Are all OpenSlide vendors supported? Is there a list of all supported vendors somewhere? I could not find it.
Steps/Code to reproduce bug
# Download the TCGA-F4-6459-01Z-00-DX1 slide from https://portal.gdc.cancer.gov/cases/9fd08502-355b-4f5a-a25c-73ec7184f6d3?bioId=57f4136d-d6aa-4537-ba73-f21fe0374005
import numpy as np
from PIL import Image
from cucmi import CuImage
slide = CuImage("TCGA-F4-6459-01Z-00-DX1")
tile = slide.read_region((0, 0), (224, 224), 0)
Image.fromarray(np.asarray(tile))
Is your feature request related to a problem? Please describe.
Since the project doesn't have a mechanism for preventing memory leaks, there are high chances to introduce memory leaks during the development.
Describe the solution you'd like
Additional context
There are many cases that we introduced memory leak problems.
Report needed documentation
It would be useful to expand the contributor documentation to explain how the python package can be installed as an "editable install".
When I tried a naive pip install -e . -v
call from within the python/cucim
folder where setup.py
is contained, this runs fine, but I am unable to import the package. I think the issue is that the cucim
module folder is within a src
subfolder rather than directly in the folder containing setup.py
.
Describe the documentation you'd like
A workaround that fixes it for me, it to add the following to the setup.cfg
:
[egg_info]
egg_base = src
which causes the cucim.egg-info
folder to get placed into the src
subfolder rather than in the same folder as setup.py
itself.
Steps taken to search for needed documentation
read CONTRIBUTING.md
For some svs files, I run into a cryptic "error":
/slide.svs: Compression scheme 33003 tile decoding is not implemented.
"Error" is in quote as it doesn't seem to have any effect, the tile is still loaded. It seems to be linked to the Aperio vendor.
Are all OpenSlide vendors not supported yet? If so, why is the tile correctly loaded? Is there any flag I can set to stop this error from spamming my logs?
Describe the bug
Since v21.06.00, cuCIM silenced ImportError exception from from .clara import CuImage, __version__, cli
to support cucim.skimage
package-only use case.
cucim/python/cucim/src/cucim/__init__.py
Line 44 in 3afa154
try:
import cupy
except ImportError:
pass
try:
from .clara import CuImage, __version__, cli
except ImportError:
from ._version import get_versions
__version__ = get_versions()['version']
del get_versions
del _version
That caused an issue on MONAI's tests because it wouldn't raise ImportError even if loading libcucim.so was failed, and it causes MONAI's optional_import() method to return True
.
MONAI team (@wyli @drbeh) has the following workaround to check cucim.CuImage
's availability.
drbeh/MONAI@a102ab7#diff-fcffca442c7cee4391ab58c3d2b71b80134da7621262457c69a3741308e156cd
Steps/Code to reproduce bug
import cucim
img = cucim.CuImage("input.tif")
Expected behavior
import cucim
if cucim.CuImage is not available.Environment details (please complete the following information):
Additional context
We may want to expose a method in (cucim
package) to to check if image loader(cucim.clara) or image processor(cucim.skimage, cucim.core) are available so that user can check the real availability.
cucim.is_available()
: check if all modules are available.cucim.is_available("clara")
: check if image loader-related modules are available.cucim.is_available("skimage")
cucim.is_available("core")
scikit-image/scikit-image#5158: Add normalized mutual information metric
scikit-image/scikit-image#5308: New illuminants were added to the color conversions
scikit-image/scikit-image#5382: Added ND butterworth filter
scikit-image/scikit-image#5420: Add no-reference perceptual blur metric.
Other easy functions are skimage.feature.blob_dog
and skimage.feature.blob_log
. Those two existed in v0.18, but had either bug fixes and/or a new argument added in v0.19.
scikit-image 0.19 adds a channel_axis
argument that should now be used instead of the multichannel
boolean. In scikit-image 1.0, the multichannel
argument will likely be removed. We should start supporting channel_axis
in cuCIM. Corresponding upstream PRs are:
scikit-image/scikit-image#5228: Decorators for helping with the multichannel->channel_axis transition
scikit-image/scikit-image#5284: multichannel to channel_axis (1 of 6): features and draw
scikit-image/scikit-image#5285: multichannel to channel_axis (2 of 6): transform functions
scikit-image/scikit-image#5286: multichannel to channel_axis (3 of 6): filters
scikit-image/scikit-image#5287: multichannel to channel_axis (4 of 6): metrics and measure
scikit-image/scikit-image#5288: multichannel to channel_axis (5 of 6): restoration
scikit-image/scikit-image#5289: multichannel to channel_axis (6 of 6): segmentation
scikit-image/scikit-image#5462: Add a channel_axis argument to functions in the skimage.color module
scikit-image/scikit-image#5427: residual multichannel->channel_axis fixes
scikit-image/scikit-image#5348: channel_as_last_axis decorator fix
We did much of this already, but need to review for consistency with upstream now that it has been implemented there. Also, can expand the test cases using parameterization over dtypes as was done upstream.
scikit-image/scikit-image#4880: Richardson-Lucy deconvolution: allow single-precision computation
scikit-image/scikit-image#5200: Add float32 support to moments_hu
scikit-image/scikit-image#5204: single precision support in skimage.registration
scikit-image/scikit-image#5219: single precision support in skimage.restoration
scikit-image/scikit-image#5220: single precision support in skimage.metrics
scikit-image/scikit-image#5344: single precision support in moments functions
scikit-image/scikit-image#5353: single precision support in skimage.features
scikit-image/scikit-image#5354: single precision support in skimage.filters
scikit-image/scikit-image#5372: improved single precision support in skimage.transform
scikit-image/scikit-image#5373: single precision support in skimage.segmentation
scikit-image/scikit-image#5443: support single precision in skimage.color
selem
argument has been renamed to footprint
throughout the library. The footprint
argument is now deprecated. (scikit-image/scikit-image#5445)in_place
in favor of the use of an explicit out
argument in skimage.morphology.remove_small_objects
, skimage.morphology.remove_small_holes
and skimage.segmentation.clear_border
input
argument of skimage.measure.label
has been renamed label_image
. The old name is deprecated.num_iter
for paramters describing the number of iterations and max_num_iter
for parameters specifying an iteration limit.skimage.measure.regionprops
have been updated so that properties are better grouped by the first word(s) of the name. The old names will continue to work for backwards compatibility.measure.label
, the deprecated neighbors
parameter has been removed (use connectivity
instead).skimage.color.rgb2grey
and skimage.color.grey2rgb
functions have been removed (use skimage.color.rgb2gray
and skimage.color.gray2rgb
instead).skimage.color.rgb2gray
no longer allows grayscale or RGBA inputs.alpha
parameter of skimage.color.gray2rgb
has now been removed. Use skimage.color.gray2rgba
for conversion to RGBA.order > 0
now raises a ValueError.anti-aliasing=True
will raise a ValueError.bg_label
parameter of skimage.color.label2rgb
is now 0.skimage.feature.register_translation
function has been removed (use skimage.registration.phase_cross_correlation
instead).skimage.feature.masked_register_translation
function has been removed (use skimage.registration.phase_cross_correlation
instead).mode
in skimage.filters.hessian
is now 'reflect'
.mode
in skimage.filters.sato
is now 'reflect'
.labels
argument renumbering in skimage.feature.peak_local_max
is avoided (scikit-image/scikit-image#5047).find_bounaries
with mode='subpixel' (scikit-image/scikit-image#5447)._label2rgb_avg
function.skimage.color.separate_stains
does not return negative values.skimage.filters.gaussian
.skimage.segmentation.random_walker
.skimage.measure.regionprops_table
) dtype bugfix.skimage.transform.rescale
when using a small scale factor.intensity_image
extra_properties in regionprops.skimage.metric.structural_similarity
when image is too small.skimage.segmentation.find_boundaries
.skimage.exposure.is_low_contrast
for boolean inputs.skimage.metric.structural_similarity
.skimage.filters.threshold_otsu
.skimage.feature.blob_dog
docstring example and normalization.skimage.exposure.adjust_gamma
.skimage.exposure.histogram
and skimage.measure.regionprops_table
. (scikit-image/scikit-image#5522)skimage.register.phase_cross_correlation
. (scikit-image/scikit-image#5461)Is cuda 10.x support feasible or planned?
I have tried install in Conda prompt:
pip install cucim
ERROR: Could not find a version that satisfies the requirement cucim (from versions: none)
ERROR: No matching distribution found for cucim
Is it available for OS Win or what I have made done wrong? Thank you for asist
As per the comment from @jakirkham on MR #100
Is your feature request related to a problem? Please describe.
We need to add Stain Normalization algorithm to a part of cuCIM's operation:
https://developer.nvidia.com/blog/accelerating-digital-pathology-pipelines-with-nvidia-clara-deploy
Describe the solution you'd like
Need to move the implementation to cuCIM's package: Project-MONAI/MONAI#2666
@drbeh helped push a draft PR
We would like to update and merge it.
Describe alternatives you've considered
N/A
Additional context
We also have C++ implementation.
Once Operator interface is available, we can provide the operation with C++ implementation.
Describe the bug
When running cucim - accelerated dataloading in Jupyter, a lot of pink warning style output can be generated that looks like an error.
Steps/Code to reproduce bug
Run the following code in a notebook with a suitably large image:
from cucim import CuImage
from dask.distributed import as_completed
from dask.distributed import Client, LocalCluster
import os
import numpy as np
cluster = LocalCluster(dashboard_address= 8789, processes=True)
client = Client(cluster)
# iterate over a set of regions from which to threshold
def process_chunk(params):
start_loc_list = params[0]
inp_file = params[1]
patch_size = params[2]
slide = CuImage(inp_file)
res = []
for start_loc in start_loc_list:
region = np.array(slide.read_region(start_loc, [patch_size, patch_size], 0))
if region.flatten().var() > 100:
res.append(start_loc)
return res
# As the results are processed, put them into a list
def compile_results(futures):
patches = []
for future in as_completed(futures):
res1 = future.result()
if res1:
for patch in res1:
patches.append(patch)
return patches
input_file = "patient_100_node_0.tif"
wsi = CuImage(input_file)
sizes=wsi.metadata["cucim"]["resolutions"]
w = sizes["level_dimensions"][0][0]
h = sizes["level_dimensions"][0][1]
patch_size = 256
num_processes = os.cpu_count()
# compute the coordinates of the image patches
start_loc_data = [(sx, sy)
for sy in range(0, h, patch_size)
for sx in range(0, w, patch_size)]
# calculate the number of patches per process/thread
chunk_size = len(start_loc_data) // num_processes
# create list of patches to process
start_loc_list_iter = [(start_loc_data[i:i+chunk_size],input_file,patch_size) for i in range(0, len(start_loc_data), chunk_size)]
# Threshold each patch asynchronously and return a future
future_result1 = list(client.map(process_chunk, start_loc_list_iter))
patches = compile_results(future_result1)`
Expected behavior
Ideally no such output should be produced unless the user specifies that they want info/warning output to be shown.
Environment details (please complete the following information):
Jupyter notebook running NGC PyTorch container 21.07
cucim v21.8.2
Additional context
Here is some example output when loading an image with 10 processes:
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
[Plugin: cucim.kit.cuslide] Loading...
[Plugin: cucim.kit.cuslide] Loading the dynamic library from: /opt/conda/lib/python3.8/site-packages/cucim/clara/[email protected]
[Plugin: cucim.kit.cuslide] loaded successfully. Version: 0
Initializing plugin: cucim.kit.cuslide (interfaces: [cucim::io::IImageFormat v0.1]) (impl: cucim.kit.cuslide)
Is your feature request related to a problem? Please describe.
cuCIM is currently supporting only jpeg/deflate-compressed RGB image (which is prevalent on Digital Pathology image) only and doesn't support multi-channel TIFF images yet.
Supporting multi-channel would help to load geospatial(remote sensing. multi-band) or microscopy (multi-channel) images.
Describe the solution you'd like
Supporting multi bands(channels) is feasible to implement in cuCIM as existing code can handle IFD(Image File Directory)s of TIFF format (each band image is stored in an IFD) and, AFAIK, GeoTIFF is a TIFF format with domain-specific metadata.
It would be a significant effort to implement the full feature of GeoTIFF format. However, we can start providing pixel array data and some important metadata to be used in DeepLearning use cases.
Describe alternatives you've considered
For parsing/recognizing metadata of the GeoTIFF image, we may want to look at GDAL's implementation that is also based on libtiff library, or libgeotiff for a reference?
Additional context
Background
Image Object Detection models predict bounding boxes (bboxes). A single model will usually predict more candidate bboxes than are necessary, therefore practitioners need techniques to combine the many predicted bboxes into fewer (and more accurate) predicted bboxes.
Additionally if we use 5-Fold models and/or multiple image object detection models, we need a way to ensemble (i.e. combine) the many bboxes from the many models.
Current State-Of-The-Art is CPU
Both of these tasks can be solved with NMS (Non-maximum Suppression) and/or WBF (Weighted Box Fusion). Most people use the GitHub repository here which has a CPU implementation of both
https://github.com/ZFTurbo/Weighted-Boxes-Fusion
PyTorch has a batched GPU version of NMS, but there are currently no GPU versions of WBF. The technique WBF is currently the most accurate state-of-the-art method for combining bboxes into fewer more accurate predictions . Research paper here
https://arxiv.org/abs/1910.13302
When using multiple 5-Fold models with thousands of images and many bbox per image, we can easily obtain millions of bboxes! Applying WBF on CPU can take up to 1 hour!
Describe the solution you'd like
It would be wonderful to implement a GPU version for WBF. And perhaps implement another GPU version of NMS too.
Is your feature request related to a problem? Please describe.
Aperio SVS format is the most popular Digital Pathology image format available in public and we see many open data sets.
For example:
Describe the solution you'd like
Additional context
Per comment from @jakirkham on MR #100
Need to check styles with clang-format in CI/CD
Ah ok. Maybe we can track C++ formatting cleanup as an issue
Originally posted by @jakirkham in #108 (comment)
GDS feature is disabled for now due to no public cuFile package available.
Once GDS package is released publicly(expected to release this week), need to update the code to use cufile.h header from the public package and/or update conda package to use the public gds package.
This issue is explaining the feature that is already being implemented and would like to use this issue for showing the feature and get feedback until the implementation is merged and available.
In many deep learning use cases, small image patches need to be extracted from the large image and they are fed into the neural network.
If the patch size doesn't align with the underlying tile layout of TIFF image (e.g., AI model such as ResNet may accept a particular size of the image [e.g., 224x224] that is smaller than the underlying tile size [256x256]), redundant image loadings for a tile are needed (See the following two figures)
Which resulted in lower performance for unaligned cases as shown in our GTC 2021 presentation
To improve image loading performance for general use cases, cuCIM needs a cache mechanism implemented.
The cache feature also need to be highly configurable (such as disabling cache) : #17 (comment)
Will we be able to disable the cache when doing random tile reads? We generally disable the OpenSlide one when training.
(As shown in Access Pattern 3 in https://github.com/rapidsai/cucim/blob/branch-0.20/notebooks/File-access_Experiments_on_TIFF.ipynb)
Other libraries have the following strategies for cache.
We would like to support three cache strategies (can be extended).
nocache
per_process
shared_memory
(interprocess)And we want to make user can select cache strategy and its configuration through a configuration file (.cucim.json
) or function calls(CuImage.cache()
method)
cuCIM has similar performance gain with aligned case when patch and tile layout are not aligned.
You can test the feature via the following command
pip install -i https://test.pypi.org/simple/ --pre --upgrade cucim
per_process
strategy performs better than shared_memory
strategy, and both strategies perform less than nocache
strategy when underlying tiles and patches are aligned.
shared_memory
strategy does some additional operations compared with per_process
strategy, and both strategies have some overhead using cache (such as memory allocation for cache item/indirect function calls)=> All three strategies (including nocache
) can have benefited if we allocate CPU/GPU memory for tiles from a fixed-sized cache memory pool (using RMM and/or PMR) instead of calling malloc() to allocate memory.
When patches to read in an image can be determined in advance (inference use case), we can load/prefetch entire compressed/decompressed image data to the memory and provide Python generator(iterator) to get a series of patches efficiently for inference use cases.
Describe the bug
While importing
"from cucim.skimage.transform import resize", I am getting error as
"AttributeError: module 'cupy' has no attribute 'byte'"
Steps/Code to reproduce bug
I am running the RAPIDS container: nvcr.io/nvidia/rapidsai/rapidsai:21.08-cuda11.2-runtime-ubuntu20.04
I have my:
cucim: 21.8.1 version
cupy: 9.0.0 version
CUDA Version: 11.4
Expected behavior
I want to test out the difference between skimage and cucim from this website:
https://www.youtube.com/watch?v=G46kOOM9xbQ
Environment details (please complete the following information):
Additional context
Just need to resolve this error.
RuntimeError Traceback (most recent call last)
/tmp/ipykernel_11392/2436574467.py in
7 level_count = resolutions["level_count"]
8
----> 9 region = img.read_region([0,0], level_dimensions[level_count - 1], level_count - 1, device="cuda")
10
11 print(region.device)
RuntimeError: This format (compression: 1, sample_per_pixel: 3, planar_config: 1, photometric: 2) is not supported yet!.
See comments in #64 (comment) .
DLPack is updated (v0.6.0) and it has breaking changes (in v0.4.0): https://github.com/dmlc/dlpack/releases
Need to update DLPack version along with rmm version.
Hi cuCIM team,
I am deploying RAPIDS with cuCIM v21.06 on ppc64le architecture (Summit cluster) and in our NV-ARM Developer Kit cluster, and noted during compilation, it grabs GDS. So my questions are:
Is GDS a hard dependency ?, if not, is there any env. variables to disable it ?
Thanks,
I would like to see GPU utilization for all benchmarks. This will allow me to know how much compute and memory was used for the benchmark. Currently there is no way to get this in the benchmark script. Eventually this could be used to create benchmarks designed to "max out" throughput available on the HW.
It would be nice to see additional metrics reported for each of the measured functions that written out of each benchmark script (example).
Is your feature request related to a problem? Please describe.
We need to look at Next-generation file formats (NGFF) (https://ngff.openmicroscopy.org/latest/) which use Zarr format for general microscopy images with distributed computing.
We want to support Zarr/NGFF after supporting major digital pathology formats (including .svs format).
Describe the solution you'd like
We can support Zarr or OME-Zarr format by 1) reusing existing library (such as ome-zarr-py or z5) or 2) implementing it from scratch.
Reusing a C++ library such as z5 may be a preferred option.
With GDS(GPUDirected-Storage, https://docs.nvidia.com/gpudirect-storage/overview-guide/index.html), the performance of loading the chunks(files) of the Zarr file(folder) could be greatly improved.
Other useful libraries that cuCIM can exploit may be available (we need to collect the information).
Additional context
Do you plan to support images with greater depth than 8 bits? More than 3 channels? (For multiple stains)
CuImage class holds DLPack's structures that can represent various data types, strides, shapes (including Channel dimension) so supporting them is possible. CuImage class already have a public API for that (dtype, dims, shape, and channel_names. see this)
We have a plan to also support microscopy-related image formats which usually store images with data types such as float and int16/32 and multi-channels (such as NGFF which is based on Zarr format), once we address the need for supporting Digital Pathology image formats.
scikit-image API functions often support more than 3 channels (with exceptions for things like rgb2gray where the input must be a 3 channel image).
Many operations involve internal conversion to floating-point. We made an effort to preserve single-precision computation when the input is single-precision and have started pushing those same modifications back upstream to scikit-image itself (which traditionally did most operations in double precision).
References
Articles
Data
Python Implementation
C/C++ Implementation
As per comment from @jakirkham in MR #100
Describe the bug
When calling color_jitter
on an individual 2D image, for instance a cupy array with dimension of (3, 2, 2)
and dtype of float32
. The output will be change the dtype and adds a batch dimension to it.
Function call:
OUTPUT = color_jitter(INPUT, brightness=1.0, contrast=1.0, saturation=1.0, hue=0.0)
INPUT:
type = cupy.ndarray
dtype = float32
shape = (3, 2, 2)
OUTPUT:
type = cupy.ndarray
dtype = unit8
shape = (1, 3, 2, 2)
As summarized by @isVoid in rapidsai/cudf#7098 , Python API documentation for RAPIDS projects has historically used the Sphinx Read The Docs theme and served a single large HTML page. This has caused the docs to load slowly as API surfaces have grown and caused users to experience potentially significant latency/delays when searching for functions.
In rapidsai/cudf#8746 , @galipremsagar updated the cuDF documentation to the more responsive PyData Sphinx theme. In rapidsai/cugraph#1793, @BradReesWork made a similar change.
Updating the cuCIM documentation to the PyData Sphinx theme should improve the user experience for our documentation.
Merge #100 to add transforms for Digital Pathology
Is your feature request related to a problem? Please describe.
Add unit/integration/system/performance testing for TIFF file loader module (under clara
).
See https://github.com/rapidsai/cucim/wiki/002_setup_tests.md for the details.
Is your feature request related to a problem? Please describe.
We have implemented Digital Pathology-related transformation operations with CuPy and we would like to make them part of cuCIM's built-in image-processing operations.
Describe the solution you'd like
First, need to discuss how to structure the package layout for non-scikit-image APIs.
Then, expose existing implementations as part of cuCIM package.
Describe alternatives you've considered
N/A
Additional context
We have been working on Digital Pathology-specific operations and need to make them available so that MONAI can make use of them.
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I wish I could use cuCIM to do [...]
It's be great if cucim implements skimage's hog algorithm. Hog feature is really useful and it'd be great if we had a GPU implementation of it.
Describe the solution you'd like
Implement skimage's Hog algorithm.
(https://scikit-image.org/docs/dev/api/skimage.feature.html?highlight=hog#skimage.feature.hog)
Describe alternatives you've considered
None
Additional context
None
Hi all,
I was just trying to setup cucim on a freshly installed Windows computer with mini-conda installed. I copied the command from the readme and I'm receiving this error during installation.
(bio1) C:\Users\rober>conda create -n cucim -c rapidsai -c conda-forge/label/cupy_rc -c conda-forge cucim cudatoolkit=11.2
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- cucim
Current channels:
- https://conda.anaconda.org/rapidsai/win-64
- https://conda.anaconda.org/rapidsai/noarch
- https://conda.anaconda.org/conda-forge/label/cupy_rc/win-64
- https://conda.anaconda.org/conda-forge/label/cupy_rc/noarch
- https://conda.anaconda.org/conda-forge/win-64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/win-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/win-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/msys2/win-64
- https://repo.anaconda.com/pkgs/msys2/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Am I missing a conda channel or something?
Any hint is appreciated!
Thanks,
Robert
Is your feature request related to a problem? Please describe.
PyTorch's DataLoader class is used in many DeepLearning training applications to load training data and pre-process the data, before feeding to AI model.
Since PyTorch's DataLoader is running in multi-processes, it is hard to use cuCIM's scikit-image APIs (which makes use of CUDA) in the pre-transformations of the DataLoader due to CUDA context issues.
It would be nice to provide a way/example to use cuCIM with DeepLearning Frameworks such as PyTorch.
Describe the solution you'd like
PyTorch's DataLoader works like this. It would be nice if we have a PyTorch's DataLoader-like utility class in Dask that mimics Pytorch's DataLoader behavior but implemented with Dask (dask-cuda) for the parallelization of data loading (so providing a generator/iterator that gives a batch of processed image data).
Describe alternatives you've considered
Additional context
Relevant information regarding CuPy+PyTorch.
With Numba to get cuda context.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.