Comments (7)
I understand now, thank you for your patience!
from diffpose.
Hi @pimatysiak, the two datasets we evaluate in this work don't have ground truth 2D landmarks per se. Instead, the authors of these datasets annotated landmarks in each 3D CT and projected these into 2D using the ground truth extrinsic/intrinsic matrices. In the DeepFluoro dataset, ground truth poses were determined with an offline global registration processes. In the Ljubljana dataset, ground truth poses were determined by registering implanted fiducials - unfortunately these 2D/3D landmarks are not available in the dataset.
This was something that confused me too when I started working with these datasets. This issue may help clarify: rg2/DeepFluoroLabeling-IPCAI2020#4
from diffpose.
Hi Vivek, thanks for the quick reply!
Specifically for the DeepFluoro dataset I know the landmarks were computed automatically, but the paper specifies that they were manually inspected and verified, which is why I considered them to be "ground truth", and they are included in the dataset.
I guess if you do not use them for the validation and instead project landmarks on the provided fluoro images from the 3D landmarks, it's fine, I guess that was just a misunderstanding on my part.
Would you think it possible to still plug in those 2D ground truth landmarks into the registration framework, so I can rule out bad automated landmarks if the results I get are unsatisfactory? Could you point out which section of the code I should modify for this? I notice you have a get_2d_fiducials method but I don't see it used.. unless I'm blind.
from diffpose.
get_2d_fiducials
is used for visualization purposes - it's been called whenever you see landmarks in figures in the paper.
The 2D landmarks aren't only used for figures, however. They're also used in the validation function that computes registration error (see, the Evaluator
class). I could have unified their implementation to use the same helper function ... something to simplify in future implementation!
from diffpose.
Ah I see, that's why I didn't find it in the validation. Thanks!
Forgive me for being annoying, but could you specify where you load the 2D landmarks? I noticed in your Evaluator class that you indeed load what you call fiducials from the specimen, but it looks to me that the ones that are loaded are only the 3D landmarks from the patient, not the individual 2D landmarks from each fluoro image used in the evaluation.. Unless I am once again mistaken?
from diffpose.
I don't load the 2D landmarks from the DeepFluoro dataset. Since they're just the perspective projection of the fiducials, I just compute them myself. For sanity, here's a check that the two are equal:
import torch
from diffpose.calibration import perspective_projection
from diffpose.deepfluoro import DeepFluoroDataset, convert_diffdrr_to_deepfluoro
idx = 1 # Specimen ID
jdx = 0 # X-ray ID
specimen = DeepFluoroDataset(idx)
# From DiffPose
_, pose = specimen[jdx]
pose = convert_diffdrr_to_deepfluoro(specimen, pose)
perspective_projection(pose, specimen.intrinsic, specimen.fiducials)
# tensor([[[1599.6984, 639.4305],
# [ 29.8002, 625.9924],
# [1409.0881, 1118.6995],
# [ 236.0820, 1138.9253],
# [1314.6082, 673.6483],
# [ 386.6854, 685.3052],
# [1156.6296, 1393.9117],
# [ 563.0671, 1408.9928],
# [ 858.9445, 1423.1156],
# [ 805.6584, 1422.8655],
# [ 985.3099, 1336.1173],
# [ 688.2995, 1325.3735],
# [ 845.8329, 1303.7023],
# [ 796.3127, 1305.0377]]])
# From DeepFluoro
torch.tensor([x[:] for x in specimen.projections[f"{jdx:03d}/gt-landmarks"].values()]).squeeze()
# tensor([[1599.6986, 639.4305],
# [ 29.8006, 625.9923],
# [1409.0884, 1118.6995],
# [ 236.0822, 1138.9249],
# [1314.6084, 673.6480],
# [ 386.6857, 685.3053],
# [1156.6298, 1393.9119],
# [ 563.0674, 1408.9928],
# [ 858.9448, 1423.1151],
# [ 805.6586, 1422.8654],
# [ 985.3102, 1336.1173],
# [ 688.2997, 1325.3733],
# [ 845.8331, 1303.7023],
# [ 796.3130, 1305.0375]])
They're equal up to floating point error, which probably contributes ~10^-3 mm of registration error.
from diffpose.
no problem at all, happy to answer any questions. hope it's helpful for your work!
from diffpose.
Related Issues (20)
- How to draw fiducials in 2D image? HOT 6
- NaN problems during training HOT 13
- DiffPose is not compatiable with the DiffDRR 0.3.10 or later HOT 3
- Issue in register.py regarding unexpected keyword argument in convert function HOT 2
- Issue with register.py file and Inquiry Regarding Code Adjustment HOT 2
- Dynamic display of registration process HOT 4
- AutoExecutor.map_array (main, id_numbers) code is not running. HOT 3
- How to learn the pose regression Model? HOT 8
- Retry `pytorch3d` now with improved numerical stability
- How to build a stable Conda environment for running Diffpose HOT 4
- How to train a model using real preoperative CT scans and intraoperative X-ray images taken with a C-arm machine? HOT 4
- About train.py - parameterization
- How to make the animation gif file? HOT 2
- Does camera preconditioning improve gradient descent runtimes?
- Remove `pytorch3d` dependency
- Questions about the optimal initial point for registration HOT 4
- Refactor `RigidTransform`
- Using Diffpose for with a simple API for custom training and inference HOT 2
- Code issue HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from diffpose.