Comments (11)
Thank you for your feedback. Yes, the dinov2_vit14
should be changed to dinov2_vitl14
.
Just now, I re-ran with the latest MMSegmentation (mmseg==1.2.2
, mmcv==2.1.0
, mmengine==0.10.2
). After installation, I follow the four steps listed here to add the DINOv2 backbone to the mmsegmentation working directory.
Then, I can successfully reproduce our results when using the command line below (the mIoU in the image is 84.8% due to single-scale test):
python3 tools/test.py configs/depth_anything/depth_anything_large_mask2former_16xb1_80k_cityscapes_896x896.py cityscapes_vitl_mIoU_86.4.pth
I did not encounter any problem during reproducing. Can you tell me more about the errors you encountered?
from depth-anything.
Yes, there is no need to install mmseg as a subfolder of Depth Anything. Just copy the necessary files mentioned in the four steps into your mmseg project. We have updated the files under semseg
. Please use the latest files. Thank you.
from depth-anything.
Please try:
pip3 install "mmdet>=3.0.0rc4"
And then re-run the test command.
from depth-anything.
Hi, sorry for the confusion. We provide the torchhub
directory here. Please put it under your working space.
from depth-anything.
Thank you for you help.
Unfortunately, it does not run with mmsegmentation. I get all the examples running that come with mmsegmentation. So this is not the issue. But when I try depth anything segmentations, it is not working. It starts to crash again.
This line is also wrong:
self.dinov2 = torch.hub.load('torchhub/facebookresearch_dinov2_main', 'dinov2_vit14', source='local', pretrained=False)
There is no dinov2_vit14. It must be dinov2_vitl14 I think. However, it does not integrate properly with mmsegmentation.
Did you use any specific version of mmsegmentation? Is the Readme some essential information missing? I am a bit puzzled why this is not working.
from depth-anything.
Thank you for your answer!
I guess mmsegmentation can be in its own conda environment? It does not need to be a subfolder of depth anything I assume.
So basically installing an isolated mmsegmentation project and then following the 4 steps?
I will retry it again later tonight. I really appreciate your help. Thank you!
from depth-anything.
So this is what I get:
Traceback (most recent call last):
File "tools/test.py", line 123, in
main()
File "tools/test.py", line 116, in main
runner = Runner.from_cfg(cfg)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/runner/runner.py", line 462, in from_cfg
runner = cls(
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/runner/runner.py", line 429, in init
self.model = self.build_model(model)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/runner/runner.py", line 836, in build_model
model = MODELS.build(model)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "/home/paja/projects/mmsegmentation/mmseg/models/segmentors/encoder_decoder.py", line 92, in init
self._init_decode_head(decode_head)
File "/home/paja/projects/mmsegmentation/mmseg/models/segmentors/encoder_decoder.py", line 102, in _init_decode_head
self.decode_head = MODELS.build(decode_head)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/registry.py", line 570, in build
return self.build_func(cfg, *args, **kwargs, registry=self)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 232, in build_model_from_cfg
return build_from_cfg(cfg, registry, default_args)
File "/home/paja/micromamba/envs/openmmlab/lib/python3.8/site-packages/mmengine/registry/build_functions.py", line 121, in build_from_cfg
obj = obj_cls(**args) # type: ignore
File "/home/paja/projects/mmsegmentation/mmseg/models/decode_heads/mask2former_head.py", line 42, in init
super().init(**kwargs)
TypeError: init() got an unexpected keyword argument 'in_channels'
I am running the following command:
python3 tools/test.py configs/depth_anything/depth_anything_large_mask2former_16xb1_80k_cityscapes_896x896.py cityscapes_vitl_mIoU_86.4.pth
In this repo (depth anything) I am on HEAD (e911178)
In the mmsegmentation repo I am on HEAD (c685fe6767c4cadf6b051983ca6208f1b9d1ccb8)
The versions are:
mmcv==2.1.0, mmengine=0.10.2 (was before on 0.10.3 but has same error)
Obviously mask2former_head.py get some variables it does not like. So that's why I think there is some incompatibility and I am scratching my head how to get this running (without diving fully into the code base to understand all details)
Thank a lot for your help!
from depth-anything.
Thank you. That did the trick! It runs. Maybe some additional info might be helpful for others to get this running as well.
from depth-anything.
Thank you for your feedback! I will update the semseg README.
from depth-anything.
@LiheYoung thank you very much for your response to this issue it was very helpful!
I just wanted to add that there seems to be some issue with mmsegmentation using Pytorch 2, running the test script. I tried both 2.1 and 2.2 with Cudatoolkit 11.8 and ended up installing Pytorch 1.13 with Cudatoolkit 11.7, which worked for me. (My GPU is ampere arch with driver 535...)
Hope this helps someone... :)
from depth-anything.
在训练seg的时候,有用到提到的一些loss吗
from depth-anything.
Related Issues (20)
- Feature Request: Mask Support for Depth Estimation
- Whether the sky area is involved in the loss calculation? HOT 3
- Metric Depth Estimation model training HOT 1
- Train a custom unlabeled dataset on Metric depth estimation
- What are the units of depth values output by the model? HOT 1
- Training metric depth problem
- TypeError: "__init__()"miss 1 requirement positional argument:"config" HOT 2
- What is the difference between Depth-Anything and control_v11f1p_sd15_depth_fp16.safetensors?
- Want more detaile about training time and GPUs! HOT 1
- How to "not" resize image when inferencing metric depth? HOT 1
- Custom dataset training HOT 3
- How to calculate distance of the depth image ?
- How to ensure that the range of depth information obtained by the two different methods is consistent?
- mmseg::model registry error HOT 1
- train HOT 1
- How can calculate the actual depth value from the metric depth output of Depth Anything V2 ? What steps and conversions are necessary to obtain a precise and accurate depth calculation from the raw metric depth data(Depth Anything V2 output)?
- Infer relative depth
- Lfeat 损失没找到 HOT 1
- Feature alignment loss HOT 1
- Getting issue with (indoor)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from depth-anything.