Comments (10)
Following the excellent lightning architecture, something to keep in mind is to avoid hiding pytorch details from the user, this is the single most loved feature from lightning
That being said, I do think we can do something similar from what you're suggesting, and it would be great if it worked both for FasterRCNN
and MaskRCNN
Currently the creating of the backbone model happens inside the constructor:
def __init__(self, ...):
self.m = fasterrcnn_resnet50_fpn(pretrained=self.pretrained, **kwargs)
What do you think about injecting this via an argument? Something like:
def __init__(self, backbone=None):
self.m = backbone or fasterrcnn_resnet50_fpn(pretrained=self.pretrained)
I like this way because we avoid the problem of unsunsupported Backbone
and we don't hide pytorch from the user
from icevision.
We also need to be careful with model_splits
, this is used for differential learning rates, independent of the method we decide to go with this, we somehow have to make sure model_splits
is behaving as expected
from icevision.
Slight issue in the backbone definition. To create the backbone we need to pass it to faster RCNN class of Pytorch then it gets created similar to ResNet 50 rpn.
Also when we add the backbone the major problem is ResNet 50 rpn by default given is trained on COCO while backbones when we add are trained on ImageNet
from icevision.
To create the backbone we need to pass it to faster RCNN class
Yeah, that's okay, we can easily refactor the code to work like that
About the pre-trained weights, we might in some cases even want to start from scratch, so it's fine to start from ImageNet, as long as is clear to the user what is happening.
What is important here, is that I think that if the user wants to change the backbone we want to make sure he has to write pytorch code, no magic should happen inside our classes
What we should do is write helper function/classes so that he has to write less boilerplate, that's all
from icevision.
That sounds good. It would make it easier.
from icevision.
Feel free to open a PR implementing these changes, I'll give you freedom to design the API as you like, just follow the principles discussed here 🎉
Ow, we don't have a contributing guide yet, but we take inspiration from lightning contribution guide you can find here
For code formatting, use black, all defaults, simple
That's it, happy coding 😉
from icevision.
Can you provide a high level example (with code just like I did before) on how that would look like?
from icevision.
I'm thinking a bit on backbones. To create a seperate folder for backbones and keep inheriting those models. Few reasons why.
- Torchvision lacks backbones such as efficient net, inception resnet etc. This would again limit us (later refactoringis very hard)
- Will give more granular control and force the user to edit the backbone Pytorch models in our folder not go to torchvision or outside library.
Heavily drawn from this https://www.github.com/qfgaohao/pytorch-ssd/tree/master/vision
We could have more backbones. It will allow users to extend it easily (edited)
2:36
Many cases we need layers that aren't in torch. How to handle it? With this we can have seperate layers folder. Only these 2 would be in torch. Remaining all in Lightning
On a high level view we would have structure of folders like this
mantishrimp
-> backbones (containing backbones like mobilenet etc)
-> layers (layers that aren't in torch e.g. dilated convolution, this is used by by both backbones and models)
-> models (which contain folder for each e.g. ssd, fasterrcnn, detr)
This will allow the user to specify the backbone and subsequent flexible model is built.
def __init__(self, backbone=None):
self.m = backbone or fasterrcnn_resnet50_fpn(pretrained=self.pretrained)
will change a bit to
def __init__(self, user_backbone=None):
ft_backbone = user_backbone.features
ft_bakbone.out_channels = (as in model)
self.m = FasterRCNN(backbone = ft_backbone)
This FasterRCNN or so models should come from folder of models.
So, it will simplify as follows, backbones, layers, and models are in Pytorch (no other alternative) and then our trainer, engines, and user access will be with Pytorch Ligthning
It will allow us to create a seperate file like I did, model.py for initializing model with user defined backbones and hyperparameters, which would be passed to engine.py which is controlled by pytorch ligthning.
End user would need lesser pain in manually defining.
from icevision.
here instead of taking self.feature extractor from torchvision it would come from our custom backbones in model folder, they would be written with Pytorch lightining
Extend this to object detection.
from icevision.
Closing this for now, as it would be more clear in #61
from icevision.
Related Issues (20)
- Tutorial about using Lightning and Fastai callbacks for model checkpoint by metric HOT 1
- Icevision no longer works with the latest `sahi` release HOT 5
- Failing to create TorchScript module using torch.jit.script()
- batch inference for semantic segmentation does not work HOT 1
- Error learn.export( pkl_export_path )
- Error: cannot import name 'intersect_dicts' from 'yolov5.utils.torch_utils when using icevision==0.12.0 with yolov5==6.2.3 HOT 1
- 'mmseg' module missing from 'icevision.models' HOT 1
- cant find module create_coco_eval
- Error when loading icevision in colab HOT 2
- Fix unit tests failing on CI and/or local env
- Support for pytorch 1.13
- Support for EncodedRLE in COCO Dataset HOT 1
- Adding a maskrcnn_resnet50_fpn_v2 from TorchVision models
- name 'process_bbox_predictions' is not defined in inference.ipynb
- 'RecordCollection' object has no attribute 'load' HOT 1
- Call to predict_from_dl failing when calling convert_raw_prediction HOT 1
- Semantic Segmentation Issue
- Unable to install icevision in google colab HOT 4
- AttributeError: module 'numpy' has no attribute 'int'.
- Not able to plot graphs and sample images HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from icevision.