Comments (34)
Okay thanks! Hold on for a day, I'll prepare a high level api design doc or an issue here and then we can proceed. I don't want you to waste any effort if we end up not including that.
from yoloexplorer.
@SkalskiP @AyushExel Sorry guys, I have busy schedule tomorrow. I won't be able to join. But @SkalskiP please update me with the summary if possible. Thanks.
from yoloexplorer.
@SkalskiP Yeah, I can definitely take a look.
from yoloexplorer.
@AyushExel and @hardikdava I can help out as well
from yoloexplorer.
Didn't get a lot of time today but here's the immediate near-future roadmap - #38
I'll add more and also the API design. Feel free to add more in suggestions in the comments there.
from yoloexplorer.
@hardikdava will do!
from yoloexplorer.
Done @SkalskiP
from yoloexplorer.
@SkalskiP @hardikdava Yes ofcourse! I've been busy with a few projects here and there, but most likely I'll be back on this one starting next week. I plan to add some features to be able to load datasets/tables directly from s3 and gcs
from yoloexplorer.
@AyushExel Thanks but I got it solved.
from yoloexplorer.
@SkalskiP Yes, I'm planning to add all data formats. A little issue of bandwidth, but I think I'll make some progress over the weekend. The plan is to cover formats like this yolo->coco->cls/img folders-> remote datasets RF, DVC etc. depending on how much time I can dedicate.
from yoloexplorer.
The other thing is to actually improve the UI. I think I've pushed streamlit to the limit and cracks are starting to show, but I also don't want to start a frontendv2 from scratch using react :)
And finally create an embeddings predictor from trochvision models so I can make ultralytics dependency on main branch in order to make a proper pypi release. Right now I can't do that as the twine won't allow me to build a package that depends on a branch
from yoloexplorer.
Awesome @AyushExel! π₯ Have you considered using supervision
for loading datasets? We offer YOLO, COCO, and PASCAL VOC dataset loaders. You should be able to add functionality in one afternoon. I hope! π
Take a look at the docs if you think it could be helpful: https://roboflow.github.io/supervision/dataset/core/
from yoloexplorer.
Ohh I didn't know supervision loads coco too. Yeah then I think it's a no brainer to use that. The GUI stuff is almost ready. I'll take a look at supervison tonight
from yoloexplorer.
Awesome! Let me know how it goes! It would be so awesome to see supervision power up that feature.
from yoloexplorer.
@SkalskiP how about training? Because the while idea is to simply allow users to get a trainable command for the new dataset they create within minutes. But ultralytics currently doesn't support training via coco format. So does supervision efficiently coverts the dataset for yolo format?
I think there is another advantage to have an intermediate step where coco/voc are converted to yolo internally(hidden from the user) because yolo format supports txt files which can combine images from various folders, which I think is not possible for coco? (I might be wrong, haven't used coco format a lot, but all imgs do need to be under one folder right?)
from yoloexplorer.
@AyushExel
With Supervision you can:
- Covert between formats. So you can start from COCO/VOC and convert into YOLO if you wish.
- Marge datasets. If user will be able to define multiple directors with YOLO or COCO format annotations / images and you can marge that into single one if you want.
- You can also split data if you want. Assuming someone gives you images and annotations for training and you want to get train and test set.
from yoloexplorer.
@AyushExel, we are also adding functionality to evaluate models such as mAP
and ConfusionMatrix
. It would be needed to compare the accuracy. Supervison
is supporting these kind of benchmarking for all supporting dataset. There might be possibility to add intermediate detection result conversion.
Let me know if I can contribute to it anyhow.
from yoloexplorer.
Okay got it @SkalskiP
@hardikdava Mmmm comparing accuracy is more of a modeling problem. I was thinking of just handing off the dataset to the user with the command to train, but not do any modeling in the app. That is to reduce the scope of this app. But I'm not sure if that's the correct approach. What do you think?
from yoloexplorer.
Got it @AyushExel . You can add supervison.DetectionDataset
api to load into user prefer format and convert it to yolo
format, make the intermediate steps and at the end train the model. Loading and converting will be the 2 lines command with supervision. If you want, I can prepare a draft PR for you if you want.
from yoloexplorer.
Would you like us to take a look at it?
from yoloexplorer.
Sure
from yoloexplorer.
Okey. It looks like supervision
could support your whole data-loading pipeline. We have:
- Detection: YOLO, COCO, and PASCAL
- Segmentation: YOLO and COCO (PASCAL coming with this release)
- Classification: Directory Structure
What would you like us to contribute first?
from yoloexplorer.
There's some parts of the codebase that aren't neat. Would you like to get on call and scope the supervision support? I can show you around and then we can get started
from yoloexplorer.
There's some parts of the codebase that aren't neat.
Your codebase or ours?
And yes! Sure! Letβs meet and talk it through.
from yoloexplorer.
Mine ofcourse.
Can you suggest a time tomorrow and provide an email
from yoloexplorer.
I'm in GMT+2. We could talk between 1:30 PM - 4:00 PM my time tomorrow. My email is [email protected].
from yoloexplorer.
@AyushExel, can we make it 30 minutes earlier?
from yoloexplorer.
@AyushExel, awesome! See you!
from yoloexplorer.
Hey guys! Any updates on this?
from yoloexplorer.
@AyushExel sorry! I was busy with latest supervision release. But Iβm back tomorrow. I can start tackling it.
from yoloexplorer.
I met with @AyushExel last week, and we agreed on the initial plan. The plan will have several phases.
- Rewrite the dataset loading logic found in the Explorer class. Currently, it uses a lot of code from the Ultralytics package. We could replace it with
sv.DetectionDataset.from_yolo
. - Over time, we will be able to expand the range of supported models. Add COCO and PASCAL VOC.
@AyushExel is this idea still alive?
from yoloexplorer.
@hardikdava will you have time to take a look at it?
from yoloexplorer.
@hardikdava awesome! Apologies for slowing down the process. I can't wait to see supervision
powering that functionality.
from yoloexplorer.
@hardikdava I see a notification from you about some error github mobile but I'm not able to find it on here for some reason. can you please point me to it again? Thanks!
from yoloexplorer.
Related Issues (11)
- Contribution HOT 2
- Create Dashboard HOT 7
- Use recordbatch iterator when creating tables
- Add top level function to find duplicate imgs HOT 2
- Add embedding visualization support HOT 1
- Make image storage optional to speed up table creation and improve memory footprint
- Support batch inference HOT 4
- Support multiple datasets
- Roadmap, Scope and API design
- How to use own dataset? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yoloexplorer.