Comments (3)
Expected behaviors should be agreed upon before we change our code to handle exceptions. This is a summary of how I think the program throws an exception during certain actions. Warnings should notify the user that the action couldn't be completed, such as an image not being able to be used for various reasons, then continue running with the rest of the data. Failures should notify the user and terminate the program.
images_to_samples.py
Risky Behavior | Expected Response | Function | Notes |
---|---|---|---|
Load config.yaml | Fail | main() | |
Load S3 bucket | Fail | main() | |
Get prep.csv from bucket | Fail | main() | |
Get prep.csv from local | Fail | main() | |
Get images from bucket | Warn | main() | |
Get images from local | Warn | main() | |
Create temp h5py files | Fail | main() | |
Get shp file from bucket | Fail | main() | |
Get shp file from local | Fail | main() | |
Assert band number | Warn | main() | |
Create shp raster image | Warn | main() | |
Validate num classes | Warn | main() | |
Read images and mask | Warn | main() | |
Create shp raster image | Warn | main() | |
Create masked raster | Warn | main() | |
Create h5py files | Fail | main() | |
Delete temp label tif | Warn | main() | |
Delete temp hdf5 files | Warn | main() | |
Add samples to the bucket | Fail | main() | |
Add labels to bucket | Warn | main() | |
Delete files downloaded from S3 bucket | Warn | main() |
train_model.py
Risky Behavior | Expected Response | Function | Notes |
---|---|---|---|
Load config.yaml | Fail | main() | |
Load S3 bucket | Fail | main() | |
Load samples from bucket | Fail | main() | |
Create dataloaders | Fail | main() | |
Save intermediate checkpoint | Fail | main() | |
Save final Checkpoint | Fail | main() | |
Write model to bucket | Fail | main() | |
Check trn sample count | Fail | verify_sample_count() | |
Check val sample count | Fail | verify_sample_count() | |
Metrics logger | Warn | train() | |
Metrics logger | Warn | validation() |
image_classification.py
Risky Behavior | Expected Response | Function | Notes |
---|---|---|---|
Load config.yaml | Fail | main() | |
Load S3 bucket | Fail | main() | |
Load model from bucket | Fail | main() | |
Load images to classify from bucket | Warn | main() | |
Assert Band number | Warn | main() | |
Read image as array | Warn | classification() | |
Create new raster from base | Warn | classification() |
from geo-deep-learning.
After discussion with ymoisan, we should set a limit to the number of warnings that we give. Once the threshold has been exceeded, the program should throw an exception and terminate. We would do this in case of systematic user error, for example trying to load images only in pdf format or having a different band number or number of classes in the config.yaml compared to the images in use.
from geo-deep-learning.
Relevant exceptions are being implemented ad hoc.
from geo-deep-learning.
Related Issues (20)
- Create Github issue template
- Resolve naming convention for duplicate image filename
- Add configurable parent directory for patches
- BUG: GDL cannot write TIFFs larger than 4 GB
- BUG: broken CI pipeline HOT 1
- FEATURE: low contrast check for images
- BUG: broken docker image potentially due to memory leak
- BUG: ValueError: can't extend empty axis 0 using modes other than 'constant' or 'empty' HOT 1
- FEATURE: read only valid portion of imagery HOT 1
- BUG: AOIs out-of-memory error
- BUG: Docker image creation fails due to missing "libarchive" HOT 1
- FEATURE: add HRNet + OCR model architecture
- "[BUG]: aoi.raster called before assignment in tilling_segmentation.py, on error logging."
- DOCS: add documentation for verify script HOT 1
- BUG: Version Issue HOT 3
- FEATURE: refactor SegmentationDataset class
- Implement a base class for scripting models
- BUG: importing torch raises ImportError
- BUG: scale to minmax range
- Feature: geo-inference integration
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geo-deep-learning.