Comments (9)
EO profile of STAC includes items such as sun azimuth and elevation : https://github.com/radiantearth/stac-spec/blob/master/extensions/eo/schema.json. Type 20170831_162740_ssc1d1
in your browser search bar and you'll en up here :
All we need is there...
I suggest we investigate creating STAC Items of the label extension type. Note : models per se are not STAC Items for now. I think there is an opportunity for us to think about how we could make that happen.
from geo-deep-learning.
Mandatory information to store with the model, for re-usability:
- Weights (.pth)
- Model definition (e.g. Unet model)
- Task type (e.g. classification or semantic segmentation)
- Number of classes and surely their definition (e.g. 1-Vegetation, 2- Lake, 3- Building, etc.)
- Number of band used for training and their definition (e.g. 4 bands: R-G-B-PIR);
- The definition should describe the source of each band:
- Sensor type (e.g. Satellite, LiDAR, aerial photos, radar, etc.)
- Acquisition date
- Wavelength (if applicable)
- Preprocess (if applicable)
- The definition should describe the source of each band:
- Spatial resolution to which the training was conducted
- Geographic location where the training/validation and tests were conducted. (e.g. bounding box or footprint, maybe?)
Optional information to store:
- Training and validation accuracy
- Training parameters (e.g. learning rate, # of epoch, class weights, etc.)
from geo-deep-learning.
A nice way of validating if inputs are applicable to a given model implemented as a decorator : see "input validation" in A comprehensive guide to putting a machine learning model in production using Flask, Docker, and Kubernetes.
from geo-deep-learning.
If we wanted to devise some kind of standard for model interoperability around HDF5, we would likely come up with a HDF5 product definition. Interesting excerpts from [HDF Product Designer](https://wiki.earthdata.nasa.gov/display/HPD/HDF+Product+Designer ++):
The Hierarchical Data Format (HDF5) provides a flexible container that supports groups and datasets, each of which can have attributes. In many ways, HDF5 is similar to a directory structure in a file and, like directory structures, the same data can be structured and annotated in many ways. This flexibility empowers HDF5 users to arrange data in ways that make sense to them. However, it can make it difficult to share data ...
Many communities have successfully addressed this problem by creating conventional structures and annotations for data in HDF5. This approach depends on data files (e.g., products) that carefully follow these conventions.
A HDF5 product is the content that should exist in a single HDF5 file.
This content is defined by the HDF5 objects (groups, attributes, datasets), their names, the hierarchies they create (links and references), and attribute values. Dataset values are typically not stored in such files (unless they qualify as metadata) thus this software cannot be used as a data server. Once completed, a HDF5 product is replicated in many files (commonly on the order of tens of thousands or more) and filled with real data.
How would the use of HDF5 help us in forming totally independent DL containers that would contain all the information needed for interoperability ? Could we implement something in relation to "standardised environments" as per OGC Testbed 14 ?
from geo-deep-learning.
How well does HDF5 play with Big Data infrastructures and OGC services like WCS ? Could the H5Server be useful ?
from geo-deep-learning.
Could we integrate STAC fields ?
from geo-deep-learning.
deepdish ? torch hdf5 ?
from geo-deep-learning.
@mpelchat04 is it something that we still want to do?
from geo-deep-learning.
Work is ongoing to develop a STAC extension applied to models. The GDL team will check on this as the extension is developed. We will close the issue for now.
from geo-deep-learning.
Related Issues (20)
- get_key_def(): unnecessary logging polutes stderr
- add PR template HOT 2
- Create Github issue template
- Resolve naming convention for duplicate image filename
- Add configurable parent directory for patches
- BUG: GDL cannot write TIFFs larger than 4 GB
- BUG: broken CI pipeline HOT 1
- FEATURE: low contrast check for images
- BUG: broken docker image potentially due to memory leak
- BUG: ValueError: can't extend empty axis 0 using modes other than 'constant' or 'empty'
- FEATURE: read only valid portion of imagery HOT 1
- BUG: AOIs out-of-memory error
- BUG: Docker image creation fails due to missing "libarchive" HOT 1
- FEATURE: add HRNet + OCR model architecture
- "[BUG]: aoi.raster called before assignment in tilling_segmentation.py, on error logging."
- DOCS: add documentation for verify script HOT 1
- BUG: Version Issue HOT 3
- FEATURE: refactor SegmentationDataset class
- Implement a base class for scripting models
- BUG: importing torch raises ImportError
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geo-deep-learning.