Comments (7)
These are great ideas and I totally agree that the ideal situation would be one in which users can provide format = tar_raster
or format = tar_vector
and geotargets
takes care of figuring out if the data is coming from terra
, sf
, stars
, etc. and has some defaults for how targets are stored. I think the tricky part, which I don't quite understand if you have a plan for, is how those targets should be read back in and "unmarshalled". How can we know if the current target assumes an upstream target is a terra
SpatVector
or a sf
MULTIPOLYGON? I would think geotargets
would have to be as specific as providing tar_<vector/raster>_<package name>
(e.g. tar_vector_terra, tar_vector_sf) as formats.
from geotargets.
excellent descript @brownag
note there is also {gdalraster} now with already support for SOZip creation and file management, and nascent 'gdalvector' support:
https://usdaforestservice.github.io/gdalraster/reference/addFilesInZip.html
https://usdaforestservice.github.io/gdalraster/articles/gdalvector-draft.html
gdalraster has become very richly featured very quickly, and could be the GDAL API that's otherwise entirely missing from R atm.
from geotargets.
Regarding Item 2 (/vsizip/), it should be possible to drop the .zip extension from file and target name. I was not aware of the alternate syntax!
From https://gdal.org/user/virtual_file_systems.html#vsizip-zip-archives:
Starting with GDAL 2.2, an alternate syntax is available so as to enable chaining and not being dependent on .zip extension, e.g.: /vsizip/{/path/to/the/archive}/path/inside/the/zip/file. Note that /path/to/the/archive may also itself use this alternate syntax.
from geotargets.
I think the tricky part, which I don't quite understand if you have a plan for, is how those targets should be read back in and "unmarshalled". How can we know if the current target assumes an upstream target is a terra SpatVector or a sf MULTIPOLYGON? I would think geotargets would have to be as specific as providing tar_<vector/raster>_ (e.g. tar_vector_terra, tar_vector_sf) as formats.
I lean towards this conclusion also-- so that the choice is explicit and not changeable through magical options or settings. How these operations would be handled on the backend could/should be more generic, but for reproducibility and clarity it is probably best to give users options that require explicit choices. This might mean many combinations of thin wrapper methods around the core functions, but I don't think that is inherently bad as long as there is some overall order to how they are named.
excellent descript @brownag
note there is also {gdalraster} now with already support for SOZip creation and file management, and nascent 'gdalvector' support:
https://usdaforestservice.github.io/gdalraster/reference/addFilesInZip.html
https://usdaforestservice.github.io/gdalraster/articles/gdalvector-draft.html
gdalraster has become very richly featured very quickly, and could be the GDAL API that's otherwise entirely missing from R atm.
Sweet! I have seen {gdalraster} and watched some of the (rapid) progress on that with interest... but I don't think I was aware of the plans to provide bindings for the OGR vector API! I have used your vapour package for some of my generic/vector GDAL needs that go beyond terra/sf
Something truly generic, mirroring the GDAL API/"close to the GDAL metal" would allow for all sorts of capabilities and customization--perhaps {gdalraster} would be a good choice for an imported package doing the core backend work for plumbing to the various user-facing types/formats.
from geotargets.
Thanks for this @brownag !
Following on from @Aariq 's #7 - I quite like the idea of tar_{pkg}_<filetype>
convention. I think that a lot of/most users would know what package they are reading/creating things with, so it should hopefully facilitate discovery in that way?
I'm still learning about a lot of spatial things, so there is a bit of this I don't quite understand, but I think that this issue could be split out into multiple components, eventually, as there are a few threads in here.
Overall my preference for syntax would be something like:
tar_terra_raster(
new_raster
raster_creation_function(args)
)
But overall this would work the same as:
tar_target(
new_raster
raster_creation_function(args),
format = "format_terra_raster"
)
Or something?
with tar_{pkg}_<filetype>
we can expose a filetype
argument to give users control, e.g.,:
tar_terra_shapefile(
my_shape,
create_shapefile(args),
filetype = "parquet"
)
But then I wonder if
tar_parquet_shapefile(
my_shape,
create_shapefile(args)
)
Would be better?
Naming things is hard. But I think that it is worthwhile thinking about the API design - once we have ideas on how we want the user to interact with the package my experience is that it is usually easier to write the code.
from geotargets.
Naming things is hard. But I think that it is worthwhile thinking about the API design - once we have ideas on how we want the user to interact with the package my experience is that it is usually easier to write the code.
I agree we should try to think through this and make some decisions before getting too far. Super helpful discussion here.
from geotargets.
I think everything in here is either in the package already, a PR, or a separate issue. Thanks for the contributions @brownag!
from geotargets.
Related Issues (20)
- tests with multiple workers HOT 6
- Combining list of SpatRasters with rast fails with multiple workers HOT 7
- consider how `tar_terra_sprc` might work in dynamic and static branching HOT 1
- `targets` 1.7.0 seems to have broken `geotargets` HOT 5
- `tar_terra_rast()` doesn't save units or varnames HOT 4
- create `tar_terra_sds()` HOT 1
- split_semicolon() function HOT 1
- Multiple calls to `geotargets_option_set()` nulls out existing options HOT 2
- User input `resources` not being passed from `tar_terra_*()` to `tar_target_raw()`
- Release geotargets 0.1.0
- Best way to use SpatRaster tiles? HOT 3
- Require minimum version of GDAL? HOT 5
- create CITATION.cff
- Supporting various split-apply-combine spatial workflows HOT 2
- Functionality for objects *containing* `terra` objects? HOT 5
- Add `description` argument to all `tar_` functions to be passed to `tar_target_raw()`
- Create `tar_terra_svc()` for SpatVectorCollection objects
- only allow `iteration = "list"` for terra targets HOT 3
- Explore timing/speed issues when using tiles that match block size vs our own spec HOT 2
- Some thoughts on specifying tile size HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from geotargets.