Comments (8)
I am willing to help with that. How to get started?
from netcdf.
There hasn't been any requests for this, but we an always make some API for reading a variable along a dimension. I think this needs some ideas for how best to create the iterator, and what would be the most useful for the end user. Maybe you have some insight?
An idea is to allow chunking along a dimension (e.g. time) and let each next
read everything in that chunk with that dimension increasing. Although with the Extents
type this might not be necessary?
from netcdf.
Having a lazy loader both in space and time is IMO mandatory. In alot of my own usecases the files are larger than RAM, while i only need very local in the spacetime sense values. Julias NCdatasets already does this (so I know where to "steal" ideas). I want to migrate to rust so I will try to implement a lazy loader for myself anyways.
from netcdf.
@krestomantsi and I been looking at this issue, how do I build the netcdf package locally? Can you also clarify the type annotation written in this issue?
from netcdf.
It should be as simple as cloning this repository and running cargo test
. If you specify cargo build --features static
you don't need to install netcdf-c
. The type annotation can be disregarded as the API is not yet known. There is some room for a creative thinker here!
from netcdf.
warning: some crates are on edition 2021 which defaults to `resolver = "2"`, but virtual workspaces default to `resolver = "1"`
note: to keep the current resolver, specify `workspace.resolver = "1"` in the workspace root's manifest
note: to use the edition 2021 resolver, specify `workspace.resolver = "2"` in the workspace root's manifest
Compiling hdf5-sys v0.8.1
Compiling netcdf-src v0.3.2 (/home/josh/netcdf/netcdf-src)
error: failed to run custom build command for `netcdf-src v0.3.2 (/home/josh/netcdf/netcdf-src)`
Caused by:
process didn't exit successfully: `/home/josh/netcdf/target/debug/build/netcdf-src-cf974f2b78ad12dc/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-changed=build.rs
CMAKE_TOOLCHAIN_FILE_x86_64-unknown-linux-gnu = None
CMAKE_TOOLCHAIN_FILE_x86_64_unknown_linux_gnu = None
HOST_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_x86_64-unknown-linux-gnu = None
CMAKE_GENERATOR_x86_64_unknown_linux_gnu = None
HOST_CMAKE_GENERATOR = None
CMAKE_GENERATOR = None
running: "cc" "--version"
exit status: 0
running: "c++" "--version"
exit status: 0
running: "cc" "--version"
exit status: 0
CMAKE_PREFIX_PATH_x86_64-unknown-linux-gnu = None
CMAKE_PREFIX_PATH_x86_64_unknown_linux_gnu = None
HOST_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_x86_64-unknown-linux-gnu = None
CMAKE_x86_64_unknown_linux_gnu = None
HOST_CMAKE = None
CMAKE = None
running: cd "/home/josh/netcdf/target/debug/build/netcdf-src-d6c4cf1c0f0b9757/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/home/josh/netcdf/netcdf-src/source" "-DBUILD_SHARED_LIBS=OFF" "-DNC_FIND_SHARED_LIBS=OFF" "-DBUILD_UTILITIES=OFF" "-DENABLE_EXAMPLES=OFF" "-DENABLE_DAP_REMOTE_TESTS=OFF" "-DENABLE_TESTS=OFF" "-DENABLE_EXTREME_NUMBERS=OFF" "-DENABLE_PARALLEL_TESTS=OFF" "-DENABLE_FILTER_TESTING=OFF" "-DENABLE_BASH_SCRIPT_TESTING=OFF" "-DENABLE_PLUGINS=OFF" "-DPLUGIN_INSTALL_DIR=OFF" "-DHDF5_VERSION=1.10.7" "-DHDF5_C_LIBRARY=hdf5_debug" "-DHDF5_HL_LIBRARY=hdf5_hl_debug" "-DHDF5_INCLUDE_DIR=/home/josh/netcdf/target/debug/build/hdf5-src-38fdb0c7c7ff9024/out/include" "-DENABLE_NCZARR=OFF" "-DENABLE_DAP=OFF" "-DENABLE_BYTERANGE=OFF" "-DENABLE_DAP_REMOTE_TESTS=OFF" "-DZLIB_ROOT=/home/josh/netcdf/target/debug/build/libz-sys-2ca0cd9f87eb13cc/out/include/.." "-DCMAKE_INSTALL_PREFIX=/home/josh/netcdf/target/debug/build/netcdf-src-d6c4cf1c0f0b9757/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/etc/profiles/per-user/josh/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/etc/profiles/per-user/josh/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_ASM_COMPILER=/etc/profiles/per-user/josh/bin/cc" "-DCMAKE_BUILD_TYPE=RelWithDebInfo"
--- stderr
CMake Warning:
Ignoring extra path from command line:
"/home/josh/netcdf/netcdf-src/source"
CMake Error: The source directory "/home/josh/netcdf/netcdf-src/source" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
thread 'main' panicked at '
command did not execute successfully, got: exit status: 1
build script failed, must exit now', /home/josh/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...```
I get this error
from netcdf.
Regarding the API, would you basically want it to be like a slicing function so that you say from lat X => Y and long X => Y and return the matrix that contains these values as a view? Or is it supposed to throw all the values into a matrix of the correct shape
from netcdf.
You will also need to pull the submodule by git submodule update --init --recursive
from netcdf.
Related Issues (20)
- Enable shuffle filter HOT 5
- Change selectors for put/get
- How to read variables with string values? HOT 3
- Typing error when compiling on ARM64 Docker container HOT 4
- Unable to link with `netcdf` library on macOS HOT 2
- Wrong values fetched HOT 6
- scale_factor & offset_factor HOT 9
- Cannot open era5-lan nc file. HOT 7
- Can't compile in Tauri on macOS (missing HDF5) HOT 2
- Regarding thread safety HOT 2
- Slow disk IO HOT 10
- Flushing file HOT 6
- Does not compile in release mode HOT 6
- Reading OpenDAP HOT 14
- Update to ndarray 0.14.0 HOT 2
- netcdf features should be enabled if available HOT 1
- Update netcdf-src to latest version of netcdf-c
- Add derive macros for more advanced types
- CF Time attribute HOT 13
- thread 'main' panicked at 'Unable to locate HDF5 root directory and/or headers.' HOT 10
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from netcdf.