Comments (1)
This should be addressed in the latest release (0.19.0).
I grabbed some random userdata1.parquet
and tried this:
# gpq describe userdata1.parquet
╭───────────────────┬────────┬────────────┬────────────┬──────────────╮
│ COLUMN │ TYPE │ ANNOTATION │ REPETITION │ COMPRESSION │
├───────────────────┼────────┼────────────┼────────────┼──────────────┤
│ registration_dttm │ int96 │ │ 0..1 │ uncompressed │
│ id │ int32 │ │ 0..1 │ uncompressed │
│ first_name │ binary │ string │ 0..1 │ uncompressed │
│ last_name │ binary │ string │ 0..1 │ uncompressed │
│ email │ binary │ string │ 0..1 │ uncompressed │
│ gender │ binary │ string │ 0..1 │ uncompressed │
│ ip_address │ binary │ string │ 0..1 │ uncompressed │
│ cc │ binary │ string │ 0..1 │ uncompressed │
│ country │ binary │ string │ 0..1 │ uncompressed │
│ birthdate │ binary │ string │ 0..1 │ uncompressed │
│ salary │ double │ │ 0..1 │ uncompressed │
│ title │ binary │ string │ 0..1 │ uncompressed │
│ comments │ binary │ string │ 0..1 │ uncompressed │
├───────────────────┼────────┴────────────┴────────────┴──────────────┤
│ Rows │ 1000 │
│ Row Groups │ 1 │
╰───────────────────┴─────────────────────────────────────────────────╯
⚠️ Not a valid GeoParquet file (missing the "geo" metadata key). Run convert to try to convert it to GeoParquet.
So then I tried to convert
it:
# gpq convert userdata1.parquet maybe-geo.parquet
gpq: error: expected a geometry column named "geometry", use the --input-primary-column to supply a different primary geometry
And then followed the suggestion to try --input-primary-column
:
# gpq convert userdata1.parquet maybe-geo.parquet --input-primary-column first_name
gpq: error: wkt: unsupported geometry
All that is expected (the first_name
is not WKT or WKB). As described in #87 (comment), this unfortunately would have worked with a non-string binary column (trusting that the data was WKB). But then validate
would fail.
A --strict
option could be added that either applied validation while writing or validated after writing in the convert
command. But that would be kind of involved.
from gpq.
Related Issues (20)
- Feature suggestion: extract command HOT 2
- gpq convert output of Overture parquet files cannot be read by GDAL HOT 9
- describe on remote source.coop files not working HOT 3
- Question: Is there is a plan to expose functionality as library code? HOT 4
- feature request: associate srs to geoparquet
- Not able to convert a geojson file HOT 2
- About compression: is it normal for it to be so low? HOT 1
- gpq: not able to view the geoparquet output in QGIS 3.28 HOT 1
- Simple way to add metadata to geoparquet file
- Allow uploading .geoparquet in Web Interface
- Support Overture parquet conversion to GeoParquet HOT 5
- Support control over number of row groups as an option HOT 1
- Support for convert to stdout HOT 7
- Issue converting a complicated GeoParquet file HOT 1
- Report info on row groups in describe? HOT 2
- Error when doing describe / validate with non-geo parquet file HOT 2
- Better warnings / info in `describe` on non-compliant GeoParquet HOT 5
- describe and validate remote geoparquet files HOT 1
- Support for reading from blob storage
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gpq.