Copying a sources schema, crs and river with **ds.meta does not work about fiona

[prologic]

Turns out doing schema=ds.schema.copy() doesn't work on some Shapefiles either. I processes two Shapefiles. One preserved the properties and schema while the other did not.

[prologic]

Here are links to the two troublesome Shapefile(s) in question:

• http://shortcircuit.net.au/~prologic/tmp/data/regions/

[sgillies]

What Fiona version are you using? I'm unable to reproduce the problem (using the head of the Fiona repo, 1.2.18 to-be):

>>> with fiona.open('/', vfs='zip:///Users/seang/Downloads/STE11aAust.zip', layer=u'STE11aAust') as c:
...     with fiona.open('/tmp/foo.shp', 'w', **c.meta) as d:
...         d.write(next(c))
...
>>>

The output:

krusty-2:~ seang$ ogrinfo -so /tmp/foo.shp foo
INFO: Open of `/tmp/foo.shp'
      using driver `ESRI Shapefile' successful.

Layer name: foo
Geometry: 3D Polygon
Feature Count: 1
Extent: (140.999475, -37.505060) - (159.109219, -28.157020)
Layer SRS WKT:
GEOGCS["GRS 1980(IUGG, 1980)",
    DATUM["unknown",
        SPHEROID["GRS80",6378137,298.257222101]],
    PRIMEM["Greenwich",0],
    UNIT["Degree",0.017453292519943295]]
STATE_CODE: String (1.0)
STATE_NAME: String (50.0)

Would appreciate if you could provide the smallest possible script that produces the problem.

[prologic]

So doing the same thing with the other Shapefile IBRA7.zip:

>>> import fiona
>>> with fiona.open("/", vfs="zip:///Users/s2092651/ccaih/ccav/data/.sources/regions/IBRA7.zip", layer=u"IBRA7_regions") as c:
...     with fiona.open("/tmp/foo.shp", "w", **c.meta) as d:
...             d.write(next(c))
... 
>>> 

And the output of:

$ ogrinfo -so /tmp/foo.shp foo
INFO: Open of `/tmp/foo.shp'
      using driver `ESRI Shapefile' successful.

Layer name: foo
Geometry: Polygon
Feature Count: 1
Extent: (132.612732, -14.365490) - (136.984990, -10.903420)
Layer SRS WKT:
GEOGCS["GRS 1980(IUGG, 1980)",
    DATUM["unknown",
        SPHEROID["GRS80",6378137,298.257222101]],
    PRIMEM["Greenwich",0],
    UNIT["Degree",0.017453292519943295]]
FID: Real (11.0)

[sgillies]

Sorry, once again I can't reproduce the problem (GDAL 1.9.2 here). What's your GDAL version?

>>> with fiona.open('/', vfs='zip:///Users/seang/Downloads/IBRA7.zip', layer=u'IBRA7_regions') as c:
...     with fiona.open('/tmp/foo.shp', 'w', **c.meta) as d:
...         d.write(next(c))
...
>>>

Results:

(toblerity27)krusty-2:shapely seang$ dumpgj -d /tmp/foo.shp
{"crs": {"ellps": "GRS80", "no_defs": true, "proj": "longlat"}, "driver": "ESRI Shapefile", "name": "/tmp/foo.shp", "schema": {"geometry": "Polygon", "properties": {"AREA": "float:19.8", "FEAT_ID": "str:50", "HECTARES": "float:19.8", "OBJECTID": "int:10", "REC_ID": "int:10", "REG_CODE_6": "str:9", "REG_CODE_7": "str:6", "REG_NAME_6": "str:50", "REG_NAME_7": "str:100", "REG_NO_61": "float:19.8", "SHAPE_AREA": "float:19.11", "SHAPE_LEN": "float:19.11", "SQ_KM": "float:19.8"}}}

(toblerity27)krusty-2:shapely seang$ ogrinfo -so /tmp/foo.shp foo
INFO: Open of `/tmp/foo.shp'
      using driver `ESRI Shapefile' successful.

Layer name: foo
Geometry: 3D Polygon
Feature Count: 1
Extent: (132.612732, -14.365490) - (136.984990, -10.903420)
Layer SRS WKT:
GEOGCS["GRS 1980(IUGG, 1980)",
    DATUM["unknown",
        SPHEROID["GRS80",6378137,298.257222101]],
    PRIMEM["Greenwich",0],
    UNIT["Degree",0.017453292519943295]]
SHAPE_AREA: Real (19.11)
REG_NAME_7: String (100.0)
REG_NAME_6: String (50.0)
OBJECTID: Integer (10.0)
AREA: Real (19.8)
FEAT_ID: String (50.0)
REC_ID: Integer (10.0)
REG_NO_61: Real (19.8)
SHAPE_LEN: Real (19.11)
HECTARES: Real (19.8)
SQ_KM: Real (19.8)
REG_CODE_7: String (6.0)
REG_CODE_6: String (9.0)

[prologic]

That's really odd. I have GDAL 1.10.0 here.
Let me do some more digging on my end... In the meantime I've used the
ogr2ogr tool with the -simplify option.

cheers
James

James Mills / prologic

E: [email protected]
W: prologic.shortcircuit.net.au

On Tue, Jul 23, 2013 at 12:38 PM, Sean Gillies [email protected]:

Sorry, once again I can't reproduce the problem (GDAL 1.9.2 here). What's
your GDAL version?

with fiona.open('/', vfs='zip:///Users/seang/Downloads/IBRA7.zip', layer=u'IBRA7_regions') as c:
... with fiona.open('/tmp/foo.shp', 'w', **c.meta) as d:
... d.write(next(c))
...

Results:

(toblerity27)krusty-2:shapely seang$ dumpgj -d /tmp/foo.shp
{"crs": {"ellps": "GRS80", "no_defs": true, "proj": "longlat"}, "driver": "ESRI Shapefile", "name": "/tmp/foo.shp", "schema": {"geometry": "Polygon", "properties": {"AREA": "float:19.8", "FEAT_ID": "str:50", "HECTARES": "float:19.8", "OBJECTID": "int:10", "REC_ID": "int:10", "REG_CODE_6": "str:9", "REG_CODE_7": "str:6", "REG_NAME_6": "str:50", "REG_NAME_7": "str:100", "REG_NO_61": "float:19.8", "SHAPE_AREA": "float:19.11", "SHAPE_LEN": "float:19.11", "SQ_KM": "float:19.8"}}}

(toblerity27)krusty-2:shapely seang$ ogrinfo -so /tmp/foo.shp foo
INFO: Open of /tmp/foo.shp' using driverESRI Shapefile' successful.

Layer name: foo
Geometry: 3D Polygon
Feature Count: 1
Extent: (132.612732, -14.365490) - (136.984990, -10.903420)
Layer SRS WKT:
GEOGCS["GRS 1980(IUGG, 1980)",
DATUM["unknown",
SPHEROID["GRS80",6378137,298.257222101]],
PRIMEM["Greenwich",0],
UNIT["Degree",0.017453292519943295]]
SHAPE_AREA: Real (19.11)
REG_NAME_7: String (100.0)
REG_NAME_6: String (50.0)
OBJECTID: Integer (10.0)
AREA: Real (19.8)
FEAT_ID: String (50.0)
REC_ID: Integer (10.0)
REG_NO_61: Real (19.8)
SHAPE_LEN: Real (19.11)
HECTARES: Real (19.8)
SQ_KM: Real (19.8)
REG_CODE_7: String (6.0)
REG_CODE_6: String (9.0)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/53#issuecomment-21389881
.

[sgillies]

The thought of GDAL 1.10 breaking Fiona is a bit terrifying. I'll have to look into this ASAP.

[prologic]

If you like I can find some time today or tomorrow morning to downgrade and
re-test.
I actually did not realize I was using GDAL 1.10.

cheers
James

James Mills / prologic

E: [email protected]
W: prologic.shortcircuit.net.au

On Wed, Jul 24, 2013 at 1:40 PM, Sean Gillies [email protected]:

The thought of GDAL 1.10 breaking Fiona is a bit terrifying. I'll have to
look into this ASAP.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/53#issuecomment-21461783
.

[sgillies]

I can't reproduce your problem with GDAL 1.10 either. I'm sorry, but will close this.

[prologic]

No problems. Thanks for trying!

James Mills / prologic

E: [email protected]
W: prologic.shortcircuit.net.au

On Tue, Jul 30, 2013 at 2:51 PM, Sean Gillies [email protected]:

I can't reproduce your problem with GDAL 1.10 either. I'm sorry, but will
close this.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/53#issuecomment-21769571
.