osgeo / gdal

GDAL is an open source MIT licensed translator library for raster and vector geospatial data formats.

GDAL - Geospatial Data Abstraction Library

GDAL is an open source MIT licensed translator library for raster and vector geospatial data formats.

• Main site: https://gdal.org - Developer and user docs, links to other resources
• GIT repository: https://github.com/OSGeo/gdal
• Bug tracker: https://github.com/OSGeo/gdal/issues
• Download: https://download.osgeo.org/gdal
• Wiki: https://trac.osgeo.org/gdal - Various user and developer contributed documentation and hints
• Mailing list: https://lists.osgeo.org/mailman/listinfo/gdal-dev

The GDAL project uses a custom governance and is fiscally sponsored by NumFOCUS. Consider making a tax-deductible donation to help the project pay for developer time, professional services, travel, workshops, and a variety of other needs.

NumFOCUS is 501(c)(3) non-profit charity in the United States; as such, donations to NumFOCUS are tax-deductible as allowed by law. As with any donation, you should consult with your personal tax adviser or the IRS about your particular tax situation.

How to build

See BUILDING.md

How to contribute

See CONTRIBUTING.md

Docker images

See docker/

Code of Conduct

See doc/source/community/code_of_conduct.rst

Security policy

See SECURITY.md

Citing GDAL/OGR in publications

See CITATION and CITATION.cff

gdal's Issues

Better progress messages in gdalwarp

Better progress messages in standard output of gdalwarp

I am often using gdalwarp to merge several hundred single images into a big Geotiff images. This process often takes several hours. To get a better idea on the progress of the command it would be nice to get better output on the progress.

Currently, the output looks like this (skipping all the options (not relevant to this topic)):

gdalwarp -of GTiff *.asc merged.tif 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done. 0...10...20...30...40...50...60...70...80...90...100 - done.

It would be nice if gdalwarp could at the beginning of the process issue a statement how many input files it is going to process, e.g. processsing xx input files. Then for each line it could output something like:
processing filename (nr x of xxx files): 0...10...20...30...40...50...60...70...80...90...100 - done.
e.g.
processing '2673_1229_dom.asc' (nr 5 of 320 files): 0...10...20...30...40...50...60...70...80...90...100 - done.

This way one could have a much better overview how many files are already processed and how many are still pending.

Thank you for having a look at this suggestion!

OGR2OGR TAB to Shapefile conversion issue

Operating system

Windows 7 Professional Service Pack 1

GDAL version and provenance

GDAL 2.2.4, released 2018/03/19 OSGeo4w

Reading NITF with image compression and single block

Hello,
I'm running gdalinfo for a NITF file.
The file contains an Image with a JP2K compression and is saved as only one block.
From the specification MIL-STD-2500C the image subheader fields

• NPPBH
• NPPBV

must be set to "0000".
In the file gdal/frmts/nitf/nitfimage.c line 573 ff. is this issue handled. But from the IF Statement in line 573
if (EQUAL(psImage->szIC, "NC"))
this works only for uncompressed Images.

Please check this issue and remove the line 573.

Best regards
Markus Overesch

ESRI shp file cannot open by gdal

Expected behavior and actual behavior.

I expected to be able to open an ESRI shape file and it returns a None type instead.

Steps to reproduce the problem.

the code is simple, which is attached below:
import osgeo.gdal
import ogr
ogr.RegisterAll()
dataset = ogr.Open('river.shp',0)
print type(dataset)

Operating system

MacOS 10.13.4

GDAL version and provenance

1.10.1 installed in Anaconda 5.0.1 by 'conda install gdal'

VFK driver refuses reading records with duplicated ID

Expected behavior and actual behavior.

Currently VFK driver skips records with duplicated ID since this column is used by the driver as primary key. It's not correct for all layers, eg. PARG see example below.

Steps to reproduce the problem.

ogrinfo ~/geodata/vfk/Export_vse.vfk --config OGR_VFK_DB_OVERWRITE YES
Warning 1: In ExecuteSQL(INSERT INTO 'PARG' VALUES(2851026306,1,'29.10.2009 12:15:30',NULL,1,3286780306,3246536306,NULL,'PKN',693936,NULL,2,1283,NULL,NULL,NULL,6780,0,7,NULL,NULL,12624,NULL,NULL,674652306,NULL,NULL,'n','n',NULL,'n',4)): UNIQUE constraint failed: PARG.ID
...
Warning 1: PARG: 11 duplicated VFK data records skipped
...

GDALCreateOverviewDataset missing since gdal-2.2

We're using warping in GDAL (via GDALWarpOperation) extensively in our vts-mapproxy server. Since this operation doesn't natively support overviews which are crucial for our server we had to write code similar to gdalwarp utility to choose and use proper overview. Sometimes, we also have to force coarser overview when memory requirements would be too high (this happens when warping near date line or poles where GDAL has problems with wrapped dataset).

It was kinda easy in gdal-2.1 because function GDALCreateOverviewDataset was exported from gdal.so. With the advent of new LTS Ubuntu (Bionic Beaver, 18.04) the gdal-2.2 will become the default gdal version. Unfortunately, this version doesn't export GDALCreateOverviewDataset function. The function was un-exported from the dynamic library in this particular commit gdal_priv.h:1576.

I've been looking around and the function is indirectly available through GDALWarp utility function. that encapsulates whole warp functionality. So far so good. However, this function expects its options from argv-style cmdline options, which means that our program whould have to serialize options into string only to let GDALWarp to decode them instead of simply filling them into the GDALWarpOptions struct.

There's a possibility to pass OVERVIEW_LEVEL option to the GDALOpenEx call. However, at the time of warp operation we have got an already open dataset. Re-opening of the dataset counts on the knowledge of the original dataset path and opens a full-fledged dataset.

Would it be too much to ask you to make GDALCreateOverviewDataset function available in the installed library again? What is the intended programatic use of overview functionality?

WCS: unusual metadata in SUBDATASETS domain and missing SUBDATASET_x_NAME

@ajolma

gdalinfo WCS:https://beta-karttakuva.maanmittauslaitos.fi/wcs/service/ows?coverage=ortotesti__ortokuva
reports

Subdatasets:
  SUBDATASET_1_NAME=WCS:https://beta-karttakuva.maanmittauslaitos.fi/wcs/service/ows?service=WCS&version=2.0.1&coverageId=ortotesti__ortokuva
  SUBDATASET_1_WGS84BBOX=15.5706177104992,59.5209776951808,33.1881061199753,70.1359269604917
  SUBDATASET_1_BBOX(minX,minY,maxX,maxY)=61254.6580619824,6623749.72946716,735433.573091962,7782000,http://www.opengis.net/def/crs/EPSG/0/EPSG:3067
  SUBDATASET_1_TYPE=RectifiedGridCoverage
  DOMAIN=E,N
  DIMENSION_0_AXIS=E
  DIMENSION_0_UOM=m
  DIMENSION_0_INTERVAL=61254.6580619824,735433.573091962
  DIMENSION_1_AXIS=N
  DIMENSION_1_UOM=m
  DIMENSION_1_INTERVAL=6623749.72946716,7782000
  DIMENSION_2_AXIS=time
  DIMENSION_2_UOM=s
  FIELD_1_NAME=GRAY_INDEX
  FIELD_1_DESCR=GRAY_INDEX
  FIELD_1_INTERVAL=0 255

First issue is the lack of SUBDATASET_1_DESC metadata item
Second issue is the presence of all the other metadata items except SUBDATASET_1_NAME. They should likely be in the default metadata domain or in a dedicated one, but SUBDATASETS metadata domain should only contains SUBDATASET_x_DESC and SUBDATASET_x_NAME

Some GRIB2 files are not handled properly

Expected behavior and actual behavior.

It is not possible to code negative longitudes in GRIB2 files (i.e. 360° is added to the negative values). It is possible to have GRIB2 files with first grid point longitude greater than the last grid point longitude (even with data ordered by increasing longitude), which seems to break GDAL computations. For instance, gdallocationinfo triggers the "Location is off this file!" error when asking the value of a valid point.

I fixed the problem by adding two lines in frmts/grib/gribdataset.cpp to ensure longitudes are in the [-180,180] interval before starting any computation.

Steps to reproduce the problem.

GRIB2.zip
Unzip the file and execute : gdallocationinfo -wgs84 GRIB2.grb 0 45

Operating system

Ubuntu 17.10 64 bit

GDAL version and provenance

GDAL 2.2.1 (official ubuntu package) and even GDAL 2.2.4 (manual compilation)

Syntax errors in gdaltest.py

Pylint reported the following errors in gdaltest.py:

autotest/pymod/gdaltest.py:550: [E1305(too-many-format-args), GDALTest.testOpen] Too many arguments for format string
autotest/pymod/gdaltest.py:574: [E1305(too-many-format-args), GDALTest.testOpen] Too many arguments for format string

The sources lines are:

https://github.com/OSGeo/gdal/blob/master/autotest/pymod/gdaltest.py#L550
https://github.com/OSGeo/gdal/blob/master/autotest/pymod/gdaltest.py#L574

Not quite sure what was really intended.

DXF: large processing time due to spline interpolation

@atlight https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=7965 timeouts due to a spline with 1579 control points being processed

#7  0x00007ffff6ab7184 in GDALLinearSystemSolve (nDim=1579, nRHS=3, adfA=0x7fffcbaba010, adfRHS=0x805160, adfOut=0x80e570) at gdallinearsystem.cpp:84
#8  0x00007ffff6e52733 in GetBSplineControlPoints (adfParameters=..., adfKnots=..., aoDataPoints=..., nDegree=3, oStartTangent=..., oEndTangent=...) at ogrdxf_leader.cpp:1325
#9  0x00007ffff6e4d142 in InterpolateSpline (poLine=0x7ca630, oEndTangentDirection=...) at ogrdxf_leader.cpp:1435
#10 0x00007ffff6e4b7f5 in OGRDXFLayer::TranslateLEADER (this=0x7cb300) at ogrdxf_leader.cpp:253

In 766090b#diff-38f73a6f576242358161ae31080fd5fc , I already added a protection to avoid memory issues. Can we lower this default value again ? What are typical values for the number of control points ?

Clarify whether GetNoDataValue has scale/offset applied

The documentation for GDALRasterBand::GetNoDataValue does not say whether the returned value has to be applied to the raw data returned by ReadBlock/RasterIO or whether it already has the scale/offset applied. I assume the former, but a clarification would be great.

GeoTIFF: COGifying .msk sidecars substantially increases their size

Starting with https://map.openaerialmap.org/#/25.28981541199998,53.89605287200003,14/latest/5ad7a75e91b5310010e0d54f?_k=aaveo9 (direct links: https://oin-hotosm.s3.amazonaws.com/5ad7a59991b5310010e0d54b/1/5ad7a59991b5310010e0d54d.tif, https://oin-hotosm.s3.amazonaws.com/5ad7a59991b5310010e0d54b/1/5ad7a59991b5310010e0d54d.tif.msk) (which is already a COG, but that's irrelevant), creating a new COG (including a .msk sidecar, which needs to be transcoded separately) results in a mask that is substantially larger than the non-COG mask.

Adding overviews to the mask seems necessary in order to apply the mask when using GeoTIFF overviews. However, adding overviews breaks the COG structure and requires the mask to be explicitly converted to a COG. Somehow this final gdal_translate substantially increases the size of the mask, presumably by dropping optimizations like NBITS=1 (I couldn't figure out what's different or how to recreate the initial mask output; context here: hotosm/OpenAerialMap#102 (comment))

Processing Steps

# download source image
curl -sLO https://oin-hotosm.s3.amazonaws.com/5ad7a59991b5310010e0d54b/1/5ad7a59991b5310010e0d54d.tif
curl -sLO https://oin-hotosm.s3.amazonaws.com/5ad7a59991b5310010e0d54b/1/5ad7a59991b5310010e0d54d.tif.msk

# convert to target profile (produces test.tif.msk sidecar)
gdal_translate \
  -q \
  -b 1 -b 2 -b 3 \
  -mask mask \
  -co TILED=yes \
  -co BLOCKXSIZE=512 \
  -co BLOCKYSIZE=512 \
  -co NUM_THREADS=ALL_CPUS \
  -co COMPRESS=JPEG -co PHOTOMETRIC=YCbCr \
   5ad7a59991b5310010e0d54d.tif test.tif

# add overviews
gdaladdo \
  -r lanczos \
  --config GDAL_TIFF_OVR_BLOCKSIZE 512 \
  --config TILED_OVERVIEW yes \
  --config BLOCKXSIZE_OVERVIEW 512 \
  --config BLOCKYSIZE_OVERVIEW 512 \
  --config NUM_THREADS_OVERVIEW ALL_CPUS \
  --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCbCr \
  test.tif \
  2 4 8 16 32 64 128

# convert to COG (skip the mask)
gdal_translate \
  -b 1 -b 2 -b 3 \
  -co TILED=yes \
  -co BLOCKXSIZE=512 \
  -co BLOCKYSIZE=512 \
  -co NUM_THREADS=ALL_CPUS \
  -co COMPRESS=JPEG -co PHOTOMETRIC=YCbCr \
  --config COMPRESS_OVERVIEW JPEG --config PHOTOMETRIC_OVERVIEW YCbCr \
  -co COPY_SRC_OVERVIEWS=YES \
  --config GDAL_TIFF_OVR_BLOCKSIZE 512 \
  test.tif cog.tif

# add overviews to the mask
gdaladdo \
  --config GDAL_TIFF_OVR_BLOCKSIZE 512 \
  --config TILED_OVERVIEW yes \
  --config COMPRESS_OVERVIEW DEFLATE \
  --config BLOCKXSIZE_OVERVIEW 512 \
  --config BLOCKYSIZE_OVERVIEW 512 \
  --config SPARSE_OK_OVERVIEW yes \
  --config NUM_THREADS_OVERVIEW ALL_CPUS \
  test.tif.msk \
  2 4 8 16 32 64 128

# convert the mask to a COG
gdal_translate \
  -of GTiff \
  -co TILED=yes \
  -co BLOCKXSIZE=512 \
  -co BLOCKYSIZE=512 \
  -co NUM_THREADS=ALL_CPUS \
  -co COPY_SRC_OVERVIEWS=YES \
  -co COMPRESS=DEFLATE \
  -co PREDICTOR=2 \
  --config GDAL_TIFF_OVR_BLOCKSIZE 512 \
  test.tif.msk cog.tif.msk

Output

gdal_translate mask: http://mojodna-temp.s3.amazonaws.com/test.tif.msk - 1.7M
http://mojodna-temp.s3.amazonaws.com/cog.tif.msk - 3.8M

COG validation errors for test.tif.msk

The offset of the first block of overview of index 5 should be after the one of the overview of index 6
The offset of the first block of overview of index 4 should be after the one of the overview of index 5
The offset of the first block of overview of index 3 should be after the one of the overview of index 4
The offset of the first block of overview of index 2 should be after the one of the overview of index 3
The offset of the first block of overview of index 1 should be after the one of the overview of index 2
The offset of the first block of overview of index 0 should be after the one of the overview of index 1
The offset of the first block of the main resolution image should be after the one of the overview of index 6

GDAL version and provenance

2.3.0beta1 using this Docker image: quay.io/mojodna/gdal:v2.3.0beta1 (docker run -it --rm quay.io/mojodna/gdal:v2.3.0beta1)

I originally noticed this w/ 2.2.x.

Reading NITF with GDALINFO and problem in RPC content / JSON output

Hello,
I'm trying to read a NITF file that contains RPC Information. I call
gdalinfo -json -mm -hist -nogcp -nomd -norat -noct -nofl "filepath"

Because there are some problems with RPC Information the line
"RPC Extension not Populated!"
is printed to the output.
After that the metadata of the file is printed as a correct JSON string.

The problem is that I'm not able to parse the output of gdalinfo correctly as a JSON object.
Is there an option to avoid this warning, so I can get a correct JSON output?

Thank you
Markus Overesch

Add field name control for several layers in OGR to PDF

Expected behavior and actual behavior.

When using gdal_translate -of PDF imagery1.tif testpdf.pdf -co OGR_DATASOURCE=layers.vrt -co OGR_DISPLAY_LAYER_NAMES=points1 -co OGR_DISPLAY_FIELD=attribute1, I'm able to create a PDF with imagery1.tif as background image and the vector layer specified in a VRT file. This the expected actual behavior and also works correctly.

But if I have two different vector layers in my VRT file, and try to use gdal_translate -of PDF imagery1.tif testpdf.pdf -co OGR_DATASOURCE=layers.vrt -co OGR_DISPLAY_LAYER_NAMES=points1,points2 -co OGR_DISPLAY_FIELD=attribute1,attribute2, the features of the second vector layer are displayed as feature1, feature2, feature3....

This is probably because of:

OGR_DISPLAY_FIELD=name : Name of the field (matching the name of a field from the OGR layer definition) to use to build the label of features that appear in the "Model Tree" UI component of a well-known PDF viewer. For example, if the OGR layer has a field called "ID", this can be used as the value for that option : features in the "Model Tree" will be labelled from their value for the "ID" field. If not specified, sequential generic labels will be used ("feature1", "feature2", etc... ).

Nevertheless, it would be nice to have OGR_DISPLAY_FIELD work like OGR_DISPLAY_LAYER_NAMES, where it is possible to insert a comma separated list of names.

This issue is based on this question at gis.stackexchange.com

Creating VRTDerivedRasterBand programmatically looses PixelFunctionLanguage

Expected behavior and actual behavior.

Adding 'PixelFunctionLanguage=Python' in the options of a band of type VRTDerivedRasterBand when creating a VRT programmatically looses the same option again when writing to disk. I expect that it should keep the option as this produces invalid files.

Steps to reproduce the problem.

    driver = gdal.GetDriverByName('VRT') 
    vrt_ds = driver.Create(path, 10, 10, 0) 
    vrt_ds.SetProjection("+proj=latlong +datum=WGS84 +no_defs")
    vrt_ds.SetGeoTransform((0, 1, 0, 10, 0, -1))
    options = [
        'subClass=VRTDerivedRasterBand',
        'PixelFunctionLanguage=Python',
        'PixelFunctionType=foo.my_func'
    ]
    vrt_ds.AddBand(gdal.GDT_Byte, options)
    del vrt_ds

The resulting XML file does not contain PixelFunctionLanguage.

Operating system

Windows 10

GDAL version and provenance

2.2.4 bundled with QGIS

OGR2OGR TAB to Shapefile conversion issue

Operating system

Windows 7 Professional Service Pack 1

GDAL version and provenance

GDAL 2.2.4, released 2018/03/19 OSGeo4w

Improve VFK parser logic

There are issues with VFK parser. On lines below the parser fails.

• record with semicolon placed after quotes

&DLISTIN;725491306;20;"ze dne 9.11.1993 (""parcely""; z přídělu č. 15 pro¤
k.ú. Řepešín).";"n";;"";"";"98";"01.01.1992 00:00:00";"POLVZ:101994   KATUZ:789151 LISTDRUH:20 I:1";"";2821351306;"a";"";"";"11.06.2001 01:36:14";""

Warning 1: LISTIN: invalid number of properties 15 should be 17
OGR-VFK: Invalid VFK data record skipped (line 123561).
&DLISTIN;725491306;20;"ze dne 9.11.1993 (;""parcely"" z p°Ýdýlu Ŕ. 15 pro
k.˙. ěepe╣Ýn).";"n";;"";"";"98";"01.01.1992 00:00:00";"POLVZ:101994   KATUZ:789151 LISTDRUH:20 I:1";"";2821351306;"a";"";"";"11.06.2001 01:36:14";""

• record with semicolon placed before quotes

&DLISTIN;725491306;20;"ze dne 9.11.1993 (""parcely""; z přídělu č. 15 pro¤
k.ú. Řepešín).";"n";;"";"";"98";"01.01.1992 00:00:00";"POLVZ:101994   KATUZ:789151 LISTDRUH:20 I:1";"";2821351306;"a";"";"";"11.06.2001 01:36:14";""

Warning 1: LISTIN: invalid number of properties 14 should be 17
OGR-VFK: Invalid VFK data record skipped (line 123561).
&DLISTIN;725491306;20;"ze dne 9.11.1993 (;""parcely"" z p°Ýdýlu Ŕ. 15 pro
k.˙. ěepe╣Ýn).";"n";;"";"";"98";"01.01.1992 00:00:00";"POLVZ:101994   KATUZ:789151 LISTDRUH:20 I:1";"";2821351306;"a";"";"";"11.06.2001 01:36:14";""

• inconsistent number of quotes

&DUCAST;92457092010;41604162010;1;"";"";"";"";"";"";"";"";;"""Vystrkov 15-17 drustvo spoluvlastnk domu v Praze 4, Modanech, K Vystrkovu 1497/15";"""VYSTRKOV 15-17 DRUZSTVO SPOLUVLASTNIKU DOMU V PRAZE 4, MODRANECH, K VYSTRKOVU 1497/15";"";63083027;;"";2;2

GDAL For android

Expected behavior and actual behavior.

I use gdal to open image data on android,but it is problem when image data is bigger than 4GB;
and it works normal on windows

Steps to reproduce the problem.

I use gdal to open image data on android,but it is problem when image data is bigger than 4GB;
and it works normal on windows

Operating system

Android 5.0 ARM

GDAL version and provenance

GDAL2.2.3

Geometry Column not working correctly for Mysql spatial database

I have two linux systems accessing the same mysql spatial database using two different versions of GDAL. The first system works as expected, but the second system does not work as expected with newer GDAL software.

Expected behavior and actual behavior.

The first system:
ogrinfo MYSQL:,user=,password=,host=,port=3306 file -summary

Output:

Layer name: file
Geometry: Unknown (any)
Feature Count: 45508773
Extent: (-110.245340, 29.385490) - (-89.754660, 45.532940)
Layer SRS WKT:
(unknown)
FID Column = id
Geometry Column = shape
dataset_id: Integer (0.0)
host: String (0.0)

The second system using the same command:

Output:

Layer name: file
Metadata:
OLMD_FID64=YES
Geometry: Unknown (any)
Feature Count: 45508773
Extent: (-110.245340, 29.385490) - (-89.754660, 45.532940)
Layer SRS WKT:
(unknown)
FID Column = id
dataset_id: Integer (0.0) NOT NULL DEFAULT 0
host: String (0.0) NOT NULL DEFAULT 'localhost'

Notice that the Geometry Column does not appear in the second system output. Geometry Column should default to SHAPE as detailed in this web page:

http://www.gdal.org/drv_mysql.html

This causes mapserver, on the second system, to display the image in the wrong location, since mapserver is not receiving the coordinates from the SHAPE field from GDAL.

Operating system

first system: Linux 3.10.0-693.11.6.el7.x86_64
second system: Linux 3.10.0-693.17.1.el7.x86_64

GDAL version and provenance

First system: GDAL 1.11.4, released 2016/01/25
Second system: GDAL 2.2.3, released 2017/11/20

Exception when used gdal.VectorTranslate and -a_srs (or t_srs)

Expected behavior and actual behavior.

I am trying to use gdal.VectorTranslate in my python-program and setting the coordinate system for the output. As input I am using a SQL-statement on a postgis-database, output is a ESRI shapefile. When there is the option -a_srs or -t_srs in the options-string, the function raises an exception, and the outputfile is created, but empty.
Omitting this option, everything works fine (except that the prj-file for the output-shapefile is not created).
Also when I am using tables instead of -SQL option, everything works fine.
When using these options with ogr2ogr also everything works fine.
I am using a comparable options-string with -SQL and -a_srs when transfering data from oracle to postgis, and there the options work well and rais no error.

Exception:
File "C:\OSGEO4~1\apps\Python27\lib\site-packages\osgeo\gdal.py", line 684, in VectorTranslate
return wrapper_GDALVectorTranslateDestName(destNameOrDestDS, srcDS, opts, callback, callback_data)
RuntimeError: Terminating translation prematurely after failed

Steps to reproduce the problem.

The code below is a modified version, of what I am using in the python code. So there are maybe some typos.

optionsString = "-sql @D:\sql_files/GIP_ORA_ORTSSTRASSEN_UNION_V3.txt -overwrite -lco ENCODING=UTF-8 -t_srs epsg:31254"
pDs2 = openDBConnection("PG: host=hostname dbname=dbname user=user password=pwd", gdal.OF_VECTOR)
destFile = "D:\RueckexportShapes/ortsstrassen.shp"
gdal.VectorTranslate(destFile, pDs2, options=optionsString)

GIP_ORA_ORTSSTRASSEN_UNION_V3.txt

Operating system

Windows 7, 64 Bit
Postgres 9.6.1
Postgis 2.3.1
Python 2.7.5

GDAL version and provenance

2.1.3

Issue template !

Let's create an issue template 😄

here is the one we are using in rasterio

<!--

WELCOME ABOARD!

Hi and welcome to the Rasterio project. We appreciate bug reports, questions
about documentation, and suggestions for new features. This issue template
isn't intended to ward you off; only to intercept and redirect one particular
category of reports, and to collect a few important facts that issue reporters
often omit.

You think you've found something? We believe you.

Please note: Rasterio contains extension modules and is thus susceptible to
C library compatibility issues. If you are reporting an installation or module
import issue, please note that this project only accepts reports about problems
with packages downloaded from the Python Package Index. Conda users should take
issues to one of the following trackers:

- https://github.com/ContinuumIO/anaconda-issues/issues
- https://github.com/conda-forge/rasterio-feedstock

Also please note: we are currently working on 1.0 and pre-releases. Bugs found
in version 0.36 will not be fixed except in a 1.0 alpha or beta release. In 
some cases, 0.36 bugs have already been fixed in recent pre-releases.
-->

## Expected behavior and actual behavior.

For example: I expected to read a band from a file and an exception occurred.

## Steps to reproduce the problem.

For example: a brief script with required inputs.

## Operating system

For example: Mac OS X 10.12.3.

## Rasterio version and provenance

For example: the 1.0a11 manylinux1 wheel installed from PyPI using pip 9.0.1.

cc @rouault

OGR2OGR TAB to Shapefile conversion issue

Operating system

Windows 7 Professional Service Pack 1

GDAL version and provenance

GDAL 2.2.4, released 2018/03/19 OSGeo4w

Parameters for SQL files

Starting from ogr2ogr 2.1 it is possible to use as query a pointer to a text file "-sql @query.sql". It would be useful to be able to put some placeholder in the query and pass some additional argument to set their values.

For example,

$ cat query.sql
SELECT * FROM table WHERE id BETWEEN ? AND ?
$ ogr2ogr -sql @query.sql -sql-arg 42 -sql-arg 9999[...]

generates the same output of

$ ogr2ogr -sql "SELECT * FROM table WHERE id BETWEEN 42 AND 9999"  [...]

Of course, named arguments could be more useful for complex queries, especially when the same argument should be put in several places.

UnboundLocalError when iterating over a Feature [Python]

Expected behavior and actual behavior.

When you iterate over a Feature object (using the Python bindings) an Exception is raised. Even though an Exception probably makes sense, the error could/should probably be handled in a similar way as when requesting an invalid field (eg ft['doesnt_exist']). The current exception ('variable referenced before assignment') doesn't look intentional to me. If it is, this issue can be closed right away.

Instead of getting:
UnboundLocalError: local variable 'fld_index' referenced before assignment

It perhaps should be:
ValueError: Illegal field requested in GetField()

Raised at:
https://github.com/OSGeo/gdal/blob/master/gdal/swig/python/osgeo/ogr.py#L4686

In this case __getitem__ is called with an int, and currently fld_index is only set when using a str.

Steps to reproduce the problem.

import gdal

ds = gdal.OpenEx(vector_file)
lyr = ds.GetLayer(0)
                            
for ft in lyr:
    
    for field in ft:
        print(field)

Alternative behavior.

Additionally, it might be nice to actually return "something", instead of raising the error. For me it happened when I was trying to get the attributes of the features. Instead of looping over the Feature object, I should have used ft.keys() in combination with something like value = ft[key] inside the iteration.

If my use-case makes sense, perhaps returning a tuple of (key, value) could be an option. But the added benefit is marginal. It would allow something like:

for ft in lyr:    
    for attribute_name, attribute_value in ft:
        # ...

Instead of the current behavior:

for ft in lyr:    
    for attribute_name in ft.keys:
        attribute_value = ft[attribute_name]
        # ...

Operating system

Windows 10 64bit

GDAL version and provenance

from Conda-Forge:

gdal.VersionInfo()
'2020400'

Error 1: maximum number of characters allowed

I am getting this error with some of my files:

ERROR 1: Maximum number of characters allowed reached.

using this function call:

"/Library/Frameworks/GDAL.framework/Versions/2.2/Programs/gdalwarp" -multi -overwrite -te 243981.149256099 4169328.58664336 316574.679779182 4239670.70617033 -te_srs "+proj=utm +zone=11 +datum=WGS84 +units=m +no_defs" -tr 1000 1000 -t_srs "+proj=utm +zone=11 +datum=WGS84 +units=m +no_defs" -r "near" -dstnodata "-99" -of "GTiff" "data/snodas/swe/conus/us_ssmv11034tS__T0001TTNATS2017120105HP001.Hdr" "data/snodas/swe/tuo/swe_tuo_20171201.tif"

you can download the files here:
ftp://sidads.colorado.edu/DATASETS/NOAA/G02158/masked/2017/07_Jul/SNODAS_20171201.tar

Is there a character limit for lines in the header file?
Most of the other dates work fine, 20171101 is another example that doesn't work

macOS 10.12.6
GDAL 2.2.4, released 2018/03/19, from kyngchaos.com

AutoCAD DWG driver in FreeBSD

The issue

The cad driver (in 2.2.3 and 2.2.4) does not compile in FreeBSD and it is not possible to not include the driver. The error message is

../cadheader.h:82:5: error: constructor cannot be redeclared
CADVariant( time_t val );
^
../cadheader.h:71:5: note: previous declaration is here
CADVariant( int val );

It would be good - also as a general policy - to be able to switch off any driver to avoid problems like this becoming blockers - in this case GDAL is just one component of a test build and the driver in question is not needed. Now GDALmake.opt.in simply has

HAVE_CAD = yes

Operating system

FreeBSD 11.1-RELEASE (GENERIC) #0 r321309

GDAL version and provenance

Tested with 2.2.3 and 2.2.4. The git master does not have the line in source code having the problem any more.

Crash during gdal_translate from VRT to JP2 on Centos7 64bits + Kakadu 7.10.3

B01.zip

Expected behavior and actual behavior.

I try to make a reprojection to SRS 3857 of a jpeg2000 file, but the gdaltranslate operation crashes (segmentation fault). I get the following log :
ERROR 1 : Kakadu Core Error:
Malformed attribute string, "QStep=0.0000152588" !
Problem encountered at ".0000152588"
Record must be separated by commas
terminate called after throwing an instance of 'kdu_cpl_error_message::JP2KAKException"

I attach to this post the input JP2 file.

Steps to reproduce the problem.

gdal_warp -of VRT -t_srs EPSG:3857 input.jp2 output.VRT
gdaltranslate -of JP2KAK output.VRT output.JP2 --> crash

but :
gdal_warp -t_srs EPSG:3857 input.jp2 output.tif
gdaltranslate -of JP2KAK output.tif output.JP2 --> ok

Operating system

Centos 7.3.1611 64b

GDAL version and provenance

the 2.2.3 version (sources from http://download.osgeo.org/gdal/2.2.3/gdal223.zip)
Kakadu 7.10.3

Add versioning to vsis3

Expected behavior and actual behavior.

We have versioning turned on for our s3 buckets so we need the ability to access specific s3 versions when using vsis3.Currently, vsis3 will retrieve the latest version of a s3 file. We are looking for a functionality similar to boto where you can specify a version id when accessing a s3 file.

Steps to reproduce the problem.

obj = "s3://testing/test.txt"
sobj = .objlstrip('s3://')
strpth = "vsis3/" + sobj
img = gdal.Open(strpth)

Operating system

Centos 7

GDAL version and provenance

GDAL 2.2.1, released 2017/06/23

请问有中文使用文档吗

Expected behavior and actual behavior.

For example: I expected to be able to open this raster file (with a link to
the raster file, or it as an attachement) and it returns an error message
instead.

Steps to reproduce the problem.

For example: "gdalinfo myfile"

Operating system

For example: Ubuntu 16.04 64 bit

GDAL version and provenance

For example: the 2.2.3 version from ubuntugis-unstable PPA

If GDAL build is in fact dependant on 64 bit integer support, fail early if not available.

A configuration script bug when setting up a GDAL cross-compilation environment (#529) resulted in a build configuration that didn't think it's C compiler supported 64 bit integers (linux and GCC, HAVE_LONG_LONG not defined in cpl_config.h). Despite a comment in cpl_port.h suggesting this wasn't likely to work very well, compilation proceeded with a whole host of integer overflow compiler warnings (as GIntBig and GUIntBig ended up defined as 32 bit numbers) before ultimately failing.

If in fact GDAL is fundamentally dependant on 64 bit integer support, a #error in cpl_port.h would have saved me considerable time in figuring out what was going wrong.

OGR2OGR TAB to Shapefile conversion issue

Operating system

Windows 7 Professional Service Pack 1

GDAL version and provenance

GDAL 2.2.4, released 2018/03/19 OSGeo4w

Python crash at: SetField('plaintext', chr(56572))

Expected behavior and actual behavior.

Python (QGIS) crash

Steps to reproduce the problem.

# -*- coding: utf-8 -*-
import os, tempfile, ogr
shpDat=tempfile.gettempdir() + "/test.shp"
if os.path.exists(shpDat):
    os.remove(shpDat)

driver = ogr.GetDriverByName("ESRI Shapefile")
data_source = driver.CreateDataSource(shpDat)
layer = data_source.CreateLayer("test")
layer.CreateField(ogr.FieldDefn('plaintext', ogr.OFTString))
feature = ogr.Feature(layer.GetLayerDefn())
feature.SetField('plaintext', chr(56572)) # python crash
layer.CreateFeature(feature)
data_source.Destroy()

That's just a test code. The real problem occurs when OGR2OGR generates such a shape.

Operating system

Windows

GDAL version and provenance

GDAL 2.2.3, Python '3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 07:18:10) [MSC v.1900 32 bit (Intel)]'

Non-existing GetUnits() referenced in doc strings

GDALRasterBand::GetOffset and GDALRasterBand::GetScale have the following sentence:

This value (in combination with the GetOffset() value) is used to transform raw pixel values into the units returned by GetUnits().

There is no such method GetUnits. Should that be GetUnitType? Also, now I'm a little unsure whether GDALRasterBand::RasterIO and GDALRasterBand::ReadBlock already have the scale and offset applied or whether that must be a post-processing step. I think the doc strings could be more clearer on this.

OGR2OGR TAB to Shapefile conversion issue

Operating system

Windows 7 Professional Service Pack 1

GDAL version and provenance

GDAL 2.2.4, released 2018/03/19 OSGeo4w

Jpeg2000 and transparency

Expected behavior and actual behavior.

I would like to reproject a JP2 File from EPSG 2154 to EPSG 3857 and to store the target image in a JP2 File with transparent background.
I expect the output.jp2 to have tranparent pixel areas (due to the reprojection), but background is black.
Is there something particular to do ?

Steps to reproduce the problem.

gdalwarp -of VRT -t_srs 3857 -dstalpha input.jp2 output.vrt
gdal_translate -of JP2KAK output.vrt output.jp2

Operating system

Centos 7.3.1611 64bits

GDAL version and provenance

Gdal 2.2.3 (sources from http://download.osgeo.org/gdal/)
Kakadu 7.10.4

Autotools configure script inconsistent in use of CFLAGS versus CCFLAGS when invoking C compiler

configure script contains many invocations of C and C++ compilers to determine compiler capabilities. Some C compiler invocations specify content of GNU make standard CFLAGS environment variable as compiler arguments, while others specify non-standard CCFLAGS. None of this matters much when doing a native build as the compilers will generally work without any flag values, however when using a cross-compiler the toolchain is often critically dependant on compiler flags directing target processor details and will fail if nothing is provided. This can lead to erroneous evaluation of compiler capabilities by the configure script.

In my particular case I was cross-compiling for an ARM linux target and passed the target architecture compiler flags in via CFLAGS. This resulted in a system that was mostly configured correctly, but was of the opinion that it's C compiler didn't support 64 bit integers, with ensuing hilarity during the subsequent build process.

Suggested fix would be to aggregate both CFLAGS and CCFLAGS env. variable contents into a superset variable and use it consistently across all C compiler invocations.

gdal.TermProgress failing using gdal_fillnodata.py

The following command for gdal_fillnodata.py (with the GDAL installation from https://www.lfd.uci.edu/~gohlke/pythonlibs/#gdal)

gdal_fillnodata.py -of GTiff C:/tmp/grid_withnodata.asc C:/tmp/grid_filled.tif

Returns the following error:

Traceback (most recent call last):
  File "C:\Python35\Lib\site-packages\osgeo\scripts\gdal_fillnodata.py", line 19
7, in <module>
    callback = prog_func )
  File "C:\Python35\lib\site-packages\osgeo\gdal.py", line 2750, in FillNodata
    return _gdal.FillNodata(*args, **kwargs)
RuntimeError: Object given is not a Python function

The pointer comparison for the gdal.TermProgress Python/SWIG function and the GDALTermProgress C function seems to fail at:
https://github.com/OSGeo/gdal/blob/release/2.2/gdal/swig/include/python/typemaps_python.i#L1320.

if ($input && $input != Py_None ) {
    void* cbfunction = NULL;
    CPL_IGNORE_RET_VAL(SWIG_ConvertPtr( $input,
                     (void**)&cbfunction,
                     SWIGTYPE_p_f_double_p_q_const__char_p_void__int,
                     SWIG_POINTER_EXCEPTION | 0 ));

    if ( cbfunction == GDALTermProgress ) {
        $1 = GDALTermProgress;
    } else {
        if (!PyCallable_Check($input)) {
            PyErr_SetString( PyExc_RuntimeError,
                             "Object given is not a Python function" );
            SWIG_fail;
        }
        psProgressInfo->psPyCallback = $input;
        $1 = PyProgressProxy;
    }

}

Maybe a SWIG 3.0.12 bug? File of the command can be obtained from here: grid_withnodata.zip

Christoph Gohlke noticed a workaround, run gdal_fillnodata.py in quiet mode (-q) to avoid the callback.

Automatically promote 1-bit external masks to 8-bit

In #468, we discovered/discussed that 1-bit external masks (created with NBITS=1) are handled differently from internal masks, specifically that internal masks have their values expanded to 0..255 while (1-bit) external masks are not (values remain 0..1). This means that images w/ 1-bit external masks are not handled correctly in GDAL-linked tools like QGIS (mask values are presumably used as transparency, so the entire image remains opaque).

LIKE '%Wert'” not working correct with ogr.ExecuteSQL and dialect = SQLITE

Expected behavior and actual behavior.

I am trying to count features that have the text "Fehler" in a certain field, so i was using WHERE rstatus LIKE '%Fehler'. This only works, when using the OGRSQL-dialect, but returns None when using SQLITE. I want to use the SQLITE-dialect, because I am looping over several SQL-Strings, and some of them contain SUM(LENGTH)/1000, which only works in SQLITE, and not with OGRSQL. I checked the field-values, there are no trailing spaces. When using trim(rstatus) like '%Fehler' the statement also works with SQLITE dialect.
Further I was wondering, why the result is None, rather than a layer-object where GetFeatureCount = 0, when no records meet the criteria.

OGR_Select.zip

Steps to reproduce the problem.

import os

try:
    from osgeo import ogr
    from osgeo import gdal
except:
    sys.exit("Failed to import OGR-module. Please check the GDAL-Installation!")

srcFile = r"D:\Didi\Tmp\OGR_Select\Export.shp"
driver = ogr.GetDriverByName('ESRI Shapefile')
destFileDS = driver.Open(srcFile, 0)
sql = u"SELECT COUNT(*) as Anzahl from %s WHERE rstatus like '%%Fehler'" % (os.path.basename(srcFile)[:-4])

#Works
actLayer =  destFileDS.ExecuteSQL(statement=sql.encode('utf-8'), dialect='OGRSQL')
actFeat = actLayer.GetNextFeature()
destFileDS.ReleaseResultSet(actLayer)

#Returns none
actLayer =  destFileDS.ExecuteSQL(statement=sql.encode('utf-8'), dialect='SQLITE')
actFeat = actLayer.GetNextFeature()
destFileDS.ReleaseResultSet(actLayer)

Operating system

Windows 7, 64 Bit
Python 2.7.5

GDAL version and provenance

GDAL 2.1.3

GTiff/GeoTIFF driver uses wrong metada from files with simillar names

Expected behavior and actual behavior.

For example opening the file with the name:
DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001_ql16.tif
reads metada from the files:
DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001.XML
and
RPC_PHR1B_PMS-N_201708210848215_SEN_2471996101-001.XML
in the same folder.
gdalinfo gives the following info:
Files: c:\temp\DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001_ql16.tif
c:\temp\DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001.XML
c:\temp\RPC_PHR1B_PMS-N_201708210848215_SEN_2471996101-001.XML

as a result calling GDALDeleteDataset for the dataset of DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001_ql16.tif deletes the XML files as well!

It's worth to note, that renaming the TIF file into DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001-ql16.tif or DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001.ql16.tif gives normal behavior and skips using the XML files

Steps to reproduce the problem.

files.zip

Put the files from the files.zip archive into the same folder, and execute
gdalinfo DIM_PHR1B_PMS-N_201708210848215_SEN_2471996101-001_ql16.tif

Operating system

Windows 7 x64

GDAL version and provenance

For example: GDAL 2.2.4, released 2018/03/19

CSV type autodetection wrongly identifies number field as string

Expected behavior and actual behavior.

I expected GDAL CSV driver to correctly detect an all-numbers field as a real type column but instead it guesses it as a string field.

Here is an example csv file to reproduce this problem.

But this other csv file does not trigger the problem.

Steps to reproduce the problem.

curl -O https://gist.githubusercontent.com/rafatower/40077db244271cbbcc975ecddcb7e039/raw/2fac5c32ba24adba9fcf72d911c37282cfed7012/wrong_type_guessing.csv
ogrinfo -al -so -oo AUTODETECT_TYPE=YES wrong_type_guessing.csv

outputs my_number_column: String (0.0) (incorrectly auto-detected):

INFO: Open of `wrong_type_guessing.csv'
      using driver `CSV' successful.

Layer name: wrong_type_guessing
Geometry: None
Feature Count: 5
Layer SRS WKT:
(unknown)
cartodb_id: Integer (0.0)
my_number_column: String (0.0)

whereas this:

curl -O https://gist.githubusercontent.com/rafatower/bfcc88318ec2c270b55e31fecba4f238/raw/a283f4116e2a59846bc4699447206c6c382d1fc9/correct_type_guessing.csv
ogrinfo -al -so -oo AUTODETECT_TYPE=YES correct_type_guessing.csv

outputs my_number_column: Real (0.0) (as expected):

INFO: Open of `correct_type_guessing.csv'
      using driver `CSV' successful.

Layer name: correct_type_guessing
Geometry: None
Feature Count: 5
Layer SRS WKT:
(unknown)
cartodb_id: Integer (0.0)
my_number_column: Real (0.0)

Operating system

Ubuntu 16.04.4 LTS 64 bit

GDAL version and provenance

2.2.3 version from source repository (commit e70471b) compiled in my ubuntu machine.

Additional info

I suspect it has a lot to do with the "promotion rules" and the decimal separators:

gdal/gdal/ogr/ogrsf_frmts/csv/ogrcsvlayer.cpp

Lines 1183 to 1215 in eb9ed3f

	else if( aeFieldType[iField] != eOGRFieldType )
	{
	// Promotion rules.
	if( aeFieldType[iField] == OFTInteger )
	{
	if( eOGRFieldType == OFTInteger64 ||
	eOGRFieldType == OFTReal )
	aeFieldType[iField] = eOGRFieldType;
	else
	{
	aeFieldType[iField] = OFTString;
	nStringFieldCount++;
	}
	}
	else if( aeFieldType[iField] == OFTInteger64 )
	{
	if( eOGRFieldType == OFTReal )
	aeFieldType[iField] = eOGRFieldType;
	else if( eOGRFieldType != OFTInteger )
	{
	aeFieldType[iField] = OFTString;
	nStringFieldCount++;
	}
	}
	else if( aeFieldType[iField] == OFTReal )
	{
	if( eOGRFieldType != OFTInteger &&
	eOGRFieldType != OFTReal )
	{
	aeFieldType[iField] = OFTString;
	nStringFieldCount++;
	}
	}

What it has guessed so far about the type can influence the decision about the type given another type in a further row, and it tries to promote to the super-type (e.g int to bigint and so forth). It always defaults to string and I think it lacks some conversion rules/types.

VFK driver: suppress geometry

Expected behavior and actual behavior.

VFK driver resolves features' geometry by default. It can be time-consuming task when reading huge VFK files. Some of users are not interested at features geometries. It would be reasonable to add new open option to suppress resolving geometry. Than all reported layers would have wkbNone type.

Cannot create and write a zip file with vsizip...

I'm trying to write a GeoTIFF to zip file with GDAL library and its virtual filesystem driver vsizip. Alas, it gives me an error:

ERROR 1: Random access not supported for writable file in /vsizip
ERROR 4: Attempt to create new tiff file `/vsizip/D:\Projects\oy_vey.zip\my.tif'
 failed: No such file or directory

How can I cope with it?

P.S. Here's my code:

#include "stdafx.h"
#include "MainWnd.h"
#include <QtWidgets/QApplication>

#include <gdal.h>
#include <gdal_priv.h>
#include <gdal_frmts.h>
#include <ogr_spatialref.h>

int main(int argc, char *argv[])
{
	//QApplication a(argc, argv);
	//MainWnd w;
	//w.show();
	//return a.exec();

	GDALRegister_GTiff();

	int numLatPx = 480;
	int numLonPx = 640;

	double northDeg = 3;
	double southDeg = 0;
	double westDeg = 0;
	double eastDeg = 4;


	double latStep = (northDeg - southDeg) / numLatPx;
	double lonStep = (eastDeg - westDeg) / numLonPx;


	GDALDriver* poDriver;
	char** papszMetadata;

	GDALDriverManager* driverManager = GetGDALDriverManager();
	poDriver = driverManager->GetDriverByName("GTiff");
	if (poDriver == NULL)
	{
		qDebug() << "GEOTIFF DRIVER NULL!!!";
		exit(1);
	}
	//it's okay so far...

	GDALDataset* poDstDS;

	double adfGeoTransform[6];
	adfGeoTransform[0] = westDeg; /* top left x */
	adfGeoTransform[1] = lonStep; /* w-e pixel resolution */
	adfGeoTransform[2] = 0; /* 0 */
	adfGeoTransform[3] = northDeg; /* top left y */
	adfGeoTransform[4] = 0; /* 0 */
	adfGeoTransform[5] = -latStep;/* n-s pixel resolution (negative value) */

	char** papszOptions = NULL;

	poDstDS = poDriver->Create("/vsizip/D:\\Projects\\oy_vey.zip\\my.tif", numLonPx, numLatPx, 1, GDT_Int16, papszOptions);
	// poDstDS will give NULL

	OGRSpatialReference oSRS;
	char *pszSRS_WKT = NULL;
	GDALRasterBand* poBand;

	poDstDS->SetGeoTransform(adfGeoTransform);
	//oSRS.importFromEPSG(4284); // PULKOVO 1942
	oSRS.exportToWkt(&pszSRS_WKT);

	poDstDS->SetProjection(pszSRS_WKT);
	CPLFree(pszSRS_WKT);
	poBand = poDstDS->GetRasterBand(1);

	GInt16* abyRaster = new GInt16[numLatPx * numLonPx];

	abyRaster[10000] = 0;
	abyRaster[10001] = 1;

	poBand->RasterIO(GF_Write, 0, 0, numLonPx, numLatPx, abyRaster, numLonPx, numLatPx, GDT_Int16, 0, 0);

	double minVal, maxVal, meanVal, stdDevVal;
	poBand->ComputeStatistics(false, &minVal, &maxVal, &meanVal, &stdDevVal, NULL, NULL);
	poBand->SetStatistics(minVal, maxVal, meanVal, stdDevVal);
	poBand->SetColorInterpretation(GCI_GrayIndex);


	/* Once we're done, close properly the dataset */
	GDALClose((GDALDatasetH)poDstDS);



	qDebug() << "^_^" << endl;
	return 0;
}

Autotools configure script doesn't honour --with-sysroot directive when locating external driver libraries

Unless a specific location for an optional driver/package/library has been specified, configure script searches from the build platform root, even when a --with-sysroot directive has been supplied, and on finding it potentially drags build platform include paths into the build configuration.

Cross-compiling with a custom sysroot, I'm not only having to specify the sysroot with a --with-sysroot=/foo/bar directive, but having to repeat myself on every optional package inclusion directive --with-sqlite3=/foo/bar

gdalinfo returns wrong maximum for digital elevation map such as *.BIL or *.csv

Expected behavior and actual behavior.

For example: I expected to be able to open this raster file (with a link to
the raster file, or it as an attachement) and it returns an error message
instead.

Steps to reproduce the problem.

For example: "gdalinfo myfile"

Operating system

For example: Ubuntu 16.04 64 bit

GDAL version and provenance

For example: the 2.2.3 version from ubuntugis-unstable PPA

Cannot Load DEM defined in UTM zone 19 Southern Hemisphere

There is a bug in the latest version of GDAL 2.2.3 that when we trying to load a DEM whose UTM zone is defined as "-19" (UTM zone 19 Southern Hemisphere). GDAL throws "Invalid Zone -19" exception. However, in the earlier version GDAL201, I was able to load the same DEM perfectly. There is a bug which recently introduced.

The DEM should load perfectly in the UTM zone 19 Southern Hemisphere as it was loading.

Steps to reproduce the problem.

GDAL.Open(DEMFIle) and it throws the "Invalid Zone -19" exception.

Operating system

Windows 10 x64

GDAL version and provenance

GDAL version 2.2.3 version

WCS: Failing SSL handshake blocks subsequent trials

Expected behavior and actual behavior.

@ajolma

If WCS driver fails with SSL handshaking because of certificate problem an empty xml file like "XYZZXY.xml" is written into the wcs_cache directory. If the request if repeated by setting the config option GDAL_HTTP_UNSAFESSL=YES the request fails again but now because WCS driver tries to use the cached xml file, which does exist but which is empty and invalid.

WCS driver should not write an empty XML file if it cannot make a proper connection with the WCS server. If it finds an existing but invalid xml file it should do something reasonable, perhaps to delete the faulty xml file from the cache and try to read the contents again from the WMS server.

Steps to reproduce the problem.

The error can be reproduced today with the following commands but probably the server certificate will be fixed soon.

First request:
gdalinfo WCS:"https://beta-karttakuva.maanmittauslaitos.fi/wcs/service/ows?"
ERROR 1: SSL certificate problem: unable to get local issuer certificate
gdalinfo failed - unable to open 'WCS:https://beta-karttakuva.maanmittauslaitos.fi/wcs/service/ows?'

Second request:
gdalinfo WCS:"https://beta-karttakuva.maanmittauslaitos.fi/wcs/service/ows?" --config
GDAL_HTTP_UNSAFESSL YES
ERROR 3: Cannot open file 'C:\Users\JRAHKONEN.gdal\wcs_cache\RFWGM.aux.xml'
gdalinfo failed - unable to open 'WCS:https://beta-karttakuva.maanmittauslaitos.fi/wcs/service/ows?'

Operating system

GDAL version and provenance

GDAL version 2.2.4-dev from gisinternals.com

Add support for iterative geojson parser

Expected behavior and actual behavior.

When opening data from https://www.alltheplaces.xyz/ I only get the first record and not all several 100,000 records in QGIS. QGIS is using the ORG GeoJSON driver to open the file.

Steps to reproduce the problem.

Go to https://www.alltheplaces.xyz/ and choose download. The latest download I used was: https://s3.amazonaws.com/placescraper-results/runs/2018-03-25-18-54-21/output.geojson.gz. Uncompress and open the file into QGIS. Only the first record is imported.

I believe this is because this file is almost a GeoJSON feature collection, but not quite. It's just missing the Feature Collection declaration at the top and closure at the bottom.

Instead it's a newline delimitated file where each GeoJSON feature is on it's own line as following this spec: http://ndjson.org/, making it easier to stream the file. It would be great if OGR could support this GeoJSON variant directly.

Operating system

macOS 10.13.3

GDAL version and provenance

2.1.3 version from QGIS 2.18.15

'gdal_contour' without '-3d' option for output file in 'GeoJSON' (and KML) format return geometry with '3D Line String' type instead 'Line String'

Expected behavior and actual behavior.

'gdal_contour' (http://www.gdal.org/gdal_contour.html) without '-3d' option (Force production of 3D vectors instead of 2D. Includes elevation at every vertex.) for output file in 'GeoJSON' (and KML) format return geometry with '3D Line String' type instead 'Line String'

Steps to reproduce the problem.

Test data: https://demo.webodm.org/api/projects/3/tasks/904c1c56-10c5-48df-a104-0138f699dd69/download/dsm.tif (Surface Model https://demo.webodm.org/map/project/3/task/904c1c56-10c5-48df-a104-0138f699dd69/)

$ gdal_contour -a elev -i 2.0 -f "ESRI Shapefile" dsm.tif contours_2.shp
$ ogrinfo contours_2.shp
INFO: Open of `contours_2.shp'
      using driver `ESRI Shapefile' successful.
1: contours_2 (Line String)

$ gdal_contour -a elev -i 2.0 -f "GeoJSON" dsm.tif contours_2.geojson
$ ogrinfo contours_2.geojson
INFO: Open of `contours_2.geojson'
      using driver `GeoJSON' successful.
1: contour (3D Line String)                      <---- '3D Line String' instead 'Line String'

$ gdal_contour -a elev -3d -i 2.0 -f "ESRI Shapefile" dsm.tif contours_3d_2.shp
ogrinfo contours_3d_2.shp
INFO: Open of `contours_3d_2.shp'
      using driver `ESRI Shapefile' successful.
1: contours_3d_2 (3D Line String)

$ gdal_contour -a elev -3d -i 2.0 -f "GeoJSON" dsm.tif contours_3d_2.geojson
ogrinfo contours_3d_2.geojson
INFO: Open of `contours_3d_2.geojson'
      using driver `GeoJSON' successful.
1: contour (3D Line String)

Operating system

Windows 10, QGIS 3.0.1 (GDAL 2.2.4, released 2018/03/19)

GDAL version and provenance

GDAL 2.2.4, released 2018/03/19

postgis sql with gdal.VectorTranlsate()

Expected behavior and actual behavior.

Apologies if this is a usage error, not a bug.

Using the Python bindings' gdal.VectorTranslate() function, I expect that when I translate from PostGIS to other formats using the SQLStatement option, gdal/ogr will pass the SQL to Postgres as it does when calling ogr2ogr with -sql. However, when using PostGIS functions in the sql, this error is given: ERROR 1: Undefined function 'ST_Point' used. It looks like gdal.VectorTranslate is trying to use the default OGRSQL dialect?

Steps to reproduce the problem.

$ python
Python 2.7.14 (default, Mar  9 2018, 23:57:12)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from osgeo import gdal
>>> gdal.VectorTranslate(
...     "t.shp",
...     "PG:host='localhost' port='5432' dbname='postgis' user='postgres' password='postgres'",
...     SQLStatement="SELECT 1 AS id, ST_Point(1, 1) as geom",
...     accessMode='overwrite')
ERROR 1: Undefined function 'ST_Point' used.
>>>

Operating system

macOS 10.13.3

GDAL version and provenance

GDAL 2.2.4, released 2018/03/19, from homebrew

PostGIS version

$ psql postgis -U postgres
psql (10.3)
Type "help" for help.

postgis=# select postgis_full_version();
                                                                                                                                                                                             postgis_full_version
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="2.4.4 r16526" PGSQL="100" GEOS="3.6.2-CAPI-1.10.2 4d2925d6" SFCGAL="1.3.2" PROJ="Rel. 5.0.1, April 1st, 2018" GDAL="GDAL 2.2.4, released 2018/03/19" LIBXML="2.9.7" LIBJSON="0.12.1" (core procs from "2.4.3 r16312" need upgrade) TOPOLOGY (topology procs from "2.4.2 r16113" need upgrade) RASTER (raster procs from "2.4.3 r16312" need upgrade) (sfcgal procs from "2.4.2 r16113" need upgrade)
(1 row)