feature: facilitate a wfs 2 as a backend about pygeoapi

[jorgejesus]

Use WFS2.0 as data source for pygeoapi

[tomkralidis]

+1 great feature! Considerations:

• we can use OWSLib’s WFS functionality to interact with WFS 1 or 2 sources
• we can also use OWSLib’s Filter XML functionality to convert WFS 3 property filters into WFS 1/2 Filter XML. This may get tricky depending on the underlying WFS 1/2 implementation
• the devil will be in the WFS 1/2 GetFeature responses. OWSLib simply returns string/XML object representations which would need to be converted into GeoJSON. Suggest we start with common WFS 1/2 implementations (MapServer, GeoServer, others?)

[jorgejesus]

• Need dockers with WFS services for testing on travis (https://docs.travis-ci.com/user/docker)
• Starting with mapserver WFS2.0 support and then WFS2.0 in Geoserver

[justb4]

Think this can be solved in a compact way with a single OGRProvider: a PyGeoAPI Provider that uses the GDAL/OGR Python bindings. Did some tests with both GeoServer and MapServer WFS v2 backends like from PDOK (Dutch National SDI). Many of the WFS-FES capabilities can be supported directly (if backend supports WFS v2):

• result type hits: feature count
• paging: translates to WFS paging with configurable pagesize
• spatial filtering: e.g. bounding box or any other
• attribute filtering, including get-feature-by-id: e.g. get feature by (gml) id

Advantages are:

• no need for GML request construction (like Filters) and response parsing and GML-GeoJSON mapping
• support not just WFS v2 but (m)any OGR source-types like GPKG, Shapefile, FGDB etc
• Proxy support for URL-based backends
• multiple auth methods support for URL-based backends

Things to consider:

• OGR WFS Driver may issue WFS GetCapabilities request on each Open() call (need investigation)
• the regular WFS v1, v2 performance considerations like complex filters bringing a WFS down

Software Design consideration:

• Develop a single OGRProvider as a one-in-all Provider, or as an ABC with specialized OGRProvider-derived classes like OGRWFSProvider, OGRShapefileProvider etc.

I would aim first for OGRProvider with specialized Driver/OGRSource-type-specific helper-classes.
We do something very similar within a Stetl OGRInput ETL class. Main difference (per Source-type) is really on per-OGR-Driver-specific Source options (next to standard GDAL/OGR options). We can catch this within the PGA config YAML like:

                        .
                        .
    provider:
        name: OGR
        data:
            source_type: WFS
            source: WFS:http://geodata.nationaalgeoregister.nl/rdinfo/wfs?
            source_options:
                VERSION: 2.0.0
                OGR_WFS_PAGING_ALLOWED: YES
                OGR_WFS_LOAD_MULTIPLE_LAYER_DEFN: NO
             gdal_ogr_options:
                GDAL_CACHEMAX: 64
                GDAL_HTTP_PROXY: (optional proxy)
                GDAL_PROXY_AUTH: (optional auth for remote WFS)
                CPL_DEBUG: NO
        id_field: gml_id
        layer: rdinfo:stations
                        .
                        .

    provider:
        name: OGR
        data:
            source_type: ESRI Shapefile
            source: tests/data/rdinfo.shp
            source_options:
                        .
            gdal_ogr_options:
                GDAL_CACHEMAX: 64
        id_field: fid
        layer: stations

I can take stab at this, would be nice to have something running to e.g. wrap all PDOK WFS-es.

refs:

• WGS WFS Driver: https://gdal.org/drv_wfs.html
• Python GDAL-OGR Cookbook: https://pcjericks.github.io/py-gdalogr-cookbook/vector_layers.h

[tomkralidis]

Thanks @justb4. This approach works. I'm glad the plugin framework is flexible enough to encompass an OGR driver. In one way, OGR can replace every other provider that implements the same thing. It will be good to have them all, and let them mature/evolve over time.

[justb4]

update: OGR Provider can even do (paged) fetch from ESRI FeatureServers!

[justb4]

Think we're done with OGRProvider for WFS2 backends. Tested with GeoServer, MapServer and deegree WFSs (see tests), including bbox queries. Other OGR backends (Drivers) work as well like GPKG, ESRI Shapefile, ESRI FeatureServer, except for paging (will need to use OGR SQL for most). Will open separate issue(s) for those.

So closing here, re-open if still issues with WFS2 OGR backends.