geopandas / xyzservices Goto Github PK

View Code? Open in Web Editor NEW

143.0 143.0 26.0 3.92 MB

Source of XYZ tiles providers

Home Page: https://xyzservices.readthedocs.io/

License: BSD 3-Clause "New" or "Revised" License

Python 99.46% Makefile 0.54%

xyzservices's Introduction

GeoPandas

Python tools for geographic data

Introduction

GeoPandas is a project to add support for geographic data to pandas objects. It currently implements GeoSeries and GeoDataFrame types which are subclasses of pandas.Series and pandas.DataFrame respectively. GeoPandas objects can act on shapely geometry objects and perform geometric operations.

GeoPandas geometry operations are cartesian. The coordinate reference system (crs) can be stored as an attribute on an object, and is automatically set when loading from a file. Objects may be transformed to new coordinate systems with the to_crs() method. There is currently no enforcement of like coordinates for operations, but that may change in the future.

Documentation is available at geopandas.org (current release) and Read the Docs (release and development versions).

The GeoPandas project uses an open governance model and is fiscally sponsored by NumFOCUS. Consider making a tax-deductible donation to help the project pay for developer time, professional services, travel, workshops, and a variety of other needs.

Install

See the installation docs for all details. GeoPandas depends on the following packages:

pandas
shapely
pyogrio
pyproj
packaging

Further, matplotlib is an optional dependency, required for plotting. Those packages depend on several low-level libraries for geospatial analysis, which can be a challenge to install. Therefore, we recommend to install GeoPandas using the conda package manager. See the installation docs for more details.

Get in touch

Ask usage questions ("How do I?") on StackOverflow or GIS StackExchange.
Get involved in discussions on GitHub
Report bugs, suggest features or view the source code on GitHub.
For a quick question about a bug report or feature request, or Pull Request, head over to the gitter channel.
For less well defined questions or ideas, or to announce other projects of interest to GeoPandas users, ... use the mailing list.

Examples

>>> import geopandas
>>> from shapely.geometry import Polygon
>>> p1 = Polygon([(0, 0), (1, 0), (1, 1)])
>>> p2 = Polygon([(0, 0), (1, 0), (1, 1), (0, 1)])
>>> p3 = Polygon([(2, 0), (3, 0), (3, 1), (2, 1)])
>>> g = geopandas.GeoSeries([p1, p2, p3])
>>> g
0         POLYGON ((0 0, 1 0, 1 1, 0 0))
1    POLYGON ((0 0, 1 0, 1 1, 0 1, 0 0))
2    POLYGON ((2 0, 3 0, 3 1, 2 1, 2 0))
dtype: geometry

Some geographic operations return normal pandas objects. The area property of a GeoSeries will return a pandas.Series containing the area of each item in the GeoSeries:

>>> print(g.area)
0    0.5
1    1.0
2    1.0
dtype: float64

Other operations return GeoPandas objects:

>>> g.buffer(0.5)
0    POLYGON ((-0.3535533905932737 0.35355339059327...
1    POLYGON ((-0.5 0, -0.5 1, -0.4975923633360985 ...
2    POLYGON ((1.5 0, 1.5 1, 1.502407636663901 1.04...
dtype: geometry

GeoPandas objects also know how to plot themselves. GeoPandas uses matplotlib for plotting. To generate a plot of a GeoSeries, use:

>>> g.plot()

GeoPandas also implements alternate constructors that can read any data format recognized by pyogrio. To read a zip file containing an ESRI shapefile with the boroughs boundaries of New York City (the example can be fetched using the geodatasets package):

>>> import geodatasets
>>> nybb_path = geodatasets.get_path('nybb')
>>> boros = geopandas.read_file(nybb_path)
>>> boros.set_index('BoroCode', inplace=True)
>>> boros.sort_index(inplace=True)
>>> boros
               BoroName     Shape_Leng    Shape_Area  \
BoroCode
1             Manhattan  359299.096471  6.364715e+08
2                 Bronx  464392.991824  1.186925e+09
3              Brooklyn  741080.523166  1.937479e+09
4                Queens  896344.047763  3.045213e+09
5         Staten Island  330470.010332  1.623820e+09

                                                   geometry
BoroCode
1         MULTIPOLYGON (((981219.0557861328 188655.31579...
2         MULTIPOLYGON (((1012821.805786133 229228.26458...
3         MULTIPOLYGON (((1021176.479003906 151374.79699...
4         MULTIPOLYGON (((1029606.076599121 156073.81420...
5         MULTIPOLYGON (((970217.0223999023 145643.33221...

>>> boros['geometry'].convex_hull
BoroCode
1    POLYGON ((977855.4451904297 188082.3223876953,...
2    POLYGON ((1017949.977600098 225426.8845825195,...
3    POLYGON ((988872.8212280273 146772.0317993164,...
4    POLYGON ((1000721.531799316 136681.776184082, ...
5    POLYGON ((915517.6877458114 120121.8812543372,...
dtype: geometry

xyzservices's People

Contributors

Stargazers

Watchers

xyzservices's Issues

Conquer the world

Once xyzservices is relased and has a stable API, it would be great to see it adopted as the one-stop shop for XYZ services in Python. Opening this ticket to brainstorm on which for which packages we would love to adopt the library, and what we could do to make adoption easy and smooth.

So far:

contextily: version 1.2 will swap its current providers submodule for xyzservices under the hood, adding deprecation notices. This means the contextily user will be able to pass a tile provider to, say, add_basemap and it'll magically use that without any attribution, URL, etc. being passed. -> geopandas/contextily#183
geopandas explore: first release will rely directly on xyzservices for tile providers. The user will be able to pass a TileProvider object to specify the basemap to be used

It'd be great to also get on board:

folium: the user would be able to pass a TileProvider object and avoid specifying URL, attribution, etc. arguments -> python-visualization/folium#1498; folium will depend on xyzservices from 0.16 -> python-visualization/folium#1827
ipyleaflet: the user would be able to pass a TileProvider object and avoid specifying URL, attribution, etc. arguments. Conversations have started over at contextily#153. -> jupyter-widgets/ipyleaflet#854
leafmap: the user would be able to pass a TileProvider object and avoid specifying URL, attribution, etc. arguments -> opengeos/leafmap#91
holoviews -> holoviz/holoviews#5057
geoviews -> holoviz/geoviews#517
here-map-widget-for-jupyter -> heremaps/here-map-widget-for-jupyter#27
bokeh has similar machinery as Holoviews. Could support TileProvider as input or depend on us directly when specifying metadata. (https://docs.bokeh.org/en/latest/docs/reference/tile_providers.html#bokeh-tile-providers) -> bokeh/bokeh#11500 and bokeh/bokeh#11512
plotly is another one but I am not sure about their implementation

a lot of edits by @martinfleis

Can not query `NASAGIBS.ASTER_GDEM_Greyscale_Shaded_Relief`

Hi. Here is my problem. I try to use the query_name function to query the NASAGIBS.ASTER_GDEM_Greyscale_Shaded_Relief tile providers (note the _ in this name). Unfortunately, it is impossible.

from xyzservices import providers
providers.query_name('NASAGIBS.ASTER_GDEM_Greyscale_Shaded_Relief')

When you look at the implementation of query_name, you realize it seems impossible to query this tile provider.

xyz_flat_lower = {
    k.replace(".", "").lower(): v for k, v in self.flatten().items()
}
name_clean = name
remove = "., -_/"
for string in remove:
    name_clean = name_clean.replace(string, "")
name_clean = name_clean.lower()
if name_clean in xyz_flat_lower:
    return xyz_flat_lower[name_clean]

raise ValueError(f"No matching provider found for the query '{name}'.")

In the xyz_flat_lower dictionary, only dots are replaced in the tile provider name while for the provided name variable, _ are also replaced. Thus, the name_clean variable is 'nasagibsastergdemgreyscaleshadedrelief' while the key in the dictionary is 'nasagibs.aster_gdem_greyscale_shaded_relief' (with underscore).

Write docs

Explaining:

User land
Architechture to expand for developers

2022.6.0: sphinx warnings

First of all currently it is not possible to use straight sphinx-build command to build documentation out of source tree

+ /usr/bin/sphinx-build -n -T -b man doc/source build/sphinx/man
Running Sphinx v5.0.2

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 343, in eval_config_file
    exec(code, namespace)
  File "/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/doc/source/conf.py", line 16, in <module>
    import xyzservices  # noqa
ModuleNotFoundError: No module named 'xyzservices'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/cmd/build.py", line 272, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
  File "/usr/lib/python3.8/site-packages/sphinx/application.py", line 202, in __init__
    self.config = Config.read(self.confdir, confoverrides or {}, self.tags)
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 165, in read
    namespace = eval_config_file(filename, tags)
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 356, in eval_config_file
    raise ConfigError(msg % traceback.format_exc()) from exc
sphinx.errors.ConfigError: There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 343, in eval_config_file
    exec(code, namespace)
  File "/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/doc/source/conf.py", line 16, in <module>
    import xyzservices  # noqa
ModuleNotFoundError: No module named 'xyzservices'


Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/sphinx/config.py", line 343, in eval_config_file
    exec(code, namespace)
  File "/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/doc/source/conf.py", line 16, in <module>
    import xyzservices  # noqa
ModuleNotFoundError: No module named 'xyzservices'

This can be fixed by patch like below:

--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -10,9 +10,9 @@
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 #
-# import os
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+import os
+import sys
+sys.path.insert(0, os.path.abspath("../.."))
 import xyzservices  # noqa

 # -- Project information -----------------------------------------------------

This patch fixes what is in the comment and that can of fix is suggested in sphinx example copy.py https://www.sphinx-doc.org/en/master/usage/configuration.html#example-of-configuration-file

Here is the output with warnings:

+ /usr/bin/sphinx-build -n -T -b man doc/source build/sphinx/man
Running Sphinx v5.0.2
making output directory... done
[autosummary] generating autosummary for: api.rst, changelog.rst, contributing.md, index.md, introduction.ipynb, registration.md
myst v0.17.2: MdParserConfig(commonmark_only=False, gfm_only=False, enable_extensions=[], linkify_fuzzy_links=True, dmath_allow_labels=True, dmath_allow_space=True, dmath_allow_digits=True, dmath_double_inline=False, update_mathjax=True, mathjax_classes='tex2jax_process|mathjax_process|math|output_area', disable_syntax=[], all_links_external=False, url_schemes=('http', 'https', 'mailto', 'ftp'), ref_domains=None, highlight_code_blocks=True, number_code_blocks=[], title_to_header=False, heading_anchors=None, heading_slug_func=None, footnote_transition=True, sub_delimiters=('{', '}'), words_per_minute=200)
WARNING: 'jupyter_execute_notebooks' is deprecated for 'nb_execution_mode' [mystnb.config]
myst-nb v0.15.0: NbParserConfig(custom_formats={}, metadata_key='mystnb', cell_metadata_key='mystnb', kernel_rgx_aliases={}, execution_mode='force', execution_cache_path='', execution_excludepatterns=(), execution_timeout=30, execution_in_temp=False, execution_allow_errors=False, execution_raise_on_error=False, execution_show_tb=False, merge_streams=False, render_plugin='default', remove_code_source=False, remove_code_outputs=False, number_source_lines=False, output_stderr='show', render_text_lexer='myst-ansi', render_error_lexer='ipythontb', render_image_options={}, render_figure_options={}, render_markdown_format='commonmark', output_folder='build', append_css=True, metadata_to_fm=False)
Using jupyter-cache at: /home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/build/sphinx/.jupyter_cache
building [mo]: targets for 0 po files that are out of date
building [man]: all manpages
updating environment: [new config] 6 added, 0 changed, 0 removed
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/doc/source/introduction.ipynb: Executing notebook using local CWD [mystnb]
reading sources... [100%] registration
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/xyzservices/lib.py:docstring of xyzservices.lib.TileProvider:73: WARNING: autosummary: stub file not found 'xyzservices.TileProvider.build_url'. Check your autosummary_generate setting.
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/xyzservices/lib.py:docstring of xyzservices.lib.TileProvider:73: WARNING: autosummary: stub file not found 'xyzservices.TileProvider.from_qms'. Check your autosummary_generate setting.
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/xyzservices/lib.py:docstring of xyzservices.lib.TileProvider:73: WARNING: autosummary: stub file not found 'xyzservices.TileProvider.requires_token'. Check your autosummary_generate setting.
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/xyzservices/lib.py:docstring of xyzservices.lib.Bunch:38: WARNING: autosummary: stub file not found 'xyzservices.Bunch.filter'. Check your autosummary_generate setting.
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/xyzservices/lib.py:docstring of xyzservices.lib.Bunch:38: WARNING: autosummary: stub file not found 'xyzservices.Bunch.flatten'. Check your autosummary_generate setting.
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/xyzservices/lib.py:docstring of xyzservices.lib.Bunch:38: WARNING: autosummary: stub file not found 'xyzservices.Bunch.query_name'. Check your autosummary_generate setting.
CHANGELOG.md:28: WARNING: Title underline too short.

xyzservices 2022.02.0 (February 10, 2022)
----------------------------------------
CHANGELOG.md:28: WARNING: Title underline too short.

xyzservices 2022.02.0 (February 10, 2022)
----------------------------------------
CHANGELOG.md:51: WARNING: Title underline too short.

xyzservices 2021.11.0 (November 06, 2021)
----------------------------------------
CHANGELOG.md:51: WARNING: Title underline too short.

xyzservices 2021.11.0 (November 06, 2021)
----------------------------------------
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/doc/source/introduction.ipynb: WARNING: Executing notebook failed: CellExecutionError [mystnb.exec]
/home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/doc/source/introduction.ipynb: WARNING: Notebook exception traceback saved in: /home/tkloczko/rpmbuild/BUILD/xyzservices-2022.6.0/build/sphinx/man/reports/introduction.err.log [mystnb.exec]
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
writing... python-xyzservices.3 { introduction registration api contributing changelog } done
build succeeded, 13 warnings.

What would the process be for adding a new tile service like NowCoast

https://new.nowcoast.noaa.gov/help/#!section=rest-usage

It would unlock a lot, a lot of new tiles (see list @ https://new.nowcoast.noaa.gov/help/#!section=rest-layer-ids)

Create GHA to ensure providers JSON are up to date

We should create a GHA which would periodically (weekly?) fetch the list of providers from leaflet and compress our providers.json.

ENH: add `flatten` method on `Bunch`

It would be great if xyzservices can add a function that returns a non-nested dictionary of all tiles, similar to the get_xyz_dict function I implemented in leafmap. My implementation is probably not optimal. https://github.com/giswqs/leafmap/blob/master/leafmap/basemaps.py#L206

From @giswqs https://twitter.com/giswqs/status/1423265380236201984

I guess that a method Bunch.flatten() returning a flat dict would be a good one. Thanks for the suggestion!

Add method to build full URL

This would be in lib and would build the full URL, including tokens, etc. for a TileProvider object.

Add JSON compress on GA

The GA that runs periodically to pull sources from Leaftlet and merge them with our community-contributed ones should also generate a compressed JSON and place it under xyzservices/data.

Add Official Switzerland provider wmts Geo.Admin.ch

Via Geo.Admin.ch a WMTS which is Opendata can be used to be added as provider
https://api3.geo.admin.ch/services/sdiservices.html#wmts

Glad to help if needed

JSON loader

We need a method to load a JSON file with providers and return either a new Bunch or append the JSON content to an existing Bunch. Behaviour could be:

import xyzservices.providers as xyz
import xyzservices

# Either
new_providers = xyzservices.load_json("my_list.json")

# Or:
xyz.load_json("my_list.json")

This should support both local and remote JSON files.

Supporting MVT-like vector tiles?

I wonder if vector tiles are already on the to-do list? I see them only briefly mentioned in #49. I have some experience in this context which can be summarized like: it's kind of easy to get the data, but kind of difficult to render it with some proper style, partly because there are multiple standards.

may it support baidu map?

Add new providers in session

Develop a method within a Bunch that can append new TileProvider instances to a loaded set of providers. It should work something like:

import xyzservices.providers as xyz

xyz.add_provider({'url': 'http://myservice/{x}/{y}/{z}'})

Mandatory attributes

name, url, and attribution should be mandatory attributes of TileProvider and we should raise otherwise.

Tests are installed incorrectly

The sdist correctly includes tests. However, if you build and install from the sdist, the tests are installed in .../site-packages/tests/. This is at the top-level and the wrong place for such a generic package name. If installed, they should be in the xyzservices directory, but probably you don't want to be installing them at all.

Gallery of tiles

It would be nice to have a gallery similar to https://ipyleaflet.readthedocs.io/en/latest/api_reference/basemaps.html. It can be used to check tiles (xref #59) and would provide a nice overview to users. Ideally automatically generated.

Provider sources model

This issue describes the model for managing sources.

Currently, we envision at least two sources of providers:

Automatically scraped from Leaflet providers: these is parsed with parse_leaflet_providers.py and stored under provider_sources/leaflet-providers-parsed.json
Community contributed additional providers: these are sources provided independently from the official Leaflet list by the community (following guidelines detailed in the docs). They get contributed to provider_sources/community_providers.json, which is a file that should be stored in prettified JSON format. Community contributions come as PRs against that file.

In terms of what gets shipped with the package that can be installed, one idea is to write all the sources into a single, compressed JSON file that ships with the package and loads on import. That way, neither the scraped Leaflet, nor the community, pretty JSON files would be shipped, just the merged and compressed one.

To implement this, we will need:

Tooling to take all JSON files in provider_sources and merge/compress into a single provider_sources.json
Load providers on import only from provider_sources.json

BUG: html repr not legible in dark themes

Our HTML repr works only in light IDE themes like the default JLab. Dark mode makes it illegible.

Test tiles

As I have mentioned in another issue (#22), we should write tests fetching the actual tiles to ensure they work. There are a few complications along the way:

some need API keys (I'd just skip those and rely on reports from people using them)
some work only when requested from within a web browser (e.g. Stadia)

NLS testing

NLS now requires API key which is not propagated from upstream and requires setting up a key. Need to wait for leaflet-extras/leaflet-providers#547 to ensure we get correct definition and then set it up on our side.

Add `search` or `filter` method

We have over 200 providers at this moment and it may be helpful to have a function that is able to filter providers based on some criteria. I would imagine keywords (e.g. I want all HERE maps) or specs (only those that do not need a token).

API-wise, maybe something like:

Bunch.filter(keyword=None, name=None, requires_token=None...) # maybe some other (zoom, bounds?)

It would return a smaller Bunch or an individual tile provider if name==TileProvider.name.

Add method to check if token is required

Some providers require an access token to get the tiles. It could be handy to have a method in TileProvider checking this requirement and returning boolean. Now you have to check if the accessToken key is present. Something like

>>> xyz.MapBox.requires_token()
True

>>> xyz.Stamen.Toner.requires_token()
False

Allow keeping `{s}` untouched in `build_url`

I've noticed that ipyleaflet URLs still have {s} placeholder in their URLs. With TileProvider.build_url() we currently fill that with a. It may be good to allow a bit more customisation of that, from passing the custom server to leaving it as it is.

Add Esri World Dark Gray Canvas basemap

Moving contextily#181 over here to integrate it with community-contributed providers.

Adds the Esri World Dark Gray Canvas basemap to the provider list.

See:

https://www.arcgis.com/home/item.html?id=1970c1995b8f44749f4b9b6e81b5ba45

https://server.arcgisonline.com/arcgis/rest/services/Canvas/World_Dark_Gray_Base/MapServer

cc' @EwoutH

Add Google basemaps

It would be great to add Google basemaps to providers.json.

Google Roadmap: https://mt1.google.com/vt/lyrs=m&x={x}&y={y}&z={z}
Google Terrain: https://mt1.google.com/vt/lyrs=p&x={x}&y={y}&z={z}
Google Satellite: https://mt1.google.com/vt/lyrs=s&x={x}&y={y}&z={z}
Google Hybrid: https://mt1.google.com/vt/lyrs=y&x={x}&y={y}&z={z}

Use background color to pretty print of `TileProvider`

Currently, we print a provider in a bunch with the same background:

It'd be nice to use a different background color (e.g. light gray) for the space where the provider is printed, to avoid the reader continuing reading into the next provider

Add maptiler's missing layers

Some of the maptiler's layers are not included, like the new cloud-free sentinel 2. See https://cloud.maptiler.com/tiles/.

Add Stadia tiles

Leaflet has a few Stadia tiles that don't seem to work by passing the URL to Contextily.

http://leaflet-extras.github.io/leaflet-providers/preview/#filter=Stadia.AlidadeSmooth
http://leaflet-extras.github.io/leaflet-providers/preview/#filter=Stadia.AlidadeSmoothDark
http://leaflet-extras.github.io/leaflet-providers/preview/#filter=Stadia.OSMBright
http://leaflet-extras.github.io/leaflet-providers/preview/#filter=Stadia.Outdoors

Not sure what the incompatibility is. I get PIL.UnidentifiedImageError: cannot identify image file <_io.BytesIO object at 0x000001E410D79630> when trying the URLs in bounds2img.

TST: add tests for callable TileProvider

The TileProvider is callable so you can update its attributes (eg to specify a key or token, or to override the attribution text) by calling the object (docs: https://contextily.readthedocs.io/en/latest/providers_deepdive.html#Specifying-options-for-a-provider)

We should add tests for this functionality here. A test in contextily: https://github.com/geopandas/contextily/blob/799c70802892f0276ace74159df3ed24779cedbf/tests/test_providers.py#L56-L64

OpenFireMap tiles are broken

Our CI is now failing as OpenFireMap tiles are apparently broken. http://www.openfiremap.de also fails as shown on the screenshot below. Will give them some time but if nothing changes, will mark them as broken and report to upstream.

Roadmap for 0.1

Document the ecosystem

We should add a page to our docs mapping the adoption of xyzservices across the ecosystem. Who uses it, how can you do that... Could be done as a follow-up on #49 but we can already start as there are packages supporting the input at the moment (at least leafmap, here widget and bokeh).

NASAGIBS BlueMarble3413

In ipyleaflet we used to have a NASAGIBS.BlueMarble3413 basemap, see e.g. this notebook. I see xyzservices doesn't provide it, would it make sense to add it?

Gallery doesn't render on RTD

I have merged #122 and it somehow broke the rendering of the gallery on the RTD. Which is strange, because the build made from that PR worked.

https://xyzservices--122.org.readthedocs.build/en/122/gallery.html

https://xyzservices.readthedocs.io/en/latest/gallery.html

It seems that the JS loading the data actually works, but the CSS and JS calling leaflet are not.

Failing test

One of the tests is failing for me. I'm testing xyzservices 2022.9.0

Here is the test output.

$ python3 -mpytest -x -v
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.10.7, pytest-7.1.2, pluggy-1.0.0+repack -- /usr/bin/python3
cachedir: .pytest_cache
benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/edward/src/debian/python-modules/xyzservices/.hypothesis/examples')
rootdir: /home/edward/src/debian/python-modules/xyzservices, configfile: pytest.ini
plugins: benchmark-3.2.2, astropy-header-0.2.2, forked-1.4.0, flaky-3.7.0, hypothesis-6.36.0, doctestplus-0.12.1, kgb-7.1.1, repeat-0.9.1, django-4.5.2, timeout-2.1.0, anyio-3.6.1, pylama-7.4.3, cov-4.0.0, tornasync-0.6.0.post2, remotedata-0.3.3, mock-3.8.2, requests-mock-1.9.3, xdist-2.5.0, asyncio-0.19.0
asyncio: mode=strict
collected 451 items                                                                                                                                                                         

xyzservices/tests/test_lib.py::test_expect_name_url_attribution PASSED                                                                                                                [  0%]
xyzservices/tests/test_lib.py::test_build_url PASSED                                                                                                                                  [  0%]
xyzservices/tests/test_lib.py::test_requires_token PASSED                                                                                                                             [  0%]
xyzservices/tests/test_lib.py::test_html_repr PASSED                                                                                                                                  [  0%]
xyzservices/tests/test_lib.py::test_copy PASSED                                                                                                                                       [  1%]
xyzservices/tests/test_lib.py::test_callable FAILED                                                                                                                                   [  1%]

========================================================================================= FAILURES ==========================================================================================
_______________________________________________________________________________________ test_callable _______________________________________________________________________________________

    def test_callable():
        # only testing the callable functionality to override a keyword, as we
        # cannot test the actual providers that need an API key
        updated_provider = xyz.GeoportailFrance.plan(apikey="mykey")
        assert isinstance(updated_provider, TileProvider)
        assert "url" in updated_provider
        assert updated_provider["apikey"] == "mykey"
        # check that original provider dict is not modified
>       assert xyz.GeoportailFrance.plan["apikey"] == "essentiels"
E       AssertionError: assert 'choisirgeoportail' == 'essentiels'
E         - essentiels
E         + choisirgeoportail

xyzservices/tests/test_lib.py:189: AssertionError
================================================================================== short test summary info ==================================================================================
FAILED xyzservices/tests/test_lib.py::test_callable - AssertionError: assert 'choisirgeoportail' == 'essentiels'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
================================================================================ 1 failed, 5 passed in 0.34s ================================================================================
$

IGN (cx.providers.GeoportailFrance.orthos) migrated their services (wxs.ign.fr to data.geopf.fr)

IGN just changed their routing this month (March 2024).
https://geoservices.ign.fr/services-geoplateforme-diffusion

try:
    cx.add_basemap(
        m.ax,
        source=cx.providers.GeoportailFrance.orthos,
        attribution=True,
        crs=Maps.CRS.GOOGLE_MERCATOR,
        zoom=zoom_level,
    )
except HTTPError as e:
    logger.error("Failed to load basemap: %s", e)
    logger.warning("Continuing without basemap.")

Versioning model

I was thinking about versioning and I'd suggest we adopt some version of calendar versioning instead of semantic versioning we use elsewhere. Because once stable, I assume that most of the releases will be just updates of available providers and putting a date on the release makes much more sense than a number.

I was thinking about following dask model with YYYY.MM.MINOR, so if we'd make a release now, it would be 2021.05.0.

In practice, we'll just need to use this schema when creating tags, nothing else.

BUG: package is broken for latest commit

6740b22 seems to have broken xyzservices on import. I get this traceback running pytest . locally on main:

xyzservices\__init__.py:2: in <module>
    from .providers import providers  # noqa
xyzservices\providers.py:15: in <module>
    providers = _load_json(json)
xyzservices\lib.py:600: in _load_json
    {i: TileProvider(provider[i]) for i in provider}
xyzservices\lib.py:600: in <dictcomp>
    {i: TileProvider(provider[i]) for i in provider}
xyzservices\lib.py:358: in __init__
    super().__init__(*args, **kwargs)
E   ValueError: dictionary update sequence element #0 has length 1; 2 is required

or see https://github.com/geopandas/geopandas/actions/runs/7770406362/job/21190571888 downstream.

I'm not at all familiar with how xyzservices works, but from a quick glance it looks to me to do with the fact that the name "NLS" changed to "NLS.oslondon1k1893" and now that doesn't match with "NLS" in BROKEN_PROVIDERS?

(Also it doesn't seem like tests are running on CI - otherwise this would have shown up, I don't know if that's intentional or not)

Stadia and Stamen map tiles

Are there any plans to support the Stamen map tiles that have moved to Stadia?

Write tests

Aiming always for 100% coverage :-)

To add yandex map services ( present in mercator projection by default)

It would be helpful to have the yandex map services in the providers.json.

  Yandex maps: http://vec0{1-4}.maps.yandex.net/tiles?l=map&x={x}&y={y}&z={z}

OpenFireMap is down

OpenFireMap tiles are throwing 404 errors at this moment. Not sure if they're down temporarily or for good. Their twitter is dead for 8 years and I am not sure how to report an issue.

Exclude broken sources from leaflet-providers

We currently create leaflet-providers-parsed.json automatically but it comes with some providers which are not working anymore (e.g. #22). These should not be available in xyzservices but we should not manually delete them from the JSON because such a change would break with every update (which will be later automatised using GHA).

We should find a clever way to mark broken resources and exclude them during compression of source files to the final one shipped with the package. One way could be a simple list of keys in _compress_providers.py (once #23 is in) which we can pop out of the dict. Might be the easiest one.

Add example of integration with Quick Map Services

Quick Map Services (QMS) is an open catalog of geodata sources since 2016 with over 1500 services (700+ are XYZ), more details here. It also provides REST API to consume data from it. I think it would be useful to have an example of integration between xyzservice and QMS. Something like this might be added to the documentation:

import folium
import requests
from xyzservices.lib import TileProvider

QMS_API = "https://qms.nextgis.com/api/v1/geoservices"

def get_service_details(service_name):
    def get_service_id(service_name):
        services = requests.get(f"{QMS_API}/?search={service_name}&type=tms&epsg=3857&limit=1")
        services = services.json()
        if services["results"]:
            return services["results"][0]["id"]
    service_id = get_service_id(service_name)
    service_details = requests.get(f"{QMS_API}/{service_id}")
    return service_details.json()

service_details = get_service_details("OpenTopoMap")

tiles = TileProvider(
    name=service_details["name"],
    url=service_details["url"],
    attribution=service_details["copyright_text"]
)

folium.Map(
    location=[53.4108, -2.9358],
    tiles=tiles.build_url(),
    attr=tiles.attribution,
)

It also might be used as a solution for #63:

service_details = get_service_details("Google Satellite")

google = TileProvider(
    name=service_details["name"],
    url=service_details["url"],
    attribution="dummy"
)

folium.Map(
    location=[53.4108, -2.9358],
    tiles=google.build_url(),
    attr=google.attribution,
)

Land Information New Zealand XYZ maps

Land Information New Zealand has a free and openly licensed XYZ service (requires a developer key) that could be added here. If you are interested in country specific services.

https://basemaps.linz.govt.nz

Github: https://github.com/linz/basemaps

Outline community contribution process

Populating CONTRIBUTING.md

Hydda basemaps

We have been having issues with Hydda basemaps (at least Full and Base) for the last year or so, the server seems to be definitely down.
Looking at the ipyleaflet documentation, MtbMap seems to be down too (and Strava but it is not listed in the providers here).

HTML attribution

We currently store attribution as a basic string only. I've noticed that interactive packages like ipyleaflet tend to add HTML links to OpenStreetMap and others. It might be good to add a support of html_attribution which would return HTML string with proper links.

Consolidate provider sources

Hello, ipyleaflet developer here.

I noticed that JSON listing the provider sources is generated by _parse_leaflet_providers.py which extracts the data from https://github.com/leaflet-extras/leaflet-providers with Selenium. This seems a bit overkill. Did anyone try reaching out to the developers of leaflet-providers so that this information is directly available in a standalone JSON file?

This JSON file could also directly be packaged independently of the rest of the JS logic of leaflet-providers.

cc @HaudinFlorence.