preligens-lab / buzzard Goto Github PK
View Code? Open in Web Editor NEWAdvanced raster and geometry manipulations
License: Apache License 2.0
buzzard's People
Contributors
Stargazers
Watchers
buzzard's Issues
Outdated recommendation?
Is the recommendation to use python >= 3.6.7 still relevant? It dates from Nov 7, 2018 but the changes introduced in https://bugs.python.org/issue34172 were reverted on Dec 6, 2018.
Flaky test: `buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles0]`
#!/bin/bash -eo pipefail
. venvs/37/bin/activate
python scripts/pytest_parallel.py -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage.xml --junitxml=test-reports/pytest-report.xml
ls -l test-reports
# codecov first concatenate the coverage*.xml files and upload it
# it does not seem to care about the pytest-report*.xml, is it a problem?
# https://circleci.com/gh/airware/buzzard/1564#queue-placeholder/containers/0
# It used to upload those.
# https://circleci.com/gh/airware/buzzard/1506#artifacts/containers/0
codecov
-- Discovering tests --
$ bash -c "pytest --collect-only -x &>/tmp/pytest-collection-tmp"
bash -c "pytest --collect-only -x &>/tmp/pytest-collection-tmp" (took 10.3sec)
Found 17520 tests scattered on 20 files
buzzard/test/test_cached_raster_recipe.py -> (12 calls of 1 test)
buzzard/test/test_dataset_activations.py -> (1 call of 3 tests)
buzzard/test/test_dataset_modes.py -> (1 call of 4 tests)
buzzard/test/test_dataset_registering.py -> (1 call of 2 tests)
buzzard/test/test_env.py -> (1 call of 1 tests)
buzzard/test/test_footprint.py -> (1 call of 17 tests)
buzzard/test/test_footprint_convs.py -> (1 call of 576 tests)
buzzard/test/test_footprint_findburn_polygons.py -> (1 call of 51 tests)
buzzard/test/test_footprint_intersection.py -> (1 call of 4 tests)
buzzard/test/test_footprint_move.py -> (1 call of 504 tests)
buzzard/test/test_footprint_precision.py -> (1 call of 36 tests)
buzzard/test/test_footprint_tile.py -> (1 call of 1638 tests)
buzzard/test/test_footprint_tile_count.py -> (1 call of 5121 tests)
buzzard/test/test_footprint_tile_occurrence.py -> (1 call of 2177 tests)
buzzard/test/test_multi_ordered_dict.py -> (1 call of 1 tests)
buzzard/test/test_rastersource_getsetdata_basic.py -> (1 call of 192 tests)
buzzard/test/test_rastersource_opencreate.py -> (1 call of 29 tests)
buzzard/test/test_rastersource_resampling.py -> (1 call of 6960 tests)
buzzard/test/test_vectorsource_getsetdata_general.py -> (6 calls of 1 test)
buzzard/test/test_vectorsource_opencreate.py -> (1 call of 186 tests)
-- Running tests, 36 calls to pytest, 36 simulateneous --
$ COVERAGE_FILE=.coverage.e996de86-b2e3-4f8b-8cb3-202ff0fae316 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-e996de86-b2e3-4f8b-8cb3-202ff0fae316.xml --junitxml=test-reports/pytest-report-e996de86-b2e3-4f8b-8cb3-202ff0fae316.xml 'buzzard/test/test_vectorsource_opencreate.py' &>/tmp/e996de86-b2e3-4f8b-8cb3-202ff0fae316"
$ COVERAGE_FILE=.coverage.7fd395b1-60c6-440d-b1a9-37fe22e8ee5d bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-7fd395b1-60c6-440d-b1a9-37fe22e8ee5d.xml --junitxml=test-reports/pytest-report-7fd395b1-60c6-440d-b1a9-37fe22e8ee5d.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_value_error[ESRI Shapefile-.shp]' &>/tmp/7fd395b1-60c6-440d-b1a9-37fe22e8ee5d"
$ COVERAGE_FILE=.coverage.2bf08196-ca6c-437c-9dab-9b081da6c4be bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-2bf08196-ca6c-437c-9dab-9b081da6c4be.xml --junitxml=test-reports/pytest-report-2bf08196-ca6c-437c-9dab-9b081da6c4be.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[Memory--True-True]' &>/tmp/2bf08196-ca6c-437c-9dab-9b081da6c4be"
$ COVERAGE_FILE=.coverage.669d967d-182c-406b-a3e3-9d7353a81350 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-669d967d-182c-406b-a3e3-9d7353a81350.xml --junitxml=test-reports/pytest-report-669d967d-182c-406b-a3e3-9d7353a81350.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[GeoJson-.json-True-True]' &>/tmp/669d967d-182c-406b-a3e3-9d7353a81350"
$ COVERAGE_FILE=.coverage.5dd205ee-0652-4af7-a28c-86e1ad2487cf bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-5dd205ee-0652-4af7-a28c-86e1ad2487cf.xml --junitxml=test-reports/pytest-report-5dd205ee-0652-4af7-a28c-86e1ad2487cf.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[ESRI Shapefile-.shp-True-True]' &>/tmp/5dd205ee-0652-4af7-a28c-86e1ad2487cf"
$ COVERAGE_FILE=.coverage.56a7abb0-7e96-4a49-a0d4-fbf1d1d21711 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-56a7abb0-7e96-4a49-a0d4-fbf1d1d21711.xml --junitxml=test-reports/pytest-report-56a7abb0-7e96-4a49-a0d4-fbf1d1d21711.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_iter_data_fields_behavior[ESRI Shapefile-.shp]' &>/tmp/56a7abb0-7e96-4a49-a0d4-fbf1d1d21711"
$ COVERAGE_FILE=.coverage.9423a3b0-b47d-42b0-924f-a6dd211d2a42 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-9423a3b0-b47d-42b0-924f-a6dd211d2a42.xml --junitxml=test-reports/pytest-report-9423a3b0-b47d-42b0-924f-a6dd211d2a42.xml 'buzzard/test/test_rastersource_opencreate.py' &>/tmp/9423a3b0-b47d-42b0-924f-a6dd211d2a42"
$ COVERAGE_FILE=.coverage.eb813f53-5f93-4c12-86e9-84cd5f500a6b bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-eb813f53-5f93-4c12-86e9-84cd5f500a6b.xml --junitxml=test-reports/pytest-report-eb813f53-5f93-4c12-86e9-84cd5f500a6b.xml 'buzzard/test/test_multi_ordered_dict.py' &>/tmp/eb813f53-5f93-4c12-86e9-84cd5f500a6b"
$ COVERAGE_FILE=.coverage.9a72611b-0f72-4ebf-a445-eec0dedf0a23 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-9a72611b-0f72-4ebf-a445-eec0dedf0a23.xml --junitxml=test-reports/pytest-report-9a72611b-0f72-4ebf-a445-eec0dedf0a23.xml 'buzzard/test/test_footprint_tile_occurrence.py' &>/tmp/9a72611b-0f72-4ebf-a445-eec0dedf0a23"
$ COVERAGE_FILE=.coverage.e72af6be-f9c9-4eee-adb0-344f319fbaa1 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-e72af6be-f9c9-4eee-adb0-344f319fbaa1.xml --junitxml=test-reports/pytest-report-e72af6be-f9c9-4eee-adb0-344f319fbaa1.xml 'buzzard/test/test_footprint_tile.py' &>/tmp/e72af6be-f9c9-4eee-adb0-344f319fbaa1"
$ COVERAGE_FILE=.coverage.35fd1b49-9896-4d08-899b-744a58ff293a bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-35fd1b49-9896-4d08-899b-744a58ff293a.xml --junitxml=test-reports/pytest-report-35fd1b49-9896-4d08-899b-744a58ff293a.xml 'buzzard/test/test_rastersource_resampling.py' &>/tmp/35fd1b49-9896-4d08-899b-744a58ff293a"
$ COVERAGE_FILE=.coverage.72a30369-6698-4cf0-8d16-9266705a24ea bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-72a30369-6698-4cf0-8d16-9266705a24ea.xml --junitxml=test-reports/pytest-report-72a30369-6698-4cf0-8d16-9266705a24ea.xml 'buzzard/test/test_footprint_intersection.py' &>/tmp/72a30369-6698-4cf0-8d16-9266705a24ea"
$ COVERAGE_FILE=.coverage.b592e168-f550-45da-85df-005672b3598e bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-b592e168-f550-45da-85df-005672b3598e.xml --junitxml=test-reports/pytest-report-b592e168-f550-45da-85df-005672b3598e.xml 'buzzard/test/test_footprint_findburn_polygons.py' &>/tmp/b592e168-f550-45da-85df-005672b3598e"
$ COVERAGE_FILE=.coverage.63b69eed-173f-4a91-aee0-af7730c71962 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-63b69eed-173f-4a91-aee0-af7730c71962.xml --junitxml=test-reports/pytest-report-63b69eed-173f-4a91-aee0-af7730c71962.xml 'buzzard/test/test_footprint_convs.py' &>/tmp/63b69eed-173f-4a91-aee0-af7730c71962"
$ COVERAGE_FILE=.coverage.87a325ae-9ea5-4a90-8934-5e95fd15d177 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-87a325ae-9ea5-4a90-8934-5e95fd15d177.xml --junitxml=test-reports/pytest-report-87a325ae-9ea5-4a90-8934-5e95fd15d177.xml 'buzzard/test/test_footprint_move.py' &>/tmp/87a325ae-9ea5-4a90-8934-5e95fd15d177"
$ COVERAGE_FILE=.coverage.96a9823b-fc40-4106-8284-dac00b8fae9b bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-96a9823b-fc40-4106-8284-dac00b8fae9b.xml --junitxml=test-reports/pytest-report-96a9823b-fc40-4106-8284-dac00b8fae9b.xml 'buzzard/test/test_dataset_registering.py' &>/tmp/96a9823b-fc40-4106-8284-dac00b8fae9b"
$ COVERAGE_FILE=.coverage.8af50906-06ef-4c1c-95de-22c31854f572 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-8af50906-06ef-4c1c-95de-22c31854f572.xml --junitxml=test-reports/pytest-report-8af50906-06ef-4c1c-95de-22c31854f572.xml 'buzzard/test/test_dataset_modes.py' &>/tmp/8af50906-06ef-4c1c-95de-22c31854f572"
$ COVERAGE_FILE=.coverage.f3c43c8a-56fd-4e2a-a7c3-14445e0693cb bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-f3c43c8a-56fd-4e2a-a7c3-14445e0693cb.xml --junitxml=test-reports/pytest-report-f3c43c8a-56fd-4e2a-a7c3-14445e0693cb.xml 'buzzard/test/test_dataset_activations.py' &>/tmp/f3c43c8a-56fd-4e2a-a7c3-14445e0693cb"
$ COVERAGE_FILE=.coverage.07cc5e29-1459-4ef7-8b2a-747cb57da700 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-07cc5e29-1459-4ef7-8b2a-747cb57da700.xml --junitxml=test-reports/pytest-report-07cc5e29-1459-4ef7-8b2a-747cb57da700.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools3-cache_tiles2]' &>/tmp/07cc5e29-1459-4ef7-8b2a-747cb57da700"
$ COVERAGE_FILE=.coverage.35342788-06ad-4b51-a456-4417168a30ff bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-35342788-06ad-4b51-a456-4417168a30ff.xml --junitxml=test-reports/pytest-report-35342788-06ad-4b51-a456-4417168a30ff.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools3-cache_tiles0]' &>/tmp/35342788-06ad-4b51-a456-4417168a30ff"
$ COVERAGE_FILE=.coverage.268dcf02-8d38-4e22-88d2-d6028d8278e5 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-268dcf02-8d38-4e22-88d2-d6028d8278e5.xml --junitxml=test-reports/pytest-report-268dcf02-8d38-4e22-88d2-d6028d8278e5.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[DXF-.dxf-False-False]' &>/tmp/268dcf02-8d38-4e22-88d2-d6028d8278e5"
$ COVERAGE_FILE=.coverage.a69df320-f5fe-4b7e-ad55-5ad78301b7f5 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-a69df320-f5fe-4b7e-ad55-5ad78301b7f5.xml --junitxml=test-reports/pytest-report-a69df320-f5fe-4b7e-ad55-5ad78301b7f5.xml 'buzzard/test/test_rastersource_getsetdata_basic.py' &>/tmp/a69df320-f5fe-4b7e-ad55-5ad78301b7f5"
$ COVERAGE_FILE=.coverage.6eab878a-d7ff-412f-9411-a30663ae511a bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-6eab878a-d7ff-412f-9411-a30663ae511a.xml --junitxml=test-reports/pytest-report-6eab878a-d7ff-412f-9411-a30663ae511a.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools2-cache_tiles2]' &>/tmp/6eab878a-d7ff-412f-9411-a30663ae511a"
$ COVERAGE_FILE=.coverage.96c59d0b-dd97-4573-a2cb-a540d91b210e bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-96c59d0b-dd97-4573-a2cb-a540d91b210e.xml --junitxml=test-reports/pytest-report-96c59d0b-dd97-4573-a2cb-a540d91b210e.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools2-cache_tiles1]' &>/tmp/96c59d0b-dd97-4573-a2cb-a540d91b210e"
$ COVERAGE_FILE=.coverage.70e1b85b-4880-4e01-a901-530d96440fe6 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-70e1b85b-4880-4e01-a901-530d96440fe6.xml --junitxml=test-reports/pytest-report-70e1b85b-4880-4e01-a901-530d96440fe6.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools2-cache_tiles0]' &>/tmp/70e1b85b-4880-4e01-a901-530d96440fe6"
$ COVERAGE_FILE=.coverage.6fc250c7-8111-4823-9ca3-441f6370f7f6 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-6fc250c7-8111-4823-9ca3-441f6370f7f6.xml --junitxml=test-reports/pytest-report-6fc250c7-8111-4823-9ca3-441f6370f7f6.xml 'buzzard/test/test_footprint_tile_count.py' &>/tmp/6fc250c7-8111-4823-9ca3-441f6370f7f6"
$ COVERAGE_FILE=.coverage.6631d530-1da8-4707-b167-0bd6c9cfd460 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-6631d530-1da8-4707-b167-0bd6c9cfd460.xml --junitxml=test-reports/pytest-report-6631d530-1da8-4707-b167-0bd6c9cfd460.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools3-cache_tiles1]' &>/tmp/6631d530-1da8-4707-b167-0bd6c9cfd460"
$ COVERAGE_FILE=.coverage.c112d267-dfc7-44ca-b4ef-cf2b1f600e05 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-c112d267-dfc7-44ca-b4ef-cf2b1f600e05.xml --junitxml=test-reports/pytest-report-c112d267-dfc7-44ca-b4ef-cf2b1f600e05.xml 'buzzard/test/test_footprint.py' &>/tmp/c112d267-dfc7-44ca-b4ef-cf2b1f600e05"
$ COVERAGE_FILE=.coverage.5c16d734-f536-47f7-bb93-c1f4cfb5536d bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-5c16d734-f536-47f7-bb93-c1f4cfb5536d.xml --junitxml=test-reports/pytest-report-5c16d734-f536-47f7-bb93-c1f4cfb5536d.xml 'buzzard/test/test_footprint_precision.py' &>/tmp/5c16d734-f536-47f7-bb93-c1f4cfb5536d"
$ COVERAGE_FILE=.coverage.ceb18bb4-50a0-4020-9645-e3ac38538b20 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-ceb18bb4-50a0-4020-9645-e3ac38538b20.xml --junitxml=test-reports/pytest-report-ceb18bb4-50a0-4020-9645-e3ac38538b20.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools1-cache_tiles2]' &>/tmp/ceb18bb4-50a0-4020-9645-e3ac38538b20"
$ COVERAGE_FILE=.coverage.2dc7561e-02c5-4a4c-ae4a-19add5cb09f8 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-2dc7561e-02c5-4a4c-ae4a-19add5cb09f8.xml --junitxml=test-reports/pytest-report-2dc7561e-02c5-4a4c-ae4a-19add5cb09f8.xml 'buzzard/test/test_env.py' &>/tmp/2dc7561e-02c5-4a4c-ae4a-19add5cb09f8"
$ COVERAGE_FILE=.coverage.a4a59c9d-fda9-4b9f-be3a-28e72e810a4b bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-a4a59c9d-fda9-4b9f-be3a-28e72e810a4b.xml --junitxml=test-reports/pytest-report-a4a59c9d-fda9-4b9f-be3a-28e72e810a4b.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools1-cache_tiles1]' &>/tmp/a4a59c9d-fda9-4b9f-be3a-28e72e810a4b"
$ COVERAGE_FILE=.coverage.7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c.xml --junitxml=test-reports/pytest-report-7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools1-cache_tiles0]' &>/tmp/7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c"
$ COVERAGE_FILE=.coverage.87ef8482-7c6b-4156-9575-2b4ba31427a0 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-87ef8482-7c6b-4156-9575-2b4ba31427a0.xml --junitxml=test-reports/pytest-report-87ef8482-7c6b-4156-9575-2b4ba31427a0.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles2]' &>/tmp/87ef8482-7c6b-4156-9575-2b4ba31427a0"
$ COVERAGE_FILE=.coverage.b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc.xml --junitxml=test-reports/pytest-report-b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles1]' &>/tmp/b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc"
$ COVERAGE_FILE=.coverage.a75c11b5-3410-47e4-bf79-e3f56806ba96 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml --junitxml=test-reports/pytest-report-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles0]' &>/tmp/a75c11b5-3410-47e4-bf79-e3f56806ba96"
COVERAGE_FILE=.coverage.56a7abb0-7e96-4a49-a0d4-fbf1d1d21711 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-56a7abb0-7e96-4a49-a0d4-fbf1d1d21711.xml --junitxml=test-reports/pytest-report-56a7abb0-7e96-4a49-a0d4-fbf1d1d21711.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_iter_data_fields_behavior[ESRI Shapefile-.shp]' &>/tmp/56a7abb0-7e96-4a49-a0d4-fbf1d1d21711" (took 88.5sec)
COVERAGE_FILE=.coverage.9423a3b0-b47d-42b0-924f-a6dd211d2a42 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-9423a3b0-b47d-42b0-924f-a6dd211d2a42.xml --junitxml=test-reports/pytest-report-9423a3b0-b47d-42b0-924f-a6dd211d2a42.xml 'buzzard/test/test_rastersource_opencreate.py' &>/tmp/9423a3b0-b47d-42b0-924f-a6dd211d2a42" (took 97.3sec)
COVERAGE_FILE=.coverage.96a9823b-fc40-4106-8284-dac00b8fae9b bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-96a9823b-fc40-4106-8284-dac00b8fae9b.xml --junitxml=test-reports/pytest-report-96a9823b-fc40-4106-8284-dac00b8fae9b.xml 'buzzard/test/test_dataset_registering.py' &>/tmp/96a9823b-fc40-4106-8284-dac00b8fae9b" (took 99.7sec)
COVERAGE_FILE=.coverage.2dc7561e-02c5-4a4c-ae4a-19add5cb09f8 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-2dc7561e-02c5-4a4c-ae4a-19add5cb09f8.xml --junitxml=test-reports/pytest-report-2dc7561e-02c5-4a4c-ae4a-19add5cb09f8.xml 'buzzard/test/test_env.py' &>/tmp/2dc7561e-02c5-4a4c-ae4a-19add5cb09f8" (took 99.9sec)
COVERAGE_FILE=.coverage.7fd395b1-60c6-440d-b1a9-37fe22e8ee5d bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-7fd395b1-60c6-440d-b1a9-37fe22e8ee5d.xml --junitxml=test-reports/pytest-report-7fd395b1-60c6-440d-b1a9-37fe22e8ee5d.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_value_error[ESRI Shapefile-.shp]' &>/tmp/7fd395b1-60c6-440d-b1a9-37fe22e8ee5d" (took 100.9sec)
COVERAGE_FILE=.coverage.c112d267-dfc7-44ca-b4ef-cf2b1f600e05 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-c112d267-dfc7-44ca-b4ef-cf2b1f600e05.xml --junitxml=test-reports/pytest-report-c112d267-dfc7-44ca-b4ef-cf2b1f600e05.xml 'buzzard/test/test_footprint.py' &>/tmp/c112d267-dfc7-44ca-b4ef-cf2b1f600e05" (took 102.8sec)
COVERAGE_FILE=.coverage.f3c43c8a-56fd-4e2a-a7c3-14445e0693cb bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-f3c43c8a-56fd-4e2a-a7c3-14445e0693cb.xml --junitxml=test-reports/pytest-report-f3c43c8a-56fd-4e2a-a7c3-14445e0693cb.xml 'buzzard/test/test_dataset_activations.py' &>/tmp/f3c43c8a-56fd-4e2a-a7c3-14445e0693cb" (took 108.2sec)
COVERAGE_FILE=.coverage.b592e168-f550-45da-85df-005672b3598e bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-b592e168-f550-45da-85df-005672b3598e.xml --junitxml=test-reports/pytest-report-b592e168-f550-45da-85df-005672b3598e.xml 'buzzard/test/test_footprint_findburn_polygons.py' &>/tmp/b592e168-f550-45da-85df-005672b3598e" (took 111.3sec)
COVERAGE_FILE=.coverage.72a30369-6698-4cf0-8d16-9266705a24ea bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-72a30369-6698-4cf0-8d16-9266705a24ea.xml --junitxml=test-reports/pytest-report-72a30369-6698-4cf0-8d16-9266705a24ea.xml 'buzzard/test/test_footprint_intersection.py' &>/tmp/72a30369-6698-4cf0-8d16-9266705a24ea" (took 116.5sec)
COVERAGE_FILE=.coverage.8af50906-06ef-4c1c-95de-22c31854f572 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-8af50906-06ef-4c1c-95de-22c31854f572.xml --junitxml=test-reports/pytest-report-8af50906-06ef-4c1c-95de-22c31854f572.xml 'buzzard/test/test_dataset_modes.py' &>/tmp/8af50906-06ef-4c1c-95de-22c31854f572" (took 125.6sec)
COVERAGE_FILE=.coverage.e996de86-b2e3-4f8b-8cb3-202ff0fae316 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-e996de86-b2e3-4f8b-8cb3-202ff0fae316.xml --junitxml=test-reports/pytest-report-e996de86-b2e3-4f8b-8cb3-202ff0fae316.xml 'buzzard/test/test_vectorsource_opencreate.py' &>/tmp/e996de86-b2e3-4f8b-8cb3-202ff0fae316" (took 129.2sec)
COVERAGE_FILE=.coverage.5c16d734-f536-47f7-bb93-c1f4cfb5536d bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-5c16d734-f536-47f7-bb93-c1f4cfb5536d.xml --junitxml=test-reports/pytest-report-5c16d734-f536-47f7-bb93-c1f4cfb5536d.xml 'buzzard/test/test_footprint_precision.py' &>/tmp/5c16d734-f536-47f7-bb93-c1f4cfb5536d" (took 147.2sec)
COVERAGE_FILE=.coverage.a75c11b5-3410-47e4-bf79-e3f56806ba96 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml --junitxml=test-reports/pytest-report-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles0]' &>/tmp/a75c11b5-3410-47e4-bf79-e3f56806ba96" (took 159.1sec)
COVERAGE_FILE=.coverage.70e1b85b-4880-4e01-a901-530d96440fe6 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-70e1b85b-4880-4e01-a901-530d96440fe6.xml --junitxml=test-reports/pytest-report-70e1b85b-4880-4e01-a901-530d96440fe6.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools2-cache_tiles0]' &>/tmp/70e1b85b-4880-4e01-a901-530d96440fe6" (took 168.4sec)
COVERAGE_FILE=.coverage.35342788-06ad-4b51-a456-4417168a30ff bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-35342788-06ad-4b51-a456-4417168a30ff.xml --junitxml=test-reports/pytest-report-35342788-06ad-4b51-a456-4417168a30ff.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools3-cache_tiles0]' &>/tmp/35342788-06ad-4b51-a456-4417168a30ff" (took 178.7sec)
COVERAGE_FILE=.coverage.b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc.xml --junitxml=test-reports/pytest-report-b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles1]' &>/tmp/b36c96f2-cc9a-4b91-ad20-8dd5d8e100cc" (took 185.8sec)
COVERAGE_FILE=.coverage.e72af6be-f9c9-4eee-adb0-344f319fbaa1 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-e72af6be-f9c9-4eee-adb0-344f319fbaa1.xml --junitxml=test-reports/pytest-report-e72af6be-f9c9-4eee-adb0-344f319fbaa1.xml 'buzzard/test/test_footprint_tile.py' &>/tmp/e72af6be-f9c9-4eee-adb0-344f319fbaa1" (took 194.2sec)
COVERAGE_FILE=.coverage.7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c.xml --junitxml=test-reports/pytest-report-7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools1-cache_tiles0]' &>/tmp/7e6c6c0f-099e-4c89-9ecc-1ab976bf2e2c" (took 203.8sec)
COVERAGE_FILE=.coverage.6631d530-1da8-4707-b167-0bd6c9cfd460 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-6631d530-1da8-4707-b167-0bd6c9cfd460.xml --junitxml=test-reports/pytest-report-6631d530-1da8-4707-b167-0bd6c9cfd460.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools3-cache_tiles1]' &>/tmp/6631d530-1da8-4707-b167-0bd6c9cfd460" (took 214.2sec)
COVERAGE_FILE=.coverage.96c59d0b-dd97-4573-a2cb-a540d91b210e bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-96c59d0b-dd97-4573-a2cb-a540d91b210e.xml --junitxml=test-reports/pytest-report-96c59d0b-dd97-4573-a2cb-a540d91b210e.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools2-cache_tiles1]' &>/tmp/96c59d0b-dd97-4573-a2cb-a540d91b210e" (took 227.5sec)
COVERAGE_FILE=.coverage.87a325ae-9ea5-4a90-8934-5e95fd15d177 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-87a325ae-9ea5-4a90-8934-5e95fd15d177.xml --junitxml=test-reports/pytest-report-87a325ae-9ea5-4a90-8934-5e95fd15d177.xml 'buzzard/test/test_footprint_move.py' &>/tmp/87a325ae-9ea5-4a90-8934-5e95fd15d177" (took 228.3sec)
COVERAGE_FILE=.coverage.eb813f53-5f93-4c12-86e9-84cd5f500a6b bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-eb813f53-5f93-4c12-86e9-84cd5f500a6b.xml --junitxml=test-reports/pytest-report-eb813f53-5f93-4c12-86e9-84cd5f500a6b.xml 'buzzard/test/test_multi_ordered_dict.py' &>/tmp/eb813f53-5f93-4c12-86e9-84cd5f500a6b" (took 245.0sec)
COVERAGE_FILE=.coverage.a4a59c9d-fda9-4b9f-be3a-28e72e810a4b bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-a4a59c9d-fda9-4b9f-be3a-28e72e810a4b.xml --junitxml=test-reports/pytest-report-a4a59c9d-fda9-4b9f-be3a-28e72e810a4b.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools1-cache_tiles1]' &>/tmp/a4a59c9d-fda9-4b9f-be3a-28e72e810a4b" (took 245.9sec)
COVERAGE_FILE=.coverage.a69df320-f5fe-4b7e-ad55-5ad78301b7f5 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-a69df320-f5fe-4b7e-ad55-5ad78301b7f5.xml --junitxml=test-reports/pytest-report-a69df320-f5fe-4b7e-ad55-5ad78301b7f5.xml 'buzzard/test/test_rastersource_getsetdata_basic.py' &>/tmp/a69df320-f5fe-4b7e-ad55-5ad78301b7f5" (took 250.4sec)
COVERAGE_FILE=.coverage.87ef8482-7c6b-4156-9575-2b4ba31427a0 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-87ef8482-7c6b-4156-9575-2b4ba31427a0.xml --junitxml=test-reports/pytest-report-87ef8482-7c6b-4156-9575-2b4ba31427a0.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles2]' &>/tmp/87ef8482-7c6b-4156-9575-2b4ba31427a0" (took 276.0sec)
COVERAGE_FILE=.coverage.07cc5e29-1459-4ef7-8b2a-747cb57da700 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-07cc5e29-1459-4ef7-8b2a-747cb57da700.xml --junitxml=test-reports/pytest-report-07cc5e29-1459-4ef7-8b2a-747cb57da700.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools3-cache_tiles2]' &>/tmp/07cc5e29-1459-4ef7-8b2a-747cb57da700" (took 333.9sec)
COVERAGE_FILE=.coverage.6eab878a-d7ff-412f-9411-a30663ae511a bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-6eab878a-d7ff-412f-9411-a30663ae511a.xml --junitxml=test-reports/pytest-report-6eab878a-d7ff-412f-9411-a30663ae511a.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools2-cache_tiles2]' &>/tmp/6eab878a-d7ff-412f-9411-a30663ae511a" (took 388.0sec)
COVERAGE_FILE=.coverage.9a72611b-0f72-4ebf-a445-eec0dedf0a23 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-9a72611b-0f72-4ebf-a445-eec0dedf0a23.xml --junitxml=test-reports/pytest-report-9a72611b-0f72-4ebf-a445-eec0dedf0a23.xml 'buzzard/test/test_footprint_tile_occurrence.py' &>/tmp/9a72611b-0f72-4ebf-a445-eec0dedf0a23" (took 395.6sec)
COVERAGE_FILE=.coverage.6fc250c7-8111-4823-9ca3-441f6370f7f6 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-6fc250c7-8111-4823-9ca3-441f6370f7f6.xml --junitxml=test-reports/pytest-report-6fc250c7-8111-4823-9ca3-441f6370f7f6.xml 'buzzard/test/test_footprint_tile_count.py' &>/tmp/6fc250c7-8111-4823-9ca3-441f6370f7f6" (took 404.9sec)
COVERAGE_FILE=.coverage.2bf08196-ca6c-437c-9dab-9b081da6c4be bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-2bf08196-ca6c-437c-9dab-9b081da6c4be.xml --junitxml=test-reports/pytest-report-2bf08196-ca6c-437c-9dab-9b081da6c4be.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[Memory--True-True]' &>/tmp/2bf08196-ca6c-437c-9dab-9b081da6c4be" (took 407.2sec)
COVERAGE_FILE=.coverage.ceb18bb4-50a0-4020-9645-e3ac38538b20 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-ceb18bb4-50a0-4020-9645-e3ac38538b20.xml --junitxml=test-reports/pytest-report-ceb18bb4-50a0-4020-9645-e3ac38538b20.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools1-cache_tiles2]' &>/tmp/ceb18bb4-50a0-4020-9645-e3ac38538b20" (took 414.6sec)
COVERAGE_FILE=.coverage.5dd205ee-0652-4af7-a28c-86e1ad2487cf bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-5dd205ee-0652-4af7-a28c-86e1ad2487cf.xml --junitxml=test-reports/pytest-report-5dd205ee-0652-4af7-a28c-86e1ad2487cf.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[ESRI Shapefile-.shp-True-True]' &>/tmp/5dd205ee-0652-4af7-a28c-86e1ad2487cf" (took 507.5sec)
COVERAGE_FILE=.coverage.63b69eed-173f-4a91-aee0-af7730c71962 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-63b69eed-173f-4a91-aee0-af7730c71962.xml --junitxml=test-reports/pytest-report-63b69eed-173f-4a91-aee0-af7730c71962.xml 'buzzard/test/test_footprint_convs.py' &>/tmp/63b69eed-173f-4a91-aee0-af7730c71962" (took 510.8sec)
COVERAGE_FILE=.coverage.268dcf02-8d38-4e22-88d2-d6028d8278e5 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-268dcf02-8d38-4e22-88d2-d6028d8278e5.xml --junitxml=test-reports/pytest-report-268dcf02-8d38-4e22-88d2-d6028d8278e5.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[DXF-.dxf-False-False]' &>/tmp/268dcf02-8d38-4e22-88d2-d6028d8278e5" (took 525.8sec)
COVERAGE_FILE=.coverage.35fd1b49-9896-4d08-899b-744a58ff293a bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-35fd1b49-9896-4d08-899b-744a58ff293a.xml --junitxml=test-reports/pytest-report-35fd1b49-9896-4d08-899b-744a58ff293a.xml 'buzzard/test/test_rastersource_resampling.py' &>/tmp/35fd1b49-9896-4d08-899b-744a58ff293a" (took 555.6sec)
COVERAGE_FILE=.coverage.669d967d-182c-406b-a3e3-9d7353a81350 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-669d967d-182c-406b-a3e3-9d7353a81350.xml --junitxml=test-reports/pytest-report-669d967d-182c-406b-a3e3-9d7353a81350.xml 'buzzard/test/test_vectorsource_getsetdata_general.py::test_run[GeoJson-.json-True-True]' &>/tmp/669d967d-182c-406b-a3e3-9d7353a81350" (took 559.2sec)
Traceback (most recent call last):
File "scripts/pytest_parallel.py", line 153, in <module>
list(ex.map(_run_test, tests))
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 598, in result_iterator
yield fs.pop().result()
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 428, in result
return self.__get_result()
File "/usr/local/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/usr/local/lib/python3.7/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "scripts/pytest_parallel.py", line 139, in _run_test
cmd, code, res
Exception: COVERAGE_FILE=.coverage.a75c11b5-3410-47e4-bf79-e3f56806ba96 bash -c "pytest -x --cov=buzzard --cov-report=term --cov-report=xml:test-reports/coverage-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml --junitxml=test-reports/pytest-report-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml 'buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles0]' &>/tmp/a75c11b5-3410-47e4-bf79-e3f56806ba96" failed with code 256
============= output:
============================= test session starts ==============================
platform linux -- Python 3.7.9, pytest-7.1.2, pluggy-1.0.0
rootdir: /tmp/circleci/buzzard
plugins: cov-3.0.0
collected 1 item
buzzard/test/test_cached_raster_recipe.py F [100%]
=================================== FAILURES ===================================
__________________________ test_[pools0-cache_tiles0] __________________________
pools = {'computation': {'computation_pool': None}, 'io': {'io_pool': None}, 'merge': {'merge_pool': None}, 'resample': {'resample_pool': None}}
test_prefix = '/tmp/buzz-ut-5e14f161-6525-46cc-8f29-e2a7857ab0d8'
cache_tiles = (100, 100)
test_prefix2 = '/tmp/buzz-ut-0a07cee9-8e43-4d87-b997-0940cb33a06f'
def test_(pools, test_prefix, cache_tiles, test_prefix2):
def _open(**kwargs):
d = dict(
fp=fp, dtype='float32', channel_count=2,
compute_array=functools.partial(_meshgrid_raster_in, reffp=fp),
cache_dir=test_prefix,
cache_tiles=cache_tiles,
**dict(itertools.chain(
pools['merge'].items(),
pools['resample'].items(),
pools['computation'].items(),
pools['io'].items(),
))
)
d.update(kwargs)
return ds.acreate_cached_raster_recipe(**d)
def _test_get():
arrs = r.get_data(band=-1)
assert arrs.shape == tuple(np.r_[fp.shape, 2])
x, y = arrs[..., 0], arrs[..., 1]
xref, yref = fp.meshgrid_raster
assert np.all(x == xref)
assert np.all(y == yref)
def _test_resampling(fp):
arr = r.get_data(band=-1, fp=fp)
ref = npr.get_data(band=-1, fp=fp)
assert np.allclose(arr, ref)
def _corrupt_files(files):
for path in files:
with open(path, 'wb') as stream:
stream.write(b'42')
print() # debug line
fp = buzz.Footprint(
rsize=(100, 100),
size=(100, 100),
tl=(1000, 1100),
)
compute_same_address_space = (
type(pools['computation']['computation_pool']) in {str, mp.pool.ThreadPool, type(None)}
)
with buzz.Dataset(allow_interpolation=1).close as ds:
# Create a numpy raster with the same data, useful to compare resampling
npr = ds.awrap_numpy_raster(fp, np.stack(fp.meshgrid_raster, axis=2).astype('float32'))
# Test lazyness of cache
r = _open()
files = glob.glob(os.path.join(test_prefix, '*.tif'))
assert len(files) == 0
# Test get_data results
_test_get()
files = glob.glob(os.path.join(test_prefix, '*.tif'))
assert len(files) > 0
mtimes0 = {f: os.stat(f).st_mtime for f in files}
# Test persistence of cache
# Test get_data results
r.close()
r = _open(compute_array=_should_not_be_called)
_test_get()
files = glob.glob(os.path.join(test_prefix, '*.tif'))
assert len(files) > 0
mtimes1 = {f: os.stat(f).st_mtime for f in files}
assert mtimes0 == mtimes1
# Test overwrite parameter
# Test get_data results
r.close()
r = _open(ow=True)
_test_get()
files = glob.glob(os.path.join(test_prefix, '*.tif'))
assert len(files) > 0
mtimes1 = {f: os.stat(f).st_mtime for f in files}
assert mtimes0.keys() == mtimes1.keys()
for k, t0 in mtimes0.items():
t1 = mtimes1[k]
assert t0 < t1
# Test remapping #1 - Interpolation - Fully Inside
fp_within_upscaled = fp.intersection(fp, scale=fp.scale / 2) & fp.erode(fp.rsemiminoraxis // 4)
_test_resampling(fp_within_upscaled)
# Test remapping #2 - Interpolation - Fully Outside
_test_resampling(fp_within_upscaled.move(fp.br + fp.diagvec))
# Test remapping #3 - No Interpolation - Fully Outside
_test_resampling(fp.move(fp.br + fp.diagvec))
# Test remapping #4 - Interpolation - Both in and out
_test_resampling(fp_within_upscaled.move(fp.br - fp_within_upscaled.diagvec / 2))
# Test remapping #5 - No Interpolation - Both in and out
_test_resampling(fp.move(fp.br - fp.pxvec * fp.rsemiminoraxis))
# Test remapping #6 - Interpolation - Fully Inside - Tiled
r.close()
r = _open(max_resampling_size=20)
_test_resampling(fp_within_upscaled)
# Concurrent queries that need a cache file checksum
r.close()
r = _open()
for it in [r.iter_data(fps=[fp], band=-1) for _ in range(10)]:
next(it)
# Concurrent queries that need a cache file missing, all but one computation aborted
# because already launched
r.close()
r = _open(ow=True)
for it in [r.iter_data(fps=[fp], band=-1) for _ in range(10)]:
next(it)
# Query garbage collected
it1 = r.iter_data(fps=[fp] * 2, max_queue_size=1) # 2/2 ready, 1/2 sinked
it2 = r.iter_data(fps=[fp] * 1, max_queue_size=1) # 1/1 ready, 0/1 sinked
it3 = r.iter_data(fps=[fp] * 2, max_queue_size=1) # 1/2 ready, 0/2 sinked
next(it1)
time.sleep(1/2)
del it1, it2, it3
gc.collect()
time.sleep(1 / 2)
r.get_data() # This line will reraise any exception from scheduler
# Raster closing during query
it1 = r.iter_data(fps=[fp] * 2, max_queue_size=1) # 2/2 ready, 1/2 sinked
it2 = r.iter_data(fps=[fp] * 1, max_queue_size=1) # 1/1 ready, 0/1 sinked
it3 = r.iter_data(fps=[fp] * 2, max_queue_size=1) # 1/2 ready, 0/2 sinked
next(it1)
time.sleep(1/2)
# Close Dataset instead of Raster, because Dataset.close is currently blocking
with buzz.Dataset(allow_interpolation=1).close as ds:
npr = ds.awrap_numpy_raster(fp, np.stack(fp.meshgrid_raster, axis=2).astype('float32'))
# Corrupted cache file
files = glob.glob(os.path.join(test_prefix, '*.tif'))
mtimes0 = {f: os.stat(f).st_mtime for f in files}
corrupted_path = files[0]
_corrupt_files([corrupted_path])
r = _open()
r.get_data()
mtimes1 = {f: os.stat(f).st_mtime for f in files}
assert mtimes0.keys() == mtimes1.keys()
for path in files:
if path == corrupted_path:
assert mtimes0[path] != mtimes1[path]
else:
assert mtimes0[path] == mtimes1[path]
with buzz.Dataset(allow_interpolation=1).close as ds:
npr = ds.awrap_numpy_raster(fp, np.stack(fp.meshgrid_raster, axis=2).astype('float32'))
# In iter_data, the first one(s) don't need cache, the next ones need cache file checking and then recomputation
_corrupt_files(glob.glob(os.path.join(test_prefix, '*.tif')))
r = _open()
fps = [
fp.move(fp.br + fp.diagvec), # Outside
] + [fp] * 12
arrs = list(r.iter_data(band=-1, fps=fps))
assert len(arrs) == 13
for tile, arr in zip(fps, arrs):
assert np.all(arr == npr.get_data(band=-1, fp=tile))
with buzz.Dataset(allow_interpolation=1).close as ds:
npr = ds.awrap_numpy_raster(fp, np.stack(fp.meshgrid_raster, axis=2).astype('float32'))
# Test channels order versus numpy raster
r = _open()
for channels in [
0, 1, None, slice(None), [0, 1], [1, 0], [1, 0, 1],
]:
assert np.all(r.get_data(channels=channels) == npr.get_data(channels=channels))
with buzz.Dataset(allow_interpolation=1).close as ds:
# Derived and primitive rasters not computed
if compute_same_address_space:
ac0, ac1 = _AreaCounter(fp), _AreaCounter(fp)
else:
ac0, ac1 = None, None
r0 = _open(
compute_array=functools.partial(_base_computation, area_counter=ac0, reffp=fp),
ow=True,
)
r1 = _open(
compute_array=functools.partial(_derived_computation, area_counter=ac1, reffp=fp),
queue_data_per_primitive={'prim': functools.partial(r0.queue_data, band=-1)},
cache_dir=test_prefix2,
ow=True,
)
assert len(r0.primitives) == 0
assert len(r1.primitives) == 1
assert r1.primitives['prim'] is r0
r1.get_data()
if compute_same_address_space:
ac0.check_done()
ac1.check_done()
r0.close()
r1.close()
# Derived raster not computed
if compute_same_address_space:
ac0, ac1 = _AreaCounter(fp), _AreaCounter(fp)
else:
ac0, ac1 = None, None
r0 = _open(
compute_array=functools.partial(_base_computation, area_counter=ac0, reffp=fp),
ow=False,
)
r1 = _open(
compute_array=functools.partial(_derived_computation, area_counter=ac1, reffp=fp),
queue_data_per_primitive={'prim': functools.partial(r0.queue_data, band=-1)},
cache_dir=test_prefix2,
ow=True,
)
r1.get_data()
if compute_same_address_space:
ac0.check_not_done()
ac1.check_done()
r0.close()
r1.close()
# Primitive raster not computed
if compute_same_address_space:
ac0, ac1 = _AreaCounter(fp), _AreaCounter(fp)
else:
ac0, ac1 = None, None
r0 = _open(
compute_array=functools.partial(_base_computation, area_counter=ac0, reffp=fp),
ow=True,
)
r1 = _open(
compute_array=functools.partial(_derived_computation, area_counter=ac1, reffp=fp),
queue_data_per_primitive={'prim': functools.partial(r0.queue_data, band=-1)},
cache_dir=test_prefix2,
ow=False,
)
r1.get_data()
if compute_same_address_space:
ac0.check_not_done()
ac1.check_not_done()
r0.close()
r1.close()
# Test computation tiles
if compute_same_address_space:
ac0, ac1 = _AreaCounter(fp), _AreaCounter(fp)
else:
ac0, ac1 = None, None
r0 = _open(
compute_array=functools.partial(_base_computation, area_counter=ac0, reffp=fp),
computation_tiles=(11, 11),
ow=True,
)
r1 = _open(
compute_array=functools.partial(_derived_computation, area_counter=ac1, reffp=fp),
queue_data_per_primitive={'prim': functools.partial(r0.queue_data, band=-1)},
cache_dir=test_prefix2,
computation_tiles=(22, 22),
ow=True,
)
r1.get_data()
if compute_same_address_space:
ac0.check_done()
ac1.check_done()
r0.close()
r1.close()
# Several queries, one is dropped, the rest is still working
r0 = _open(
compute_array=functools.partial(_base_computation, reffp=fp),
ow=True,
)
r1 = _open(
compute_array=functools.partial(_derived_computation, reffp=fp),
queue_data_per_primitive={'prim': functools.partial(r0.queue_data, band=-1)},
cache_dir=test_prefix2,
ow=True,
)
t = r1.cache_tiles.flatten()
fps0 = t.tolist() * 2
fps1 = fps0[::-1]
fps2 = np.roll(t, t.size // 2).tolist() * 2
fps3 = fps2[::-1]
it0 = r1.iter_data(fps=fps0)
it1 = r1.iter_data(fps=fps1)
it2 = r1.iter_data(fps=fps2)
it3 = r1.iter_data(fps=fps3)
del it1
assert len(list(it3)) == t.size * 2
assert len(list(it0)) == t.size * 2
assert len(list(it2)) == t.size * 2
> r0.close()
buzzard/test/test_cached_raster_recipe.py:361:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
buzzard/_tools/helper_classes.py:94: in __call__
self._routine()
buzzard/_a_source.py:75: in _close
self._back.close()
buzzard/_a_async_raster.py:203: in close
self.back_ds.deactivate_many(self.async_dict_path_of_cache_fp.values())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <buzzard._dataset_back.BackDataset object at 0x7f323c58da90>
uid_set = dict_values(['/tmp/buzz-ut-5e14f161-6525-46cc-8f29-e2a7857ab0d8/buzz_x000-y000_x00000-y00000_0e79770b32280b4e.tif'])
def deactivate_many(self, uid_set):
# TODO idea: allow recursive uids to group activated rasters and allow group deactivation
if len(uid_set) == 0:
return
with self._ap_lock:
being_used = uid_set & self._ap_used.keys()
if being_used:
raise RuntimeError('Attempting to deactivate {} source currently used'.format(
> len(being_used)
))
E RuntimeError: Attempting to deactivate 1 source currently used
buzzard/_dataset_back_activation_pool.py:44: RuntimeError
----------------------------- Captured stdout call -----------------------------
------------------------------ Captured log call -------------------------------
WARNING root:parameters.py:340 `band` parameter in `ASourceRaster.get_data` is deprecated since v0.6.0, use `channels` instead
WARNING root:parameters.py:340 `band` parameter in `Raster.iter_data` is deprecated since v0.6.0, use `channels` instead
WARNING buzzard._actors.cached.queries_handler:queries_handler.py:203 Dropping a query with 1/2 arrays produced.
WARNING buzzard._actors.cached.queries_handler:queries_handler.py:203 Dropping a query with 1/2 arrays produced.
WARNING buzzard._actors.cached.file_checker:file_checker.py:149 Removing /tmp/buzz-ut-5e14f161-6525-46cc-8f29-e2a7857ab0d8/buzz_x000-y000_x00000-y00000_0e79770b32280b4e.tif because invalid checksum (0000000000003234 instead of 0e79770b32280b4e)
WARNING buzzard._actors.cached.file_checker:file_checker.py:149 Removing /tmp/buzz-ut-5e14f161-6525-46cc-8f29-e2a7857ab0d8/buzz_x000-y000_x00000-y00000_0e79770b32280b4e.tif because invalid checksum (0000000000003234 instead of 0e79770b32280b4e)
WARNING root:parameters.py:340 `band` parameter in `Raster.create_raster_recipe` is deprecated since v0.6.0, use `channels` instead
=============================== warnings summary ===============================
venvs/37/lib/python3.7/site-packages/osgeo/__init__.py:8
/tmp/circleci/buzzard/venvs/37/lib/python3.7/site-packages/osgeo/__init__.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
buzzard/test/test_cached_raster_recipe.py: 316 warnings
/tmp/circleci/buzzard/buzzard/_dataset_back_scheduler.py:38: PendingDeprecationWarning: isAlive() is deprecated, use is_alive() instead
if not self._thread.isAlive():
buzzard/test/test_cached_raster_recipe.py: 21 warnings
/tmp/circleci/buzzard/buzzard/_footprint_intersection.py:127: ShapelyDeprecationWarning: The array interface is deprecated and will no longer work in Shapely 2.0. Convert the '.coords' to a numpy array instead.
points = np.concatenate(list(_exterior_coords_iterator(geom)), axis=0)
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file: /tmp/circleci/buzzard/test-reports/pytest-report-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml -
----------- coverage: platform linux, python 3.7.9-final-0 -----------
Name Stmts Miss Cover
------------------------------------------------------------------
buzzard/__init__.py 32 0 100%
buzzard/_a_async_raster.py 55 5 91%
buzzard/_a_emissary.py 26 10 62%
buzzard/_a_emissary_raster.py 6 0 100%
buzzard/_a_emissary_vector.py 9 3 67%
buzzard/_a_gdal_raster.py 109 49 55%
buzzard/_a_gdal_vector.py 153 132 14%
buzzard/_a_pooled_emissary.py 28 10 64%
buzzard/_a_pooled_emissary_raster.py 6 0 100%
buzzard/_a_pooled_emissary_vector.py 6 0 100%
buzzard/_a_raster_recipe.py 32 1 97%
buzzard/_a_source.py 55 14 75%
buzzard/_a_source_raster.py 69 15 78%
buzzard/_a_source_raster_remap.py 97 22 77%
buzzard/_a_source_vector.py 82 53 35%
buzzard/_a_stored.py 8 0 100%
buzzard/_a_stored_raster.py 47 29 38%
buzzard/_a_stored_vector.py 30 21 30%
buzzard/_actors/__init__.py 0 0 100%
buzzard/_actors/cached/__init__.py 0 0 100%
buzzard/_actors/cached/cache_extractor.py 49 4 92%
buzzard/_actors/cached/cache_supervisor.py 124 2 98%
buzzard/_actors/cached/file_checker.py 120 25 79%
buzzard/_actors/cached/merger.py 84 21 75%
buzzard/_actors/cached/producer.py 57 1 98%
buzzard/_actors/cached/queries_handler.py 89 2 98%
buzzard/_actors/cached/query_infos.py 125 0 100%
buzzard/_actors/cached/reader.py 140 35 75%
buzzard/_actors/cached/writer.py 92 16 83%
buzzard/_actors/computation_accumulator.py 32 0 100%
buzzard/_actors/computation_gate1.py 69 1 99%
buzzard/_actors/computation_gate2.py 59 1 98%
buzzard/_actors/computer.py 108 33 69%
buzzard/_actors/global_priorities_watcher.py 129 11 91%
buzzard/_actors/message.py 16 0 100%
buzzard/_actors/pool_job.py 24 11 54%
buzzard/_actors/pool_waiting_room.py 146 115 21%
buzzard/_actors/pool_working_room.py 37 26 30%
buzzard/_actors/priorities.py 16 8 50%
buzzard/_actors/production_gate.py 47 0 100%
buzzard/_actors/resampler.py 181 39 78%
buzzard/_actors/top_level.py 50 10 80%
buzzard/_cached_raster_recipe.py 74 1 99%
buzzard/_dataset.py 327 159 51%
buzzard/_dataset_back.py 10 0 100%
buzzard/_dataset_back_activation_pool.py 79 24 70%
buzzard/_dataset_back_conversions.py 89 56 37%
buzzard/_dataset_back_scheduler.py 167 14 92%
buzzard/_dataset_pools_container.py 61 33 46%
buzzard/_dataset_register.py 19 4 79%
buzzard/_debug_observers_manager.py 14 1 93%
buzzard/_env.py 80 26 68%
buzzard/_footprint.py 920 497 46%
buzzard/_footprint_intersection.py 122 47 61%
buzzard/_footprint_move.py 72 68 6%
buzzard/_footprint_tile.py 135 67 50%
buzzard/_gdal_file_raster.py 41 7 83%
buzzard/_gdal_file_vector.py 52 37 29%
buzzard/_gdal_mem_raster.py 28 17 39%
buzzard/_gdal_memory_vector.py 33 23 30%
buzzard/_numpy_raster.py 77 23 70%
buzzard/_pint_interop.py 4 0 100%
buzzard/_tools/__init__.py 5 0 100%
buzzard/_tools/helper_classes.py 49 9 82%
buzzard/_tools/multi_ordered_dict.py 66 23 65%
buzzard/_tools/parameters.py 276 148 46%
buzzard/_tools/rect.py 87 44 49%
buzzard/_tools/slices_of_matrix.py 39 33 15%
buzzard/utils/__init__.py 1 0 100%
buzzard/utils/_merge_functions.py 12 0 100%
------------------------------------------------------------------
TOTAL 5583 2086 63%
Coverage XML written to file test-reports/coverage-a75c11b5-3410-47e4-bf79-e3f56806ba96.xml
=========================== short test summary info ============================
FAILED buzzard/test/test_cached_raster_recipe.py::test_[pools0-cache_tiles0]
!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!
================= 1 failed, 338 warnings in 144.51s (0:02:24) ==================
=============
Exited with code exit status 1
CircleCI received exit code 1
Problem with channel indexing while tiling
I've been trying to tile a geotiff using buzzard per your README
r = buzz.open_raster('path/to/my.tif')
tiles = r.fp.tile((512,512))
for i, fp in enumerate(tiles.flat):
rgb = r.get_data(fp=fp, channels=(0, 1, 2))
However, the following error is thrown IndexError: too many indices for array: array is 1-dimensional, but 3 were indexed.
I checked my tiff and can confirm it has 3 channels
>>> r.channel_schema
{'nodata': [None, None, None], 'interpretation': ['redband', 'greenband', 'blueband'], 'offset': [0.0, 0.0, 0.0], 'scale': [1.0, 1.0, 1.0], 'mask': ['all_valid', 'all_valid', 'all_valid']}
So I'm not exactly sure why this is happening.
Tiling question
Currently, does buzzard support producing tiles with overlap?
Issue with adding fields while inserting data
I've encountered an error while setting fields to a file that I'm writing to.
with buzz.create_vector(path=f'out{suffix}.geojson', type='polygon', driver="GeoJSON", ow=True,
sr=self.geoim.wkt_virtual).close as out:
for det in detections:
out.insert_data(det['geom'], fields={'class': det['class'], 'score': det['score']})
Running the above snippet, throws an error as follows:
File "/home/ashwin/Desktop/Projects/demo/predictor.py", line 165, in post_process
out.insert_data(det['geom'], fields={'class': det['class'], 'score': det['score']})
File "/home/ashwin/anaconda3/envs/rdet/lib/python3.7/site-packages/buzzard/_a_stored_vector.py", line 64, in insert_data
fields = self._normalize_field_values(fields)
File "/home/ashwin/anaconda3/envs/rdet/lib/python3.7/site-packages/buzzard/_a_stored_vector.py", line 75, in _normalize_field_values
i = self._back.index_of_field_name[k]
KeyError: 'class'
Source of the error as seen from above trace is here. If the fields I'm setting are not in self._back.index_of_field_name, setting will fail.
I'm not sure why this is the case. Is this not the correct way to set fields for a vector file? What exactly is self._back and what does it have to do with the fields I'm setting?
Empty documentation pages with `latest`
Most pages of the latest documentation are empty: https://buzzard.readthedocs.io/en/latest/footprint.html
This works fine with the stable one: https://buzzard.readthedocs.io/en/stable/footprint.html
codecov broken
We have to setup codecov for the new repo
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.