Coder Social home page Coder Social logo

developmentseed / eoapi Goto Github PK

View Code? Open in Web Editor NEW
182.0 9.0 19.0 8.99 MB

[Active Development] Earth Observation API (Metadata, Raster and Vector services)

Home Page: https://eoapi.dev

License: MIT License

Python 10.41% HTML 87.32% Shell 0.92% PLpgSQL 0.97% JavaScript 0.38%

eoapi's Introduction

Create a full Earth Observation API with Metadata, Raster, and Vector services.

Test Downloads


Documentation: https://eoapi.dev

Source Code: https://github.com/developmentseed/eoAPI


Earth Observation API

eoAPI combines several state-of-the-art projects to create a full Earth Observation API. Each service can be used and deployed independently, but eoAPI creates the interconnections between each service:


๐ŸŒ eoAPI: An Open-Source Community Project

eoAPI is proudly open-source and driven by a dedicated community of contributors. We believe in the power of open collaboration and welcome anyone to contribute, discuss, and grow this tool. Join the conversations on GitHub Discussions and make a difference in the Earth Observation realm.


Getting started

The easiest way to start exploring the different eoAPI services is with Docker. Clone this repository and start the multi-container Docker applications using Compose:

git clone https://github.com/developmentseed/eoAPI.git
cd eoAPI
docker compose up

Once the applications are up, you'll need to add STAC Collections and Items to the PgSTAC database. If you don't have these available, you can follow the MAXAR open data demo (or get inspired by the other demos).

Then you can start exploring your dataset with:

If you've added a vector dataset to the public schema in the Postgres database, they will be available through the Vector service at http://localhost:8083.

Deployment with standard runtimes

This repository has current runtimes that are consistently updated with new functionality.

Local deployment

The services can be deployed altogether locally with docker compose up.

Alternatively, you may install the libraries and launch the applications manually:

python -m pip install --upgrade virtualenv
virtualenv .venv
source .venv/bin/activate

export DATABASE_URL=postgresql://username:[email protected]:5439/postgis  # Connect to the database of your choice

python -m pip install uvicorn

###############################################################################
# Install and launch the application
# Select one of the following

###############################################################################
# STAC
python -m pip install "psycopg[binary,pool]" stac-fastapi-pgstac
.venv/bin/uvicorn stac_fastapi.pgstac.app:app --port 8081 --reload

###############################################################################
# RASTER
python -m pip install "psycopg[binary,pool]" titiler-pgstac
.venv/bin/uvicorn titiler.pgstac.main:app --port 8082 --reload

###############################################################################
# VECTOR
python -m pip install tipg
.venv/bin/uvicorn tipg.main:app --port 8083 --reload

Note: Python libraries might have incompatible dependencies, which you can resolve by using a virtual environment for each one.

Deployment on the cloud

Kubernetes

eoapi-k8s contains IaC and Helm charts for deploying eoAPI services on AWS and GCP.

AWS CDK

eoapi-cdk defines a set of AWS CDK constructs that can be used to deploy eoAPI services on AWS. This repository itself makes use of these in infrastructure/aws. An official example usage of these constructs can be found at eoapi-template.

Deployment with custom runtimes

The eoAPI repository hosts customized versions of each base service which can work in parallel or in combination with each other.

eoAPI custom runtimes can be launched with docker:

docker compose -f docker-compose.custom.yml --profile gunicorn up

Alternatively, you may launch the application locally:

python -m pip install --upgrade virtualenv
virtualenv .venv
source .venv/bin/activate

python -m pip install "psycopg[binary,pool]" uvicorn
python -m pip install runtime/eoapi/{SERVICE}  # SERVICE should be one of `raster, vector, stac.`

export DATABASE_URL=postgresql://username:[email protected]:5439/postgis  # Connect to the database of your choice

.venv/bin/uvicorn eoapi.{SERVICE}.app:app --port 8000 --reload

Note: services might have incompatible dependencies, which you can resolve by using a virtual environment for each service.

Contribution & Development

We highly value and rely on our community! You can make a difference whether you're an expert or just getting started. Here's how:

  • Contribute: Check out our CONTRIBUTING.md guide to understand how you can contribute.
  • Engage in Discussions: Share your ideas, ask questions, or provide feedback through GitHub Discussions. This is where most of our project conversations take place.
  • Report Issues: Found a bug or have a feature request? Raise it on our issues page.

License

At Development Seed, we believe in open collaboration and making tools and data more accessible. In line with this ethos, we've explicitly chosen a MIT license for eoAPI.

For full license details, see LICENSE.

Authors

Nurtured by Development Seed

See contributors for a listing of individual contributors.

Changes

See CHANGES.md.

eoapi's People

Contributors

abarciauskas-bgse avatar alukach avatar bitner avatar charalamm avatar drnextgis avatar emileten avatar hrodmn avatar j08lue avatar lanesgood avatar leothomas avatar rbavery avatar sharkinsspatial avatar tylere avatar vincentsarago avatar zacharydez avatar zstatmanweil avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

eoapi's Issues

Add SQL function to create geom/datetime indexes when creating a new Table in the public schema

eoAPI is built on top a Postgres database that has a pgSTAC schema but also a public one we can use to store other geo data. We don't have a service to ingest data but I think it will be cool to create a simple SQL script to automatically create INDEX on any geometry and datetime column so eoapi.vector is fast.

CREATE OR REPLACE FUNCTION on_create_table_func()
RETURNS event_trigger AS $$
BEGIN
    -- find geom/datetime column
    -- add indexes
END
$$
LANGUAGE plpgsql;

CREATE EVENT TRIGGER
on_create_table ON ddl_command_end
WHEN TAG IN ('CREATE TABLE')
EXECUTE PROCEDURE on_create_table_func();

The function and trigger should be used in

cc @bitner

Error on demo data ingestion

Hi! I'm trying to upload some data to the database according to this manual:

$ pypgstac version --dsn postgresql://username:password@localhost:5432/postgis
0.3.4

$ pypgstac load collections oam_collection.json --dsn postgresql://username:password@localhost:5432/postgis --method insert
loading oam_collection.json into collections using insert
Traceback (most recent call last):
  File "/usr/local/bin/pypgstac", line 5, in <module>
    app()
  File "/usr/local/lib/python3.7/dist-packages/typer/main.py", line 214, in __call__
    return get_command(self)(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/dist-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/typer/main.py", line 497, in wrapper
    return callback(**use_params)  # type: ignore
  File "/opt/src/pypgstac/pypgstac/pypgstac.py", line 43, in load
    typer.echo(asyncio.run(load_ndjson(file=file, table=table, dsn=dsn, method=method)))
  File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/opt/src/pypgstac/pypgstac/load.py", line 246, in load_ndjson
    await load_iterator(f, table, conn, method)
  File "/opt/src/pypgstac/pypgstac/load.py", line 230, in load_iterator
    await copy(iter, table, conn)
  File "/opt/src/pypgstac/pypgstac/load.py", line 125, in copy
    delimiter=chr(31),
  File "/usr/local/lib/python3.7/dist-packages/asyncpg/connection.py", line 859, in copy_to_table
    return await self._copy_in(copy_stmt, source, timeout)
  File "/usr/local/lib/python3.7/dist-packages/asyncpg/connection.py", line 1042, in _copy_in
    copy_stmt, reader, data, None, None, timeout)
  File "asyncpg/protocol/protocol.pyx", line 506, in copy_in
asyncpg.exceptions.InvalidTextRepresentationError: invalid input syntax for type json
DETAIL:  The input string ended unexpectedly.

Add queryables endpoint

From a conversation with @developmentseed/atlantis-pod , the idea of a queryable endpoint was proposed. This feature would greatly enhance our user's ability to interact with our data collections. It would enable end-users to understand what attributes they can use to query a given collection.

A similar feature has been successfully implemented in our VEDA project. It allows users to understand what kind of queries they can make. For example, it lets users know if they are able to query certain collections based on cloud cover.

To implement this feature, @sharkinsspatial outlined the following three steps (I may be missing some details, please edit directly):

  • Update STAC-fastapi: The 'queryable' endpoint is a new addition to OGC Features. As such, we'll need to update our STAC-fastapi.

  • Translate it into SQL in pgstac: The next step would be to translate the new 'queryable' feature into SQL queries. These queries can then be used in the Postgres database.

  • Update the connector between pgstac and stac-fastapi: The final step involves updating the connector between pgstac and stac-fastapi to ensure the new 'queryable' endpoint is fully functional.

[Custom runtimes] Make the mosaic map endpoint more interactive

Discussed in #77

Originally posted by hrodmn May 17, 2023
@vincentsarago I think eoAPI is the perfect foundation for some work that I am doing at NCX right now. I want to be able to a) build mosaic endpoints on the fly (I think you are already looking at this here, b) manipulate the asset/rescale/colormap parameters for a mosaic and have some more interactive features (layer control, basemap control, etc) while browsing the map endpoint (I think a lot of the code for asset display control already exists here).

Here are a few ideas for some features to build out:

  1. Add a basemap! Maybe a few options like Stamen Toner and some sort of imagery
  2. Add a tile specification panel to the map that allows you to specify assets, rescale, colormap, etc (similar to raster STAC item viewer). If you add those parameters to the url (e.g. ...{searchid}/map?assets=cog&rescale=0,1000, those parameters get pre-loaded.
  3. When you load the map endpoint without any map parameters, load the basemap with the bbox for the searchid and prompt the user to provide the tile specification in the toolbar.
  4. Add a tile layer control to make it possible to toggle the tiles on/off

I think we can build most by borrowing code from the raster viewer endpoint!

deployment dockerfile.raster can't build

Since we moved to titiler.pgstac, we cannot build the lambda Dockerfile.raster image because of conflicting requirements.

Titiler.pgstac requires psycopg2, but it won't install on lambci/lambda:build-python3.8 because of missing PG libs. Earlier we were using psycopg2-binary requirement, specifically because of this. As this is defined in titiler-pgstac there is not an easy fix.

[CDK] raster <-> stac API connection

using STAC item URL (from eoAPI.stac service) as input of the eoAPI.raster service result in timeout meaning the raster service can't access the STAC service ๐Ÿคทโ€โ™‚๏ธ

Improve Documentation for Onboarding and Data Ingestion

Description

While setting up eoAPI for my project, I noticed some potential areas of improvement in the documentation, especially concerning loading data into eoAPI. The example documents on loading data are currently situated within the demo folder and aren't directly referenced in the README. In addition, I encountered a few friction points:

  • The existing documentation should provide direct instructions on setting up the dependencies.
  • There are additional steps to configure the VPC for titiler to visualize the underlying data when running eoAPI with docker-compose, which are not covered in the current docs.

I suggest we create a more streamlined onboarding experience for developers, particularly around data loading, which was the most time-consuming and confusing step in my experience.

Suggested changes

To make the onboarding process smoother, I propose adding the following documentation:

  • Getting Started Section in README: A new section in the README that guides the user through loading an interesting sample dataset into eoAPI, showcasing its capabilities without requiring any network rule configurations. This could also include a walkthrough of critical capabilities and might be a suitable replacement for the current services section.
  • Detailed Ingestion Workflow Document: A comprehensive document detailing the ingestion workflow and structure, including links to relevant examples for better understanding.
  • Could we provide a containerized service that adds some example data? @vincentsarago @ranchodeluxe @bitner

mosaic table in PG database

Enable retrieving mosaic by name instead of mosaicid

name mosaicid minzoom maxzoom bounds default assets available assets
"vincent.mosaic" "ada12e921qduashdas" 0 24 [-180, -90, 180, 90] ["cog"] ["cog", "thumbnail", "raw"]

[Raster] Gaps in mosaic response

Problem description

The mosaic returned by the /mosaic/<search-id>/tiles/WebMercatorQuad/{z}/{x}/{y}@1x.png?assets=<an-asset> has gaps as shown in the screenshots bellow even though all requests have returned. When I zoom in the regions without any data the data appear again normally.

I think that the gaps appear only when the zoom level is low (more geographical space on the screen). I have checked the logs of the API but nothings seems to be out of the ordinary. The database cpu and ram usage is around 80% but I do not see any failed database connections.

Is it something observed in other deployments as well? Is there something I can do about it?

zoom-out-2
zoom-out

Environment Information

The raster API is deployed as an azure container app and it uses the eoapi.raster-uvicorn docker image. The image is build locally using this repo docker-compose.uvicorn.yml.
The database is deployed as a azure postgres flexible server.

Transition eoAPI infrastructure to external repositories

The libraries that eoAPI demonstrates are the same ones that we already use as the foundation for many of our partner projects (both with NASA and AWS). We've had several internal discussions about how we can potentially change the structure of eoAPI to both show that this combination of libraries is a robust stack that is used in production systems and make this combination of technologies easy for other teams to deploy and customize for their projects. To do this we want to

  1. Capture the infrastructure best practices and patterns we've learned from our production systems in reusable IAC that other teams can leverage and customize when needed.

  2. Support cloud providers other than AWS through a complete k8s IAC solution.

  3. Deploy the APIs with sensible defaults that support the most commonly required basic functionality.

  4. Allow @vincentsarago to continue to use the eoAPI repo as demonstration of the newest cutting edge features in upstream libraries while also providing users more stable deployments with fully tested releases.

In our previous discussions around this I had focused on us transitioning eoAPI to a monorepo that included multiple deployments to various package managers to support IAC package releases. My rationale for this was that we've essentially copy and pasted the eoAPI runtime code for several of our projects and I thought it might be cleaner for those projects to simply use a centralized package. After chatting with @vincentsarago yesterday and noting how thin the runtime code is, we've decided on the opposite approach. We'll focus on moving IAC to external repos (with their own default runtime copies) and the eoAPI can use the AWS CDK IAC library (with runtime overrides) for deployments of its local demonstration runtimes. To tackle this we can

  1. Create a public CDK stack repo (using our networking best practices) that uses the cdk-pgstac construct. This stack should be fully self-contained and allow users to deploy with their own configuration. We have a complete example of this in an existing private DevSeed repo.

  2. Remove the current eoAPI infrastructure in favor of using the CDK stack repo from above and override its default runtimes with the demonstration runtimes in eoAPI (this is already supported in cdk-pgstac).

  3. Move @ranchodeluxe 's spike work on k8s infrastructure to a standalone repo that includes its own default runtimes (and the CI to deploy them as tagged images in public repository). Like the CDK stack, this k8s application should be fully self-contained and allow users to deploy with their own configuration.

  4. Update the landing page documentation to describe "eoAPI" as this suite of core libraries (stac-fastapi, pgSTAC, titiler-pgSTAC) that we use to build applications and point to the external IAC repositories as the place users can go to get standalone deployment packages with a regular release cadence. Differentiate the fact that the runtimes in the eoAPI repository are to demonstrate the newest features for experimentation.

These tasks should provide a good initial target. As discussed on our internal Slack, one of the current pain points with the eoAPI infrastructure is the difficulty in loading sample data for testing due to network security constraints. To remedy this and also support large scale data ingestion in both AWS and non-AWS providers we should

  1. Support the transaction extension in cdk-pgstac and have it enabled with simple security when used in the eoAPI demonstration deployment. This way users can easily load experimental data through a REST endpoint without additional library requirements or networking knowledge.

  2. cdk-pgstac already supports a large scale data ingestion solution, but it is reliant on AWS native services. We'll also need a separate effort to create an agnostic large scale ingestion solution for our k8s deployments.

replace vector and feature service by Tipg

When we started we added a simple OGC feature service which is almost a 1:1 with the stac service so it's not really useful. The vector service tried to create MVT from stac searches which could be a nice feature but maybe not scale.

Let's remove those two service and replace them by a OGC Feature/Tile service based on https://github.com/developmentseed/tipg (but not connected to the pgstac database (for now)

Services

  • pgstac db
  • stac (stac-fastapi) -> pgstac schema in db
  • raster (titiler-pgstac) -> pgstac schema in db
  • vector (tipg) -> public schema in db (we could add the possibility to connect to pgstac schema too)

requests/urllib3 installation broken in Dockerfile.db

I tried running the deployment process because eoAPI sounds AWESOME but I hit a snag on the database deployment. I got it working but it's worth a PR to fix it.

Problem description

A recent upgrade to urllib3 seems to have broken the version of requests that gets installed in Dockerfile.db so the BootstrappedDb deployment fails.

I can't seem to get a full traceback from AWS CloudWatch but here is the error message:

[ERROR] Runtime.ImportModuleError: Unable to import module 'handler': cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/var/task/urllib3/util/ssl_.py)

I was able to fix the problem by explicitly installing urllib3<2 in Dockerfile.db:

diff --git a/infrastructure/aws/dockerfiles/Dockerfile.db b/infrastructure/aws/dockerfiles/Dockerfile.db
index 52df1b4..17c4d23 100644
--- a/infrastructure/aws/dockerfiles/Dockerfile.db
+++ b/infrastructure/aws/dockerfiles/Dockerfile.db
@@ -10,7 +10,7 @@ WORKDIR /tmp
 RUN python -m pip install pip -U
 
 RUN echo "Using PGSTAC Version ${PGSTAC_VERSION}"
-RUN python -m pip install requests psycopg["binary,pool"] pypgstac==${PGSTAC_VERSION} -t /asset
+RUN python -m pip install urllib<2 requests psycopg["binary,pool"] pypgstac==${PGSTAC_VERSION} -t /asset
 
 COPY infrastructure/aws/handlers/db_handler.py /asset/handler.py

Here is a relevant issue in requests: psf/requests#6432
requests v2.30.0 should be working with urllib3>2 but maybe we are pinned to something less than v2.30.0 by another package. I haven't run the docker network locally yet but will try to dig in to find out the actual best solution.

Environment Information

Ubuntu 22.04, Python 3.10.6

Issue 1

Goal: Create a one command line deployment stack with stac-fastapi and titiler
How: Custom stac-fastapi and titiler code + full CDK app
Why: Ease setup of stac-fastapi and titiler

Integration of Radiant Earth's STAC-Browser with eoAPI

Description

The Radiant Earth's STAC-Browser is a widely adopted front-end tool to search and visualize STAC queries. It's well maintained and already uses some of the underlying technologies of eoAPI, specifically TiTiler.

Integrating STAC-Browser with eoAPI could offer a more intuitive, visual, and interactive way for users to search and visualize data available through eoAPI.

Considerations

While STAC-Browser and eoAPI share standard technologies, integrating them might have complexities. To ensure seamless functionality, it may require modifications in either the eoAPI code or in the STAC-Browser itself.

@vincentsarago , your insights on the specifics of integration would be much appreciated. Would you provide more details on what we might need to consider or change to integrate STAC-Browser successfully with eoAPI? Feel free to modify this initial ticket.

Next Steps

Once we better understand the potential hurdles and requirements, we can create a detailed plan for implementation. This plan should also include necessary testing to ensure the integration does not introduce bugs or adversely affect performance.

add recommendations about STAC Items/Collections extensions

in vincentsarago/MAXAR_opendata_to_pgstac#2 I've added the item-assets extension to the collections and then I'm using it (if available) in the mosaic-builder

if (collection_metadata && collection_metadata.item_assets) {
var tbl = document.createElement("table");
tbl.classList = ["mt6"];
tbl.style.width = '100%';
var row = tbl.insertRow(-1);
var cell1 = row.insertCell(0);
var cell2 = row.insertCell(1);
cell1.innerHTML = "Asset";
cell2.innerHTML = "Type"
for (let asset_name in collection_metadata.item_assets) {
var row = tbl.insertRow(-1);
var cell1 = row.insertCell(0);
var cell2 = row.insertCell(1);
cell1.innerHTML = asset_name;
cell2.innerHTML = collection_metadata.item_assets[asset_name].type;
}
collection_meta.appendChild(tbl)

Collections

Items

  • eo
  • projection
  • alternate-assets
  • ?

set `db_max_conn_size` to 1 when deployed in AWS Lambda

lambda doesn't need to open a pool of connection because we have a 1:1 request/lambda behaviour

stac-fastapi: db_min_conn_size / db_max_conn_size
timvt: db_min_conn_size / db_max_conn_size
tifeatures: db_min_conn_size / db_max_conn_size
titiler-pgstac: db_min_conn_size / db_max_conn_size

[Custom rutimes - Raster] Mosaic Builder UI

Let's create a simple Demo UI to create Mosaic

in titiler-pgstac (eoapi.raster), a Mosaic is a stac-api (pgSTAC) Search request, which we register through the /mosaic/register endpoint (https://stac-utils.github.io/titiler-pgstac/advanced/metadata/)

Requirements:

  • Per Collection Mosaic: Ideally a mosaic is Per Collection (because then you can assume that all items have the same assets or at least share a minimal set of assets)
  • add bounds for the mosaic from the Collection metadata
  • add set of defaults mosaic options (asset selection, expression, color correction, rescaling ....) from which we can construct Tile URLs
  • add an option to get Min/Max zoom from all the assets (or selected asset ๐Ÿคท)

Note: To be able to fully create a perfect mosaic, the user might need to get statistics/info from items so it will be nice to have a place for this.

pgstac migration do not work

def register_pgstac(cursor, db_name: str, username: str) -> None:
"""Register PgSTAC."""
version = "0.3.4"
pgstac_sql = str(
resources_files("pypgstac") / "migrations" / f"pgstac.{version}.sql"
)
sqlstrs = []
with open(pgstac_sql) as f:
sqlstrs.extend(f.readlines())
migration_sql = "\n".join(sqlstrs)
cursor.execute(sql.SQL(migration_sql))

SMALL: Use Schema JSON for Validation

Background

Currently we are using custom validators on values.yaml. Seems like internal to DS we prefer using JSON Schema validators. So move it over to that

AC

  • transition to JSON Schema via examples internal to DS

Context STAC extension

Context extension (to be more precise its matched property) is disabled by default in PGStac. I think we should find a way to make it configurable.

It seems that it can be easily added to db_handler but what about database service in the docker-compose?

SMALL: DRY Service Definitions in `helm`

Background

After getting the initial draft services up it seems the raster, stac and vector services all have about the same structure and can be DRY'd up in some general fashion. What we do know:

  • try to avoid using template partials via _helpers.tpl b/c Isayah and I agree it's not very intuitive
  • if that can't be avoided we might skip being DRY in favor of readability
  • simple loop and using intermediate temp vars might work fine here

AC

  • what else besides _helpers.tpl?
  • DRY up with different methods
  • Gut check on how we feel about the DRY methods

SMALL: CI tests for `helm` Outputs

Background

What's most important to have tests for is validation of input values.yaml and any conditionals that drastically change the meaning of the outputs

AC

  • test validation errors thrown for required params
  • test ingress annotations pass through
  • test ingress.enable_shared_ingress creates ingress with group annotation
  • test ingress port_type toggle
  • test service enabled toggle

new FastAPI lifespan might not work in lambda

if "AWS_EXECUTION_ENV" in os.environ:
loop = asyncio.get_event_loop()
loop.run_until_complete(app.router.startup())

previously we were checking if we were in AWS Lambda to run the startup event but with the new lifespan method implemented in latest FastAPI (and used in tipg https://github.com/developmentseed/tipg/blob/main/tipg/main.py#L32-L59) we might have the same issue

ref:

The infrastructure/DEPLOYMENT.md instructions do not work

Problem description

The instructions in infrastructure/DEPLOYMENT.md did not work and the CDK_EOAPI_DB_PGSTAC_VERSION is outdated.

Specifically:

  • The command CDK_EOAPI_STAGE=staging CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 npm --prefix infrastructure/aws run cdk -- synth returns AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.. I run npm --prefix infrastructure/aws run cdk bootstrap and after re-run the commands but nothing changed.
  • The command CDK_EOAPI_STAGE=staging CDK_EOAPI_DB_PGSTAC_VERSION=0.7.1 npm --prefix infrastructure/aws run cdk -- deploy eoAPI-${CDK_EOAPI_STAGE} returns No stacks match the name(s) eoAPI-staging
  • I guess CDK_EOAPI_DB_PGSTAC_VERSION should become equal to 0.7.10

Expected Output

Deploy eoAPI in aws.

Environment Information

  • Os: Ununtu
  • Python: 3.10.11
  • Node: v18.16.1
  • npm: 9.5.1
  • CDK version: 2.76.0

Tilejson endpoint redirects to the page which returns 400 Bad Request

$ curl -L -v "http://localhost:8081/collections/openaerialmap/items/6136495327f4170005130793/tilejson.json?tile_scale=1"
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8081 (#0)
> GET /collections/openaerialmap/items/6136495327f4170005130793/tilejson.json?tile_scale=1 HTTP/1.1
> Host: localhost:8081
> User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)
> Accept: */*
> Referer: 
> 
< HTTP/1.1 307 Temporary Redirect
< date: Thu, 09 Sep 2021 09:25:00 GMT
< server: uvicorn
< location: http://0.0.0.0:8082/stac/tilejson.json?url=stac%3A%2F%2FeyJpZCI6ICI2MTM2NDk1MzI3ZjQxNzAwMDUxMzA3OTMiLCAiYmJveCI6IFstODAuMzUwNDg2LCAyNy40MTgyODEsIC04MC4zNDYxNzQsIDI3LjQyNTg4NF0sICJ0eXBlIjogIkZlYXR1cmUiLCAibGlua3MiOiBbeyJyZWwiOiAiY29sbGVjdGlvbiIsICJocmVmIjogIm9wZW5hZXJpYWxtYXAiLCAidHlwZSI6ICJhcHBsaWNhdGlvbi9qc29uIn1dLCAiYXNzZXRzIjogeyJpbWFnZSI6IHsiaHJlZiI6ICJodHRwczovL29pbi1ob3Rvc20uczMuYW1hem9uYXdzLmNvbS82MTM2NDhjODI3ZjQxNzAwMDUxMzA3OTEvMC82MTM2NDhjODI3ZjQxNzAwMDUxMzA3OTIudGlmIiwgInR5cGUiOiAiaW1hZ2UvdGlmZjsgYXBwbGljYXRpb249Z2VvdGlmZjsgcHJvZmlsZT1jbG91ZC1vcHRpbWl6ZWQifX0sICJnZW9tZXRyeSI6IHsiYmJveCI6IFstODAuMzUwNDg2LCAyNy40MTgyODEsIC04MC4zNDYxNzQsIDI3LjQyNTg4NF0sICJ0eXBlIjogIk11bHRpUG9seWdvbiIsICJjb29yZGluYXRlcyI6IFtbW1stODAuMzUwNDQyLCAyNy40MjU4ODRdLCBbLTgwLjM1MDQ4NiwgMjcuNDE4MzAxXSwgWy04MC4zNDYyMTgsIDI3LjQxODI4MV0sIFstODAuMzQ2MTc0LCAyNy40MjU4NjRdLCBbLTgwLjM1MDQ0MiwgMjcuNDI1ODg0XV1dXX0sICJjb2xsZWN0aW9uIjogIm9wZW5hZXJpYWxtYXAiLCAicHJvcGVydGllcyI6IHsidGl0bGUiOiAiV2FsZ3JlZW5zIC0gMjAyMTowOTowNiAxMDoxNToxMSIsICJjb250YWN0IjogIldlcyBNYWd5YXIsd2VzbWFneWFyQGdtYWlsLmNvbSIsICJkYXRldGltZSI6IG51bGwsICJwbGF0Zm9ybSI6ICJ1YXYiLCAicHJvdmlkZXIiOiAiV2ViT0RNIiwgImVuZF9kYXRldGltZSI6ICIyMDIxLTA5LTA2VDA2OjE1OjExWiIsICJzdGFydF9kYXRldGltZSI6ICIyMDIxLTA5LTA2VDA1OjE1OjExWiJ9LCAic3RhY192ZXJzaW9uIjogIjEuMC4wIiwgInN0YWNfZXh0ZW5zaW9ucyI6IFtdfQ%3D%3D
< transfer-encoding: chunked
< 
* Ignoring the response-body
* Connection #0 to host localhost left intact
* Issue another request to this URL: 'http://0.0.0.0:8082/stac/tilejson.json?url=stac%3A%2F%2FeyJpZCI6ICI2MTM2NDk1MzI3ZjQxNzAwMDUxMzA3OTMiLCAiYmJveCI6IFstODAuMzUwNDg2LCAyNy40MTgyODEsIC04MC4zNDYxNzQsIDI3LjQyNTg4NF0sICJ0eXBlIjogIkZlYXR1cmUiLCAibGlua3MiOiBbeyJyZWwiOiAiY29sbGVjdGlvbiIsICJocmVmIjogIm9wZW5hZXJpYWxtYXAiLCAidHlwZSI6ICJhcHBsaWNhdGlvbi9qc29uIn1dLCAiYXNzZXRzIjogeyJpbWFnZSI6IHsiaHJlZiI6ICJodHRwczovL29pbi1ob3Rvc20uczMuYW1hem9uYXdzLmNvbS82MTM2NDhjODI3ZjQxNzAwMDUxMzA3OTEvMC82MTM2NDhjODI3ZjQxNzAwMDUxMzA3OTIudGlmIiwgInR5cGUiOiAiaW1hZ2UvdGlmZjsgYXBwbGljYXRpb249Z2VvdGlmZjsgcHJvZmlsZT1jbG91ZC1vcHRpbWl6ZWQifX0sICJnZW9tZXRyeSI6IHsiYmJveCI6IFstODAuMzUwNDg2LCAyNy40MTgyODEsIC04MC4zNDYxNzQsIDI3LjQyNTg4NF0sICJ0eXBlIjogIk11bHRpUG9seWdvbiIsICJjb29yZGluYXRlcyI6IFtbW1stODAuMzUwNDQyLCAyNy40MjU4ODRdLCBbLTgwLjM1MDQ4NiwgMjcuNDE4MzAxXSwgWy04MC4zNDYyMTgsIDI3LjQxODI4MV0sIFstODAuMzQ2MTc0LCAyNy40MjU4NjRdLCBbLTgwLjM1MDQ0MiwgMjcuNDI1ODg0XV1dXX0sICJjb2xsZWN0aW9uIjogIm9wZW5hZXJpYWxtYXAiLCAicHJvcGVydGllcyI6IHsidGl0bGUiOiAiV2FsZ3JlZW5zIC0gMjAyMTowOTowNiAxMDoxNToxMSIsICJjb250YWN0IjogIldlcyBNYWd5YXIsd2VzbWFneWFyQGdtYWlsLmNvbSIsICJkYXRldGltZSI6IG51bGwsICJwbGF0Zm9ybSI6ICJ1YXYiLCAicHJvdmlkZXIiOiAiV2ViT0RNIiwgImVuZF9kYXRldGltZSI6ICIyMDIxLTA5LTA2VDA2OjE1OjExWiIsICJzdGFydF9kYXRldGltZSI6ICIyMDIxLTA5LTA2VDA1OjE1OjExWiJ9LCAic3RhY192ZXJzaW9uIjogIjEuMC4wIiwgInN0YWNfZXh0ZW5zaW9ucyI6IFtdfQ%3D%3D'
*   Trying 0.0.0.0...
* TCP_NODELAY set
* Connected to 0.0.0.0 (127.0.0.1) port 8082 (#1)
> GET /stac/tilejson.json?url=stac%3A%2F%2FeyJpZCI6ICI2MTM2NDk1MzI3ZjQxNzAwMDUxMzA3OTMiLCAiYmJveCI6IFstODAuMzUwNDg2LCAyNy40MTgyODEsIC04MC4zNDYxNzQsIDI3LjQyNTg4NF0sICJ0eXBlIjogIkZlYXR1cmUiLCAibGlua3MiOiBbeyJyZWwiOiAiY29sbGVjdGlvbiIsICJocmVmIjogIm9wZW5hZXJpYWxtYXAiLCAidHlwZSI6ICJhcHBsaWNhdGlvbi9qc29uIn1dLCAiYXNzZXRzIjogeyJpbWFnZSI6IHsiaHJlZiI6ICJodHRwczovL29pbi1ob3Rvc20uczMuYW1hem9uYXdzLmNvbS82MTM2NDhjODI3ZjQxNzAwMDUxMzA3OTEvMC82MTM2NDhjODI3ZjQxNzAwMDUxMzA3OTIudGlmIiwgInR5cGUiOiAiaW1hZ2UvdGlmZjsgYXBwbGljYXRpb249Z2VvdGlmZjsgcHJvZmlsZT1jbG91ZC1vcHRpbWl6ZWQifX0sICJnZW9tZXRyeSI6IHsiYmJveCI6IFstODAuMzUwNDg2LCAyNy40MTgyODEsIC04MC4zNDYxNzQsIDI3LjQyNTg4NF0sICJ0eXBlIjogIk11bHRpUG9seWdvbiIsICJjb29yZGluYXRlcyI6IFtbW1stODAuMzUwNDQyLCAyNy40MjU4ODRdLCBbLTgwLjM1MDQ4NiwgMjcuNDE4MzAxXSwgWy04MC4zNDYyMTgsIDI3LjQxODI4MV0sIFstODAuMzQ2MTc0LCAyNy40MjU4NjRdLCBbLTgwLjM1MDQ0MiwgMjcuNDI1ODg0XV1dXX0sICJjb2xsZWN0aW9uIjogIm9wZW5hZXJpYWxtYXAiLCAicHJvcGVydGllcyI6IHsidGl0bGUiOiAiV2FsZ3JlZW5zIC0gMjAyMTowOTowNiAxMDoxNToxMSIsICJjb250YWN0IjogIldlcyBNYWd5YXIsd2VzbWFneWFyQGdtYWlsLmNvbSIsICJkYXRldGltZSI6IG51bGwsICJwbGF0Zm9ybSI6ICJ1YXYiLCAicHJvdmlkZXIiOiAiV2ViT0RNIiwgImVuZF9kYXRldGltZSI6ICIyMDIxLTA5LTA2VDA2OjE1OjExWiIsICJzdGFydF9kYXRldGltZSI6ICIyMDIxLTA5LTA2VDA1OjE1OjExWiJ9LCAic3RhY192ZXJzaW9uIjogIjEuMC4wIiwgInN0YWNfZXh0ZW5zaW9ucyI6IFtdfQ%3D%3D HTTP/1.1
> Host: 0.0.0.0:8082
> User-Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)
> Accept: */*
> Referer: http://localhost:8081/collections/openaerialmap/items/6136495327f4170005130793/tilejson.json?tile_scale=1
> 
< HTTP/1.1 400 Bad Request
< date: Thu, 09 Sep 2021 09:25:00 GMT
< server: uvicorn
< content-length: 76
< content-type: application/json
< 
* Connection #1 to host 0.0.0.0 left intact
{"detail":"assets must be defined either via expression or assets options."}

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.