hobu / ept-python

Entwine Python tools

ept-python

Python library for making queries against Entwine Point Tile data.

Install

Using git and pip:

git clone https://github.com/hobu/ept-python.git
cd ept-python
pip install .

Using just pip:

pip install git+https://github.com/hobu/ept-python.git

Usage

Example query and output to LasData object:

import ept

url = 'https://na-c.entwine.io/red-rocks'
bounds = ept.Bounds(
    482298, #xmin
    4390602, #ymin
    1762, #zmin
    482421, #xmax
    4390690, #ymax
    2113 #zmax
)

query = ept.EPT(url, bounds=bounds)
las = query.as_laspy()
print(las)

<LasData(1.2, point fmt: <PointFormat(3, 4 bytes of extra dims)>, 98050 points, 4 vlrs)>

Uninstall

pip uninstall ept-python

ept-python's Issues

Missing tasks with low semaphore value

I have a local server for ept data, running a query like:

query = ept.EPT(
    'http://localhost/data/ept/',
    bounds=ept.Bounds(774163, 3482814, 0, 774263, 3482914, 1000)
)
data = query.data()

The server shows the correct GET requests in the logs:

192.168.0.1 - - [22/Sep/2021:19:16:06 +0000] "GET /data/ept/ept.json HTTP/1.1" 200 2959 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:10 +0000] "GET /data/ept/ept-hierarchy/0-0-0-0.json HTTP/1.1" 200 529057 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:10 +0000] "GET /data/ept/ept-hierarchy/8-107-146-127.json HTTP/1.1" 200 8577 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:11 +0000] "GET /data/ept/ept-data/1-0-1-1.laz HTTP/1.1" 200 6963 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:11 +0000] "GET /data/ept/ept-data/2-1-2-2.laz HTTP/1.1" 200 5246 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:11 +0000] "GET /data/ept/ept-data/3-3-4-4.laz HTTP/1.1" 200 2171 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:11 +0000] "GET /data/ept/ept-data/4-6-9-8.laz HTTP/1.1" 200 1681 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:11 +0000] "GET /data/ept/ept-data/1-0-1-0.laz HTTP/1.1" 200 100090 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:11 +0000] "GET /data/ept/ept-data/0-0-0-0.laz HTTP/1.1" 200 106405 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:11 +0000] "GET /data/ept/ept-data/3-3-4-3.laz HTTP/1.1" 200 199839 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/4-6-9-7.laz HTTP/1.1" 200 222579 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/2-1-2-1.laz HTTP/1.1" 200 208395 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/12-1713-2349-2043.laz HTTP/1.1" 200 349128 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/12-1715-2349-2043.laz HTTP/1.1" 200 307874 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/12-1715-2348-2043.laz HTTP/1.1" 200 315234 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/12-1714-2348-2043.laz HTTP/1.1" 200 360543 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/12-1714-2349-2043.laz HTTP/1.1" 200 356531 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/5-13-18-15.laz HTTP/1.1" 200 340836 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/12-1713-2348-2043.laz HTTP/1.1" 200 458484 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/6-26-36-31.laz HTTP/1.1" 200 454641 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/7-53-73-63.laz HTTP/1.1" 200 493448 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:12 +0000] "GET /data/ept/ept-data/11-857-1174-1021.laz HTTP/1.1" 200 546097 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:13 +0000] "GET /data/ept/ept-data/10-428-587-510.laz HTTP/1.1" 200 605378 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:13 +0000] "GET /data/ept/ept-data/8-107-146-127.laz HTTP/1.1" 200 674198 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:13 +0000] "GET /data/ept/ept-data/11-856-1174-1021.laz HTTP/1.1" 200 629387 "-" "Python/3.8 aiohttp/3.7.4.post0"
192.168.0.1 - - [22/Sep/2021:19:16:13 +0000] "GET /data/ept/ept-data/9-214-293-255.laz HTTP/1.1" 200 652445 "-" "Python/3.8 aiohttp/3.7.4.post0"

However, the length on data changes with the value that is set for the task pool semaphore. If the value is 10 I get something around 12-14 task results, but if I set the value to 100 I get all 23 task results.

Replace/supplement endpoint classes with fsspec?

I was thinking about incorporating AWS S3 support, and figured a good way to go about this would be with fsspec and s3fs. This would also open the door for supporting Google Cloud Storage with gcsfs.

Any thoughts/comments? If others think it's a good idea, I can start an fsspec branch.