opengeoscience / geonotebook Goto Github PK
View Code? Open in Web Editor NEWA Jupyter notebook extension for geospatial visualization and analysis
License: Apache License 2.0
A Jupyter notebook extension for geospatial visualization and analysis
License: Apache License 2.0
Given the following patch:
diff --git a/geonotebook/kernel.py b/geonotebook/kernel.py
index 4bdcb34..6b61e64 100644
--- a/geonotebook/kernel.py
+++ b/geonotebook/kernel.py
@@ -353,6 +353,13 @@ class Geonotebook(object):
def get_map_state(self):
return self.serialize()
+ def test_method(self):
+ def callback(*args, **kwargs):
+ print('test_method callback')
+
+ return self._remote.test_method()\
+ .then(callback, self.rpc_error)
+
def add_layer(self, data, name=None, vis_url=None, **kwargs):
# Create the GeonotebookLayer - if vis_url is none, this will take
diff --git a/js/src/MapObject.js b/js/src/MapObject.js
index 55eba23..6fa1789 100644
--- a/js/src/MapObject.js
+++ b/js/src/MapObject.js
@@ -71,9 +71,14 @@ MapObject.prototype.msg_types = [
'replace_wms_layer',
'clear_annotations',
'remove_layer',
- 'add_annotation'
+ 'add_annotation',
+ 'test_method'
];
+MapObject.prototype.test_method = function () {
+ console.log('test_method called');
+};
+
MapObject.prototype._debug = function (msg) {
console.log(msg); // eslint-disable-line no-console
};
If you run M.test_method()
in the notebook, you will get a warning on the client:
WARNING: Couldn't find callback for message id: 8c51c394-eb07-4481-82f3-fe05dc9a7929
and an error on the server:
ERROR | JSONRPCError (-32700): Could not parse msg: {'id': '8c51c394-eb07-4481-82f3-fe05dc9a7929', 'error': None}
However, if you return a value, it works as expected. It would be better to raise a warning or message telling the user that a return value is expected from RPC methods.
[I 13:50:59.147 NotebookApp] Kernel started: a2f9b6a4-bfa9-48a4-9c94-5cff01855bc8
[IPKernelApp] Loading IPython extension: storemagic
[IPKernelApp] Running file in user namespace: /home/chaudhary/.auto_completion_python.py
[IPKernelApp] ERROR | Exception opening comm with target: geonotebook
Traceback (most recent call last):
File "/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/ipykernel/comm/manager.py", line 90, in comm_open
f(comm, msg)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 470, in handle_comm_open
vis_url="http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png")
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 371, in add_layer
cb = self._remote.add_osm_layer(layer.name, layer.vis_url, params)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 137, in _protocol_closure
raise e
AssertionError: Protocol add_osm_layer has an arity of 2. Called with 3
By default, example notebooks are saved with geonotebook2 kernel. If run from python 3 only geonotebook3 kernel is available. Notebooks are 2/3 compatible but switching the kernel triggers some kind of change-kernel event that we're not capturing with register_events in index.js. In any case the map does not load.
This has to do with how we store the key/value pair in M.layers._layers. the _remove_layer callback takes a name (the name of the layer currently on the map) and then tries to remove that layer on M.layers._layer. If that isn't the first layer it won't be found in M.layers._layers.
This is related to dynamically injecting css - fix is being tracked in #24
Some of the dependencies of notebook (GDAL and Mapnik) are harder to build or are not compatible on Ubuntu which makes it harder for end-users to use GeoNotebook on to their system (who prefers system installation as opposed to Docker based). This issue may need to be split into two or more but I just wanted to provide the high level goal that might be relevant to other OS (MacOS) for example. Also, we could look into supporting conda based installation.
Currently, the babel parameters transform is enabled through the ES2015
preset. This does the following:
> fn = function (a=0) {}
[Function: fn]
> fn.toString()
'function () {\n var a = arguments[0] === undefined ? 0 : arguments[0];\n}'
This breaks the optional argument parsing that we do. We can disable the babel transform (or babel entirely), but that means it will be impossible to support:
--harmony
)It does look like PhantomJS support for ES6 is coming, but it hasn't landed yet. I know decided not to support older browsers, but this would make that decision permanent and unavoidable, so I wanted to make the consequences clear before disabling babel on my own.
bar # Creating issue for geonotebook. # # Write a message for this issue. The first block of # text is the title and the rest is the description.
Currently configuration of the [ktile]
section requires specifying the url:port of the running notebook instance, we should be able to pull this information automatically out of the notebook so running on non-standard ports 'just works'
Same for re-connecting to a kernel
The annotations buttons don't seem to be working. The cursor never changes to +.
Given the following protocol:
{'procedure': 'optional_only',
'required': [],
'optional': ["x", "y", "z"]},
A call to r.optional_only(x='foo', z='bar')
will produce
{'jsonrpc': '2.0', 'params': ['foo', 'bar'],
'method': 'optional_only', 'id': 'TEST-ID'}
And be called as r.optional_only(x='foo', y='bar') on the remote side
The dependency on GDAL 2 will likely be a roadblock for many users. It would be helpful to provide instructions for a few common OS's. I started this by adding a wiki page for mac OS. I don't know if we want to move this to the README or not, but I just wanted to make a note of what I did to get it running.
This happens occasionally and presents as some tiles not rendering.
The server throws the following error:
[E 10:27:38.646 NotebookApp] 500 GET /ktile/6d3edfda-cfc0-42b5-bc1a-3cc7fa35126d/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990/21/44/7.png (192.168.30.110) 42.50ms referer=http://localhost:8888/notebooks/02_Raster_Data.ipynb
[E 10:27:38.646 NotebookApp] Uncaught exception GET /ktile/6d3edfda-cfc0-42b5-bc1a-3cc7fa35126d/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990/20/44/7.png (192.168.30.110)
HTTPServerRequest(protocol='http', host='192.168.30.110:8888', method='GET', uri='/ktile/6d3edfda-cfc0-42b5-bc1a-3cc7fa35126d/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990/20/44/7.png', version='HTTP/1.1', remote_ip='192.168.30.110', headers={'Origin': 'http://localhost:8888', 'Accept-Language': 'en-US,en;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch', 'Host': '192.168.30.110:8888', 'Accept': 'image/webp,image/*,*/*;q=0.8', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36', 'Connection': 'keep-alive', 'Referer': 'http://localhost:8888/notebooks/02_Raster_Data.ipynb'})
Traceback (most recent call last):
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/web.py", line 1469, in _execute
result = yield result
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/gen.py", line 1015, in run
value = future.result()
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/concurrent.py", line 237, in result
raise_exc_info(self._exc_info)
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/gen.py", line 1021, in run
yielded = self.gen.throw(*exc_info)
File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/handler.py", line 138, in get
layer, coord, extension)
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/gen.py", line 1015, in run
value = future.result()
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/concurrent/futures/_base.py", line 398, in result
return self.__get_result()
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/concurrent/futures/thread.py", line 55, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/handler.py", line 29, in getTileResponse
return layer.getTileResponse(coord, extension)
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/TileStache/Core.py", line 423, in getTileResponse
tile = self.render(coord, format)
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/TileStache/Core.py", line 505, in render
tile = provider.renderArea(width, height, srs, xmin, ymin, xmax, ymax, coord.zoom)
File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/provider.py", line 243, in renderArea
Map = self.style_map(Map)
File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/provider.py", line 225, in style_map
band=self.mapnik_band)
File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/mapnik/__init__.py", line 538, in Gdal
return CreateDatasource(keywords)
RuntimeError: `/tmp/geonb4Y4vo3/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990.vrt' not recognised as a supported file format.
because gdal.
There is an example of how to do this in the 'Time Series.ipynb' Ideally this will make 'colormap' an optional argument to add_layer. This means use the default matplotlib colormap (I think it is 'jet') if it is not passed in. Otherwise it should take a matplotlib colormap (and derive a suitable list of Color map entry dicts), or an explicit dictionary of values (as it does now).
If I start a kernel with a certain basemap, then change the basemap in the geonotebook.ini config, then refresh - the new basemap is not displayed. This has to do with the basemap only being set on initialization and will require a refactor to fix.
@jbeezley FYI
ConnectionError Traceback (most recent call last)
in ()
----> 1 M.add_layer(rdc[:,[1,2,3]], 'NBAR', opacity=0.8)
/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.pyc in add_layer(self, data, name, vis_url, layer_type, *_kwargs)
347 RuntimeError("RasterDataCollection layers require a 'name'")
348
--> 349 layer = TimeSeriesLayer(name, self._remote, data=data, vis_url=vis_url, *_kwargs)
350
351 else:
/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/layers.pyc in init(self, name, remote, data, vis_url, **kwargs)
88 if self.vis_url is None:
89 self.vis_url = self.config.vis_server.ingest(
---> 90 self.current, name=self.current.name)
91
92 self.vis_server_kwargs = kwargs
/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/vis/geoserver/geoserver.pyc in ingest(self, data, name)
153
154 r = self.c.post("/workspaces.json",
--> 155 json={'workspace': {'name': self.workspace}})
156
157 if r.status_code != 201:
/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/vis/geoserver/geoserver.pyc in post(self, _args, *_kwargs)
23
24 def post(self, _args, *_kwargs):
---> 25 return self._proxy(requests.post, _args, *_kwargs)
26
27 def delete(self, _args, *_kwargs):
/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/vis/geoserver/geoserver.pyc in _proxy(self, method, uri, _args, *_kwargs)
14 return method(uri, _args, *_kwargs)
15 else:
---> 16 return method(self.base_url + uri, _args, *_kwargs)
17
18 def get(self, _args, *_kwargs):
/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/api.pyc in post(url, data, json, *_kwargs)
108 """
109
--> 110 return request('post', url, data=data, json=json, *_kwargs)
111
112
/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/api.pyc in request(method, url, *_kwargs)
54 # cases, and look like a memory leak in others.
55 with sessions.Session() as session:
---> 56 return session.request(method=method, url=url, *_kwargs)
57
58
/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
473 }
474 send_kwargs.update(settings)
--> 475 resp = self.send(prep, **send_kwargs)
476
477 return resp
/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/sessions.pyc in send(self, request, *_kwargs)
594
595 # Send the request
--> 596 r = adapter.send(request, *_kwargs)
597
598 # Total elapsed time of the request (approximately)
/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
485 raise ProxyError(e, request=request)
486
--> 487 raise ConnectionError(e, request=request)
488
489 except ClosedPoolError as e:
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8080): Max retries exceeded with url: /geoserver/rest/workspaces.json (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fd370fcaa90>: Failed to establish a new connection: [Errno 111] Connection refused',))
Things were going well until I hit this issue.
Hi:
I wonder do you guys want to integrate Earthengine into geonotebook? Or just keep focus on open source gis software?
I would like to contribute in this part if you guys are interested.
Thanks,
I've just come across this project and it looks like it'd be so useful to me. I've installed following the installation instructions, but when I run the notebooks I get the map appearing behind the notebook itself. It's kinda hard to explain, but the attached image should show what I mean.
Do you have any idea what might be going on here?
@jbeezley Where does this fall on the scale of easy to impossible?
Changing the interval parameter is not doing anything on the RGB case. We might want to add that to vrt configuration if that is possible. I can investigate this.
This produces the same tiles:
M.add_layer(rd[1, 2, 3], interval=(0,1), gamma=2, opacity=1.0)
as this:
M.add_layer(rd[1, 2, 3], interval=(0,999999), gamma=2, opacity=1.0)
@kotfic If you call M.remove_layer(M.layers[-1].name) without a layer overlayed, it removes the base map. I believe we should catch that.
we call log but do not set log on remote at any point
I can consistently reproduce this error by doing the following:
M.layers.annotation.clear_annotations()
An error will show up in the console saying something like
Comm promise not found for comm id F8167B3AB2234F0D8A23A5AEE5A61835
CommManager.comm_msg @ comm.js:127
i @ jquery.min.js:4
Kernel._handle_iopub_message @ kernel.js:1087
Kernel._finish_ws_message @ kernel.js:924
(anonymous) @ kernel.js:915
Promise.resolve (async)
Kernel._handle_ws_message @ kernel.js:913
i @ jquery.min.js:4
No messages appear to come back to the geonotebook kernel.
@kotfic Note this doesn't involve restarting the kernel from the client as I mentioned before.
Currently ReplyCallback is used to manage JSONRPC replies on the Remote object. Given that these are a poor man's promise it would be nice to use an actual implementation that supports chaining promises etc. Given that Tornado already implements Futures it might be nice to start looking there - its possible a fairly large amount of the mechanics developed in kernel.py can be replaced with built in tornado features. An assessment of that would also be nice.
ref: #52
This may require minor changes to the ansible role to ensure we don't try to checkout to the directory which has already been populated by circle
Is there any documentation on how to add a tile layer to the map?
have tried
M.add_layer("layername","http://tiles{0-3}.basemap.com/url/{z}/{x}/{y}.png")
But it looks like the layer that gets added is requesting:
http://localhost:8888/notebooks/notebooks/null/9/8/4.png
etc
Something minor but can be fixed easily.
Works:
M.add_layer(rd[1, 2, 3], interval=(0,1), gamma=2, opacity=1.0)
Does not work:
M.add_layer(rd[1, 2, 3], interval=(0,1), gamma=2, opacity=1)
@kotfic @dorukozturk please update / close this task as applicable
I've seen this 404 pop up a number of times, not sure what the implications are but it should be tracked.
GET https://ec2-35-160-27-75.us-west.compute.amazonaws.com:8000/user/dan/nbextensions/widgets/notebook/js/extension.js?v=20161017203650
[I 13:50:59.147 NotebookApp] Kernel started: a2f9b6a4-bfa9-48a4-9c94-5cff01855bc8
[IPKernelApp] Loading IPython extension: storemagic
[IPKernelApp] Running file in user namespace: /home/chaudhary/.auto_completion_python.py
[IPKernelApp] ERROR | Exception opening comm with target: geonotebook
Traceback (most recent call last):
File "/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/ipykernel/comm/manager.py", line 90, in comm_open
f(comm, msg)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 470, in handle_comm_open
vis_url="http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png")
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 371, in add_layer
cb = self._remote.add_osm_layer(layer.name, layer.vis_url, params)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 137, in _protocol_closure
raise e
AssertionError: Protocol add_osm_layer has an arity of 2. Called with 3
We'll need to make sure scripts in devops
are consistent with this change
Ability to remove all layers can be good. Most of the times when I interact with the notebook I add bunch of layers and when I want to remove them I call
M.remove_layer(M.layers[0])
couple of times to remove them all.
This is a low priority feature.
@danlamanna lets coordinate on content/headings
These are disabled in the following places:
geonotebook/logging_utils.py#L11
geonotebook/vis/ktile/handler.py#L34
Currently we use files on a particular filesystem which then get ingested into KTile or GeoServer. It would be nice to have support Postgres based asset where we could add allow users to visualize data from postgres and also have queries run against postgres/postgis.
This should be possible using rasterio:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.