opengeoscience / geonotebook Goto Github PK

Given the following patch:

diff --git a/geonotebook/kernel.py b/geonotebook/kernel.py
index 4bdcb34..6b61e64 100644
--- a/geonotebook/kernel.py
+++ b/geonotebook/kernel.py
@@ -353,6 +353,13 @@ class Geonotebook(object):
     def get_map_state(self):
         return self.serialize()
 
+    def test_method(self):
+        def callback(*args, **kwargs):
+            print('test_method callback')
+
+        return self._remote.test_method()\
+            .then(callback, self.rpc_error)
+
     def add_layer(self, data, name=None, vis_url=None, **kwargs):
 
         # Create the GeonotebookLayer -  if vis_url is none,  this will take
diff --git a/js/src/MapObject.js b/js/src/MapObject.js
index 55eba23..6fa1789 100644
--- a/js/src/MapObject.js
+++ b/js/src/MapObject.js
@@ -71,9 +71,14 @@ MapObject.prototype.msg_types = [
   'replace_wms_layer',
   'clear_annotations',
   'remove_layer',
-  'add_annotation'
+  'add_annotation',
+  'test_method'
 ];
 
+MapObject.prototype.test_method = function () {
+  console.log('test_method called');
+};
+
 MapObject.prototype._debug = function (msg) {
   console.log(msg); // eslint-disable-line no-console
 };

If you run M.test_method() in the notebook, you will get a warning on the client:

WARNING: Couldn't find callback for message id: 8c51c394-eb07-4481-82f3-fe05dc9a7929

and an error on the server:

ERROR | JSONRPCError (-32700): Could not parse msg: {'id': '8c51c394-eb07-4481-82f3-fe05dc9a7929', 'error': None}

However, if you return a value, it works as expected. It would be better to raise a warning or message telling the user that a return value is expected from RPC methods.

[I 13:50:59.147 NotebookApp] Kernel started: a2f9b6a4-bfa9-48a4-9c94-5cff01855bc8
[IPKernelApp] Loading IPython extension: storemagic
[IPKernelApp] Running file in user namespace: /home/chaudhary/.auto_completion_python.py
[IPKernelApp] ERROR | Exception opening comm with target: geonotebook
Traceback (most recent call last):
File "/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/ipykernel/comm/manager.py", line 90, in comm_open
f(comm, msg)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 470, in handle_comm_open
vis_url="http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png")
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 371, in add_layer
cb = self._remote.add_osm_layer(layer.name, layer.vis_url, params)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 137, in _protocol_closure
raise e
AssertionError: Protocol add_osm_layer has an arity of 2. Called with 3

By default, example notebooks are saved with geonotebook2 kernel. If run from python 3 only geonotebook3 kernel is available. Notebooks are 2/3 compatible but switching the kernel triggers some kind of change-kernel event that we're not capturing with register_events in index.js. In any case the map does not load.

https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

https://github.com/OpenGeoscience/geonotebook/blob/master/geonotebook/wrappers/file_reader.py#L21-L31

This has to do with how we store the key/value pair in M.layers._layers. the _remove_layer callback takes a name (the name of the layer currently on the map) and then tries to remove that layer on M.layers._layer. If that isn't the first layer it won't be found in M.layers._layers.

This is related to dynamically injecting css - fix is being tracked in #24

Some of the dependencies of notebook (GDAL and Mapnik) are harder to build or are not compatible on Ubuntu which makes it harder for end-users to use GeoNotebook on to their system (who prefers system installation as opposed to Docker based). This issue may need to be split into two or more but I just wanted to provide the high level goal that might be relevant to other OS (MacOS) for example. Also, we could look into supporting conda based installation.

Currently, the babel parameters transform is enabled through the ES2015 preset. This does the following:

> fn = function (a=0) {}
[Function: fn]
> fn.toString()
'function () {\n  var a = arguments[0] === undefined ? 0 : arguments[0];\n}'

This breaks the optional argument parsing that we do. We can disable the babel transform (or babel entirely), but that means it will be impossible to support:

• IE (any version)
• Safari < 10 (9.1 is still supported by Apple)
• PhantomJS (any version)
• Node < 6 (without --harmony)

It does look like PhantomJS support for ES6 is coming, but it hasn't landed yet. I know decided not to support older browsers, but this would make that decision permanent and unavoidable, so I wanted to make the consequences clear before disabling babel on my own.

bar # Creating issue for geonotebook. # # Write a message for this issue. The first block of # text is the title and the rest is the description.

Currently configuration of the [ktile] section requires specifying the url:port of the running notebook instance, we should be able to pull this information automatically out of the notebook so running on non-standard ports 'just works'

Same for re-connecting to a kernel

The annotations buttons don't seem to be working. The cursor never changes to +.

Given the following protocol:

{'procedure': 'optional_only',
  'required': [],
  'optional': ["x", "y", "z"]},

A call to r.optional_only(x='foo', z='bar') will produce

{'jsonrpc': '2.0', 'params': ['foo', 'bar'],
  'method': 'optional_only', 'id': 'TEST-ID'}

And be called as r.optional_only(x='foo', y='bar') on the remote side

The dependency on GDAL 2 will likely be a roadblock for many users. It would be helpful to provide instructions for a few common OS's. I started this by adding a wiki page for mac OS. I don't know if we want to move this to the README or not, but I just wanted to make a note of what I did to get it running.

This happens occasionally and presents as some tiles not rendering.

The server throws the following error:

[E 10:27:38.646 NotebookApp] 500 GET /ktile/6d3edfda-cfc0-42b5-bc1a-3cc7fa35126d/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990/21/44/7.png (192.168.30.110) 42.50ms referer=http://localhost:8888/notebooks/02_Raster_Data.ipynb
[E 10:27:38.646 NotebookApp] Uncaught exception GET /ktile/6d3edfda-cfc0-42b5-bc1a-3cc7fa35126d/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990/20/44/7.png (192.168.30.110)
    HTTPServerRequest(protocol='http', host='192.168.30.110:8888', method='GET', uri='/ktile/6d3edfda-cfc0-42b5-bc1a-3cc7fa35126d/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990/20/44/7.png', version='HTTP/1.1', remote_ip='192.168.30.110', headers={'Origin': 'http://localhost:8888', 'Accept-Language': 'en-US,en;q=0.8', 'Accept-Encoding': 'gzip, deflate, sdch', 'Host': '192.168.30.110:8888', 'Accept': 'image/webp,image/*,*/*;q=0.8', 'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36', 'Connection': 'keep-alive', 'Referer': 'http://localhost:8888/notebooks/02_Raster_Data.ipynb'})
    Traceback (most recent call last):
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/web.py", line 1469, in _execute
        result = yield result
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/gen.py", line 1015, in run
        value = future.result()
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/concurrent.py", line 237, in result
        raise_exc_info(self._exc_info)
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/gen.py", line 1021, in run
        yielded = self.gen.throw(*exc_info)
      File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/handler.py", line 138, in get
        layer, coord, extension)
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/tornado/gen.py", line 1015, in run
        value = future.result()
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/concurrent/futures/_base.py", line 398, in result
        return self.__get_result()
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/concurrent/futures/thread.py", line 55, in run
        result = self.fn(*self.args, **self.kwargs)
      File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/handler.py", line 29, in getTileResponse
        return layer.getTileResponse(coord, extension)
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/TileStache/Core.py", line 423, in getTileResponse
        tile = self.render(coord, format)
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/TileStache/Core.py", line 505, in render
        tile = provider.renderArea(width, height, srs, xmin, ymin, xmax, ymax, coord.zoom)
      File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/provider.py", line 243, in renderArea
        Map = self.style_map(Map)
      File "/home/kotfic/src/geonotebook/geonotebook/vis/ktile/provider.py", line 225, in style_map
        band=self.mapnik_band)
      File "/home/kotfic/.venvs/geonotebook2/lib/python2.7/site-packages/mapnik/__init__.py", line 538, in Gdal
        return CreateDatasource(keywords)
    RuntimeError: `/tmp/geonb4Y4vo3/L57.Globe.month09.2010.hh09vv04.h6v1.doy247to273.NBAR.v3.0_504169755319211990.vrt' not recognised as a supported file format.

because gdal.

There is an example of how to do this in the 'Time Series.ipynb' Ideally this will make 'colormap' an optional argument to add_layer. This means use the default matplotlib colormap (I think it is 'jet') if it is not passed in. Otherwise it should take a matplotlib colormap (and derive a suitable list of Color map entry dicts), or an explicit dictionary of values (as it does now).

If I start a kernel with a certain basemap, then change the basemap in the geonotebook.ini config, then refresh - the new basemap is not displayed. This has to do with the basemap only being set on initialization and will require a refactor to fix.

@jbeezley FYI

ConnectionError Traceback (most recent call last)
in ()
----> 1 M.add_layer(rdc[:,[1,2,3]], 'NBAR', opacity=0.8)

/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.pyc in add_layer(self, data, name, vis_url, layer_type, *_kwargs)
347 RuntimeError("RasterDataCollection layers require a 'name'")
348
--> 349 layer = TimeSeriesLayer(name, self._remote, data=data, vis_url=vis_url, *_kwargs)
350
351 else:

/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/layers.pyc in init(self, name, remote, data, vis_url, **kwargs)
88 if self.vis_url is None:
89 self.vis_url = self.config.vis_server.ingest(
---> 90 self.current, name=self.current.name)
91
92 self.vis_server_kwargs = kwargs

/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/vis/geoserver/geoserver.pyc in ingest(self, data, name)
153
154 r = self.c.post("/workspaces.json",
--> 155 json={'workspace': {'name': self.workspace}})
156
157 if r.status_code != 201:

/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/vis/geoserver/geoserver.pyc in post(self, _args, *_kwargs)
23
24 def post(self, _args, *_kwargs):
---> 25 return self._proxy(requests.post, _args, *_kwargs)
26
27 def delete(self, _args, *_kwargs):

/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/vis/geoserver/geoserver.pyc in _proxy(self, method, uri, _args, *_kwargs)
14 return method(uri, _args, *_kwargs)
15 else:
---> 16 return method(self.base_url + uri, _args, *_kwargs)
17
18 def get(self, _args, *_kwargs):

/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/api.pyc in post(url, data, json, *_kwargs)
108 """
109
--> 110 return request('post', url, data=data, json=json, *_kwargs)
111
112

/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/api.pyc in request(method, url, *_kwargs)
54 # cases, and look like a memory leak in others.
55 with sessions.Session() as session:
---> 56 return session.request(method=method, url=url, *_kwargs)
57
58

/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/sessions.pyc in request(self, method, url, params, data, headers, cookies, files, auth, timeout, allow_redirects, proxies, hooks, stream, verify, cert, json)
473 }
474 send_kwargs.update(settings)
--> 475 resp = self.send(prep, **send_kwargs)
476
477 return resp

/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/sessions.pyc in send(self, request, *_kwargs)
594
595 # Send the request
--> 596 r = adapter.send(request, *_kwargs)
597
598 # Total elapsed time of the request (approximately)

/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/requests/adapters.pyc in send(self, request, stream, timeout, verify, cert, proxies)
485 raise ProxyError(e, request=request)
486
--> 487 raise ConnectionError(e, request=request)
488
489 except ClosedPoolError as e:

ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=8080): Max retries exceeded with url: /geoserver/rest/workspaces.json (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7fd370fcaa90>: Failed to establish a new connection: [Errno 111] Connection refused',))

Things were going well until I hit this issue.

Hi:

I wonder do you guys want to integrate Earthengine into geonotebook? Or just keep focus on open source gis software?

I would like to contribute in this part if you guys are interested.

Thanks,

I've just come across this project and it looks like it'd be so useful to me. I've installed following the installation instructions, but when I run the notebooks I get the map appearing behind the notebook itself. It's kinda hard to explain, but the attached image should show what I mean.

Do you have any idea what might be going on here?

@jbeezley Where does this fall on the scale of easy to impossible?

https://github.com/OpenGeoscience/geonotebook/blob/master/geonotebook/wrappers/raster.py#L169-L170

Changing the interval parameter is not doing anything on the RGB case. We might want to add that to vrt configuration if that is possible. I can investigate this.

This produces the same tiles:

M.add_layer(rd[1, 2, 3], interval=(0,1), gamma=2, opacity=1.0)

as this:

M.add_layer(rd[1, 2, 3], interval=(0,999999), gamma=2, opacity=1.0)

@kotfic If you call M.remove_layer(M.layers[-1].name) without a layer overlayed, it removes the base map. I believe we should catch that.

• Sample notebooks
• Sphinx documentation to provide in-depth setup details
• Would be nice to have API docs captured as well

we call log but do not set log on remote at any point

I can consistently reproduce this error by doing the following:

1. Start jupyter notebook
2. Load a new notebook
3. Reload the page
4. Run M.layers.annotation.clear_annotations()

An error will show up in the console saying something like

Comm promise not found for comm id F8167B3AB2234F0D8A23A5AEE5A61835
CommManager.comm_msg	@	comm.js:127
i	@	jquery.min.js:4
Kernel._handle_iopub_message	@	kernel.js:1087
Kernel._finish_ws_message	@	kernel.js:924
(anonymous)	@	kernel.js:915
Promise.resolve (async)		
Kernel._handle_ws_message	@	kernel.js:913
i	@	jquery.min.js:4

No messages appear to come back to the geonotebook kernel.

@kotfic Note this doesn't involve restarting the kernel from the client as I mentioned before.

Currently ReplyCallback is used to manage JSONRPC replies on the Remote object. Given that these are a poor man's promise it would be nice to use an actual implementation that supports chaining promises etc. Given that Tornado already implements Futures it might be nice to start looking there - its possible a fairly large amount of the mechanics developed in kernel.py can be replaced with built in tornado features. An assessment of that would also be nice.

• Introducing Comm Channels
• RPC Mechanism
  • Adding an RPC call (server and client side)
• Mentioning KTile/Tilestache

ref: #52

This may require minor changes to the ansible role to ensure we don't try to checkout to the directory which has already been populated by circle

Is there any documentation on how to add a tile layer to the map?

have tried

M.add_layer("layername","http://tiles{0-3}.basemap.com/url/{z}/{x}/{y}.png")

But it looks like the layer that gets added is requesting:
http://localhost:8888/notebooks/notebooks/null/9/8/4.png etc

Something minor but can be fixed easily.

Works:

M.add_layer(rd[1, 2, 3], interval=(0,1), gamma=2, opacity=1.0)

Does not work:

M.add_layer(rd[1, 2, 3], interval=(0,1), gamma=2, opacity=1)

@kotfic @dorukozturk please update / close this task as applicable

I've seen this 404 pop up a number of times, not sure what the implications are but it should be tracked.

GET https://ec2-35-160-27-75.us-west.compute.amazonaws.com:8000/user/dan/nbextensions/widgets/notebook/js/extension.js?v=20161017203650

[I 13:50:59.147 NotebookApp] Kernel started: a2f9b6a4-bfa9-48a4-9c94-5cff01855bc8
[IPKernelApp] Loading IPython extension: storemagic
[IPKernelApp] Running file in user namespace: /home/chaudhary/.auto_completion_python.py
[IPKernelApp] ERROR | Exception opening comm with target: geonotebook
Traceback (most recent call last):
File "/home/chaudhary/.virtualenvs/geonotebook/local/lib/python2.7/site-packages/ipykernel/comm/manager.py", line 90, in comm_open
f(comm, msg)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 470, in handle_comm_open
vis_url="http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png")
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 371, in add_layer
cb = self._remote.add_osm_layer(layer.name, layer.vis_url, params)
File "/home/chaudhary/tools/geonotebook/geonotebook/geonotebook/kernel.py", line 137, in _protocol_closure
raise e
AssertionError: Protocol add_osm_layer has an arity of 2. Called with 3

We'll need to make sure scripts in devops are consistent with this change

Ability to remove all layers can be good. Most of the times when I interact with the notebook I add bunch of layers and when I want to remove them I call

M.remove_layer(M.layers[0]) 

couple of times to remove them all.

This is a low priority feature.

@danlamanna lets coordinate on content/headings

These are disabled in the following places:
geonotebook/logging_utils.py#L11
geonotebook/vis/ktile/handler.py#L34

Currently we use files on a particular filesystem which then get ingested into KTile or GeoServer. It would be nice to have support Postgres based asset where we could add allow users to visualize data from postgres and also have queries run against postgres/postgis.

This should be possible using rasterio:

rasterio/rasterio#942