osmlab / learnoverpass

LearnOverpass is the definitive, one stop learning resource for the Overpass API.

LearnOverpass

This project aims to create a complete, one stop, learning resource for the overpass API.

It features a beginner-friendly, easy-to navigate documentation, as well as an interactive suite of tutorials to teach different aspects of the API.

Contribution Guide

We accept contributions/edits to the site! Feel free to do a pull request if you see an error in the docs/outdated material.

Translation to various languages is also an aspect we need contribution towards to.

A detailed contribution guide can be seen here

Development Guide

The project is/will be built using (mainly) the following dependencies/frameworks:

• Hugo - static site generator written in Go
• Gulp - task manager for preprocessing/asset pipeline
• ReactFlux - main framework for the tutorial engine
• Ace Editor/Code Mirror - for the online code editor
• Stylus - CSS meta language

Setup

TL;DR

1. Install NodeJS, npm
2. Install gulp: npm install -g gulp
3. Run npm install
4. Run gulp
5. Open http://localhost:1313/

Detailed Setup Instructions

If this is the first time you're working with the node ecosystem, install node first. This will add a binary named npm in your path.

Then, run npm install on the root directory of this project, as well as npm install -g gulp.

The theme/layout code can be found under the themes/src directory. Gulp is a task manager that compiles these files to themes/overpass_doc which is the flat html/css files used by the site.

gulp build needs to be run to compile the src theme. The default gulp task, apart from building the layout and running hugo, includes a file watcher that automatically detects changes in the src and compiles it for you, you which is ideal if you are developing the theme/layout.

Note on i18n: Hugo does not natively support i18n/localization, and certain things have been worked around with to make it work. The site uses content sections to order the content by language (en/data, de/data etc.), and uses taxonomies to categorize the different content.

Deploy Script

The project is deployed through the github pages service. The flat hugo files are pushed to the gh-pages branch.

A deploy.sh script is included in the repo to push changes to gh-pages automatically.

License

The website's source is released under the MIT License.

The documentation and courses is released under GNU FDL.

Special Thanks to:

This project is made possible through Google's Summer of Code (GSOC) 2015 Program.

The mentor for this project is none other than Martin Raifer (@tyr_asd), responsible for the excellent Overpass Turbo.

Weekly writeups on the project were posted on Arian's blog. Arian (@secretmapper) is the GSOC student for this project.

learnoverpass's Issues

error on opening http://osmlab.github.io/learnoverpass/en/exercises/intro/1/

I got

An error occured during the execution of the overpass query!
This is what overpass API returned:



<p><strong style="color:#FF0000">Error</strong>: line 1: parse error: An empty query is not allowed </p>

Popups triggered by overpass turbo map

I tried to change some of the Overpass QL code snippets in the reference section, but each time I do any change, a popup with syntax errors appears. Also, I noticed that when switching from one reference page to another reference page, I also sometimes get some intermediate popups.

No content or final recap after last exercise

Clicking on Go to Next Lesson from last exercise number 15 reloads page to the same exercise. This link uses Hugo's .NextInSection property, but because there are no more pages, it points to itself.

learnoverpass/themes/src/layouts/exercise/single.html

Line 46 in 1585e27

<a href="{{ if .NextInSection}}{{.NextInSection.Permalink }}{{ end }}" class="exercise-action-next">

Ideally Go to next lesson should congratulate learner and show final recap of things taught during the course. Maybe with links to next steps, e.g. more advanced courses or documentation for concepts used in the course.

Example for out modifiers

out modifiers are quite useful, but somehow get a bit lost on the respective page

How about adding a query, which demonstrates all of them at once:

[date:"2014-12-30T00:00:00Z"];
way(34345085);
out geom; 
out center;        // not a centroid, e.g. not guaranteed to lie inside polygon
out bb;            // bounding box

Map viewer is limited to zoom 18, we should enable zoom 19, as on osm.org

Zoom 19 would be helpful to show entrance feature in exercise 15.

Overpass turbo link does not longer work

When running the page, the lower part does not show the map. It seems the URL does not longer work: https://overpass-turbo.eu/master/map.html?silent

I modified the files:

• themes/src/layouts/exercise/single.html
• themes/src/layouts/shortcodes/docs_repl.html

To point to the new URL: https://overpass-turbo.eu/map.html?silent

---

Also, in order to run with newer Node versions, it is necessary to do the following in the package.json:

  },
  "overrides": {
    "graceful-fs": "^4.2.10"

Page fails to load via HTTPS

Opening https://osmlab.github.io/learnoverpass/en/ in Chrome gives the following error in the console:

Mixed Content: The page at 'https://osmlab.github.io/learnoverpass/en/' was loaded over HTTPS, but requested an insecure stylesheet 'http://osmlab.github.io/learnoverpass//css/main.css'. This request has been blocked; the content must be served over HTTPS.

Recurse down when already on node level

I noticed the following query in http://osmlab.github.io/learnoverpass/en/docs/filters/polygon/

(node(poly:"50.73 7.13 50.73 7.17 50.75 7.15");>;);
out;

As we're already on node level, the additional >; has no effect. You probably intended this to be a template for ways/rels as well. In that case, I would just put the other options as comment, maybe.

Example:

node(poly:"50.73 7.13 50.73 7.17 50.75 7.15");
//
// (way(poly:"50.73 7.13 50.73 7.17 50.75 7.15");>;);
out;

Intro lesson doesn't find a restaurant

In the intro lesson Query doesn't find a restaurant in (around:150). Is it possible the restaurant doesn't exist anymore?
makes the follow up tasks a little weird.

First lesson starts with an error and invalid code

Tutorial starts with the error on the map when the page is loaded.

It is not a good user experience, because it is impossible to tell if the error is intentional, or it is a broken tutorial. I suspected a broken tutorial after overpass syntax change. It will help to start tutorial with a minimal valid query.

Too much data for some queries

Some queries in the API ref. section request quite a large amount of data, making my browser sluggish and unresponsive.

Example: http://osmlab.github.io/learnoverpass/en/docs/filters/polygon/

Example for Standalone Item query needs review

The example for standalone item queries needs a bit of review: http://osmlab.github.io/learnoverpass/en/docs/standalone-queries/item/

I think, this is not really a suitable example for this statement, as .bank_set out; is just an inputset for out.

Lesson 6+ is broken

I assume the restaurant in question closed or something, because its no longer showing up in the "output".

On a side note, it would be nice if solutions were provided. Sometimes, the instructions can be a bit vague, having the solutions would help people's understanding of the instructions.

Typo in lesson 15

Lesson 15 has the misspelled word 'reemove'

Usilize i18n capabilities of Hugo

[back in 2015] Hugo doesn’t natively support i18n so I also had to do a few things to work around this.

When LearnOverpass was created, there was no support for i18n in Hugo. Content was split on language-based sections, which is not enough to have fully localized site as it leaves content in layout files untranslated.

Nowadays Hugo has multilingual mode, so we could use it for complete site localization.

The complete site localization goal is important as it would make LearnOverpass friendlier for non-English learners.

deprecation warnings on `npm install`

npm WARN deprecated [email protected]: Has been renamed to https://www.npmjs.com/package/webpack-stream
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
npm WARN deprecated [email protected]: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue```

It may be not a full list, `npm install` stalls for some reason (likely network or configuration issue rather than a bug in this project).

Race condition on first run causes disabled UI

Sometimes message listener inside iframe's map.js module is added after message is sent from exercise.js (almost constantly reproducible in incognito mode). This race condition causes message not processed and thus callback not called. The callback is essential to UI as it removes disabled state from Run Query button.

Analysis

First we send answer query, on response we set answer received flag and update map, which sends a second query, that time using code from the editor.

learnoverpass/themes/src/scripts/exercise.js

Lines 21 to 22 in be37427

	onload: function() {
	exercise.query(exerciseAnswer);

learnoverpass/themes/src/scripts/exercise.js

Lines 24 to 32 in 6ffcbd7

	onSuccessData: function(data){
	if(answer === null) {
	if( data.query === exerciseAnswer ) {
	answer = processAnswer(escapeHTML(data.resultText));
	exercise.updateMap();
	}
	return;
	}

learnoverpass/themes/src/scripts/components/docs-repl.js

Lines 113 to 118 in 6ffcbd7

	updateMap: function(){
	if(!pending_query) {
	pending_query = true;
	postTo(iframe, 'update_map', [cm.getValue()]);
	}
	},

Response from that second query activates UI, in particular the Run Query button. Without this button it is not possible to send query again, so there is no workaround for this issue.

learnoverpass/themes/src/scripts/exercise.js

Lines 35 to 44 in 6ffcbd7

	if(firstRun) {
	firstRun = false;
	classie.add(editorCover, "hide");
	classie.remove(runBtn, "disabled");
	classie.remove(resetBtn, "disabled");
	// firstRun should not produce errors
	return;
	}

Root cause

Message listener inside iframe's map.js module is added on $.ready handler. Message is sent from exercise.js from iframe's onload handler.

load event is first fired in iframe context, and only then in parent, i.e. LearnOverpass. But because $.ready handler is called asynchronously, listener is added after message is posted, causing message going unprocessed.

$.ready handler is called asynchronously starting jQuery 3.0.0 released June 9, 2016. This was a breaking change, so issue started to manifest itself only after overpass-turbo's map switched to jQuery ^3.0.0 at February 6, 2018.

Building in exercise 14 was renamed, answer code should be changed

Building from exercise 14 was renamed from Wirtschaftswissenschaften to Wirtschaftswissenschaften (WIWI). Because name is coded in the answer, it makes exercise invalid. This issue also affects exercise 15 for the same reason.

learnoverpass/content/en/exercises/intro/14.md

Line 24 in eafff0c

way["addr:city"=Passau][name="Wirtschaftswissenschaften"];

Answer should be changed to account for a name change, also exercise content should prepare learner that map is a dynamic object.

NB typing Wirtschaftswissenschaften (WIWI) is not an easy task for non-German speakers, maybe we should allow copy-paste from exercise content?

Lesson 5 incorrectly refers to node ID

Lesson 5 instructions say "See, our first statement node(2190458950)" - perhaps that's a leftover from some older version that specified node ID? Currently 4 coordinates are used to form a bbox.

Wrong number of trees in exercise 3, more trees mapped

Exercise 3 says that there are THREE trees in the area, which doesn't reflect currently mapped features, there are eight trees in the area as of September 21, 2019.

Example for Filter by inputset needs review

The following two queries on Filter by inputset probably need to be revised: http://osmlab.github.io/learnoverpass/en/docs/filters/input-set/

Right now, it would print the same info (relation + members) twice, once with full details and once just with reduced details. I think that's probably not intended?

(relation[name="Cividale del Friuli"];>;);
out body;
out skel qt;

(relation[name="Julijske Alpe"];>;);
out body;
out skel qt;

Syntax highlight code segments in exercise text

When reading the exercise text it is difficult to differentiate normal text from code segments. Adding a syntax highlighting (ideally, the same from the code box) would ease the reading …

Hardcoded answer in exercises

Somehow I'm not sure if including concrete Overpass API results in exercise files is a good idea. Tomorrow, someone might add another language to that POI or more it around -> string comparison no longer works. To me this seems rather fragile, given the highly dynamic and volatile nature of OSM data.

At the end of the day you really want to check, if the Overpass API request is correct. So either normalize the query (=remove comments, white spaces, etc.) before comparing them or maybe fire a second predefined query and compare those results with what the user got with his query.

Exercises without regex param fail to pass correct answer

This issue happens on all exercises except exercise number 4. It started to manifest itself after upgrading Hugo (and thus Go which html/template it uses) in #21.

Analysis

When regex parameter is not present in frontmatter, .Params.regex evaluates to zero value, which would render as an empty string by {{ .Params.regex }}, but our template has explicit HTML escaping, which causes {{ .Params.regex | html }} to be rendered as <no value> (before upgrade {{ .Params.regex | html }} rendered empty string).

learnoverpass/themes/src/layouts/exercise/single.html

Line 70 in be37427

<div class="exercise-regex">{{ .Params.regex | html }}</div>

<div class="exercise-regex">&lt;no value&gt;</div> fragment is read as is into RegExp constructor.

learnoverpass/themes/src/scripts/exercise.js

Line 13 in be37427

var exerciseRegex = new RegExp(htmlDecode(document.getElementsByClassName("exercise-regex")[0].innerHTML).trim());

This regular expression is later used to check that correct answer was received using an expected solution, which fails because solution definitely doesn't match <no value>. This match failure causes exercises without regexp to always fail, even on correct answers.

learnoverpass/themes/src/scripts/exercise.js

Lines 85 to 93 in be37427

	function isCorrectAnswer(result) {
	if(result === answer) {
	if (exerciseRegex) {
	return exerciseRegex.test(exercise.getValue());
	}
	return true;
	}
	return false;
	}

Solution

Given that html/template package used by Hugo has contextual autoescaping, explicit HTML escaping is not needed, so changing {{ .Params.regex | html }} to {{ .Params.regex }} will effectively fix the issue and would not introduce an HTML injection.

Misleading syntax error message on intro

In the first lesson, the query like node(48.5657, 13.44, 48.5662, 13.45); is a valid syntax. But there is still an error message there.

The better message might be "The syntax is correct. Try to enter exact GPS coordinates to pass to the next exercise."

404 on bottom iframe - app completely broken

When accessing https://osmlab.github.io/learnoverpass/en/exercises/intro/1/, or any other exercise, the bottom iframe on the page is serving a HTTP 404 Not Found error.

The URI it is failing to access is https://overpass-turbo.eu/master/map.html?silent, which apparently does not exist anymore. However, https://overpass-turbo.eu/map.html?silent does exist. (Is this the only change needed to fix the problem?)

Furthermore, the top textarea is not editable and the buttons below it are disabled too.

This renders the application completely useless at the moment.