osmlab / atlas Goto Github PK

#164 fixes a spotless issue that did make it through travis even though it should not.

There is a reference to /Users/matthieun/forLoop.sh for this test to run.

Why is PyAtlas still only Python2? Have you tried to automatically convert it with 2to3?

Thank you all for this wonderful project.

I am trying to simplify an osm network so that complex junctions such as motorway interchanges and roundabouts are collapsed into a single node. I was very excited about this project when I saw a similar [question] (https://stackoverflow.com/questions/64644887/download-roads-network-from-openstreetmap-and-simplify-graph-of-roads)
on stack overflow.

I have run BigNodeFinder on the BLZ_7-32-57.atlas example and saved output using findAndSaveBigNodesAsGeoJson method. What I am getting though, which is shown on the attached image is not exactly what I was expecting. Am I missing something? Sorry if this is the wrong platform for the question, however the question on stack overflow is already closed as it is "not according to guidelines"

Is there a method that will create a new Atlas or network of BigNodes and "BigEdges"?

I would be grateful for any guidance.

Thanks
Lawrence

It would be great to have a function to flatten a Relation tree, à la osmlab/atlas-checks#68. This would make it easier to create viewable representations of relations, as geoJSONs for example.

All the CompleteObjects classes throw "Class is not abstract and does not override" error in Intellij. Gradle build is successful but cannot build the project in Intellij. Following is the screenshot of the error message for CompletePoint class.

Description

Create additional SidewalkRight and SidewalkLeft tags for the complete coverage of OSM Sidewalk

Impact
Adding SidewalkRight/Left tags will help to cover separate sidewalk logic in Atlas-Checks.
Example:

Can anyone explain why the original Node information on a way is lost? Why is the osmId lost for these Nodes in between start and end nodes?
Is there any way to get to that information somehow?

Adding "flooded" to tunnel tag and to .isTunnel() function
In the UnusualLayerTags check there are a few false positives because I check for the tunnel tag and it flags tunnel:flooded, even though that is appropriate.
Resource: OSMWiki

May you please publish binary releases? If not officially just for me ;) ??

I've got similar issues to #351

As far as I am aware, this class does not support the following use case. Say I have a JSON file that looks like:

{
    "testobject1": {
        "testparam1": "testvalue1",
        "testparam2": "testvalue2",
        "testparam3": {
            "testparam4": "testvalue4"
        }
    },

    "testobject2": {
        "testparam1": "testvalue1"
    }
}

Assuming I have loaded the JSON file into a Resource, I can load the Resource into a StandardConfiguration and query it like so:

final StandardConfiguration configuration = new StandardConfiguration(jsonResource);
final Configurable config = configuration.get("testobject1");

This gives back a Configurable object that lets me access the parameters in testobject1. However, what I would like to do is:

final StandardConfiguration configuration = new StandardConfiguration(jsonResource);
for (Configurable config : configuration.allKeys())
{
    // do something with the config here
}

Having some kind of allKeys method would be extremely useful.

If this is already possible, or I am missing something obvious, please let me know! Otherwise this seems like a really useful class, and I would like to use the rest of its functionality. I just really need some way to iterate!

Thanks!

Summary

Atlas does not currently support ingestion of OSM polygons represented as closed Ways which contain certain tags. Updating Atlas to support ingestion of OSM polygons into Atlas as Areas instead of Edges would improve one-to-one conversion between OSM and Atlas with respect to polygons, and eliminate several variations of false positive flags generated by Atlas Checks. See the examples below for details on the various ways this issue appears.

Example 1:
A closed way is tagged with building=yes and highway=residential, and is ingested into Atlas as two Edges because of the highway tag. The image below shows this side by side (left=OSM, right=Atlas).
Link to OSM Way

In example 1, the closed way is an area polygon in OSM, but is being ingested in to Atlas as an Edge because of an erroneous highway tag. This unintentionally removed this feature from being considered in Area-based checks in Atlas Checks, and included it in Edge-based checks. It would be helpful to somehow check for closed ways with highway tags (even though in this case the highway tag is erroneous) and handle this by recognizing these features as Areas and not Edges.

Example 2:
A closed way is tagged with highway=pedestrian and area=yes, and is ingested into Atlas as two Edges. The image below shows this side by side (left=OSM, right=Atlas).
Link to OSM Way

Example 3:
A closed way is tagged with man_made=pier, and is ingested into Atlas as an Edge by inclusion in the Edge filter, and by exclusion in the Area filter.
Link to OSM Way

In examples 2 and 3, it is beneficial to maintain Edge navigability of the highway feature, but also would be an improvement to be able to somehow also recognize and interpret this as an Area in Atlas.

We currently have an Iterables.stream(Iterable<Type> source) method. It would make logical sense to have a corresponding parallel stream method.

Have the option to apply a set of changes to an Atlas as a view, without copying the whole Atlas.

Final PR to dev: #300

Related PRs:

• #273
• #280
• #284
• #285
• #286
• #287
• #288
• #289
• #290
• #291
• #292
• #293
• #295
• #298

There are some warnings that need to be fixed (crossed out means fixed):

We have a number of ways to supply test Atlas data. Some options include:

• leveraging the TestAtlas and supporting annotations
• using .txt Atlas files
• using .atlas.gz Atlas files
• supplying raw .osm files, created in JOSM

Each option has a specific use case. This task involves creating a README documenting each option, any standardization (for example - do not GZIP your text files) and when/how to use each one.

There is not a method for retrieving relations that contain a given object of any sort. Something similar to atlas.areasCovering() for relations would be of great use. The current alternative involves looping through all relations in an atlas, and is very memory intensive.

The documentation for TaggableFilters states that water->!pond,!lake should return true for "Anything without water=pond or water=lake". In attempting to implement a similar filter, highway->!service,!constructions, it did not appear that this hold true. In my case, the filter would only return false if highway was equal to service, instead of service or construction. I believe this is caused by line 101 in Taggable, as it always returns a boolean for a value proceeded by a ! regardless of there being any other values to check.
As a note, it is possible to work around this by using the filter highway->!service&highway->!construction.

From #8 (comment)

OSM Nodes that are part of a Relation are also duplicated as an Atlas Point. That should probably not be the case if they do not have any OSM tags.

Atlas way sectioning assigns identifier normally but swapping from-to nodes, which makes reverse ways ONEWAY=-1 as closed ways.
https://wiki.openstreetmap.org/wiki/Key:oneway?uselang=en
https://wiki.openstreetmap.org/wiki/Item:Q4669

Turkey. Atlas
Example:
https://www.openstreetmap.org/way/185261922
id: 185261922000001 start: POINT (30.3606031 37.766764) end: POINT (30.3611556 37.7668865)
id: 185261922000002 start: POINT (30.3603143 37.7668942) end: POINT (30.3606031 37.766764)

Example:
https://www.openstreetmap.org/way/204103539
id: 204103539000001 start: POINT (35.3743971 37.0080407) end: POINT (35.3738629 37.0082803)
id: 204103539000002 start: POINT (35.3745187 37.0079797) end: POINT (35.3743971 37.0080407)

Notice that start node of section 1 and end node of section 2 sharing the same location, which making entire segment as closed way.

Expected behavior:
Atlas way sectioning assigns identifiers according to vector.

Description
ComplexRoundabout validation combining multiple errors into single ComplexEntityError. This approach makes more difficult adding an AutoFix suggestion to MalformedRoundaboutCheck.

Proposal
Break ComplexEntityError into individual Errors:
Before
WRONG_WAY_INVALIDATION = "This roundabout is going the wrong direction, or has been improperly tagged as a roundabout.";
After
WRONG_WAY_INVALIDATION = "This roundabout is going the wrong direction";
WRONG_WAY_TAGGING = "This way has been improperly tagged as a roundabout.";

Example 1
Wrong Way
Autofix suggestion - reverse geometry

Example 2
Wrong tagging
Autofix suggestion - remove junction=roundabout tag.

Currently both cases falls with the same error message and can't be differentiated for Autofix implementation.

ComplexRoundabout is not properly handling following combination: junction=roundabout + oneway=no. (data issue)
Example:
WayID

MalformedRoundabout Check returns following instructions:
1.This roundabout is malformed. 2. This roundabout does not form a single, one-way, complete, car navigable route.
Problem occurs here:
if (this.roundaboutEdgeSet.stream()
.anyMatch(roundaboutEdge -> !HighwayTag.isCarNavigableHighway(roundaboutEdge)
|| !roundaboutEdge.isMasterEdge())
|| !this.roundaboutRoute.start().inEdges().contains(this.roundaboutRoute.end()))
Possible fix options:
Option1 - filter MaterEdge in SimpleEdgeWalker
Option2 - filter MaterEdge in building a Route fromNonArrangedEdgeSet.

Edges with ID 0 break the logic of master/reverse edges. Those are mostly in @TestAtlas tests, and need to be cleaned up.

Example issue that needed a workaround:
https://github.com/osmlab/atlas/pull/251/files/949f93717fb33c668cd171378908120a585740f6#diff-86c0d01cf7dc9d477f4b732038c0de8bR576

Problem

I am trying to use the command pbf2atlas to create an atlas file from a .osm.pbf file, but the resulting atlas file seems to be corrupted.

What I did

I have downloaded the Belize file from here. Then I ran the command atlas pbf2atlas --countryName BLZ belize-latest.osm.pbf. This gave me the file "BLZ_belize-latest.osm.atlas". Then I ran this code:

import org.openstreetmap.atlas.geography.atlas.Atlas;
import org.openstreetmap.atlas.geography.atlas.AtlasResourceLoader;
import org.openstreetmap.atlas.streaming.resource.File;

public class Main {
    public static void main(String[] args) {
        File atlasFile = new File("BGR_map.osm.atlas");
        Atlas atlas = new AtlasResourceLoader().load(atlasFile);
        atlas.nodes().forEach(System.out::println);
    }
}

,but it gives the following error:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Exception in thread "main" org.openstreetmap.atlas.exception.CoreException: MetaData not here!
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.load(PackedAtlasSerializer.java:96)
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas.load(PackedAtlas.java:190)
	at org.openstreetmap.atlas.geography.atlas.AtlasResourceLoader.load(AtlasResourceLoader.java:73)
	at org.openstreetmap.atlas.geography.atlas.AtlasResourceLoader.load(AtlasResourceLoader.java:96)
	at Main.main(Main.java:8)
Caused by: org.openstreetmap.atlas.exception.CoreException: Unable to read Atlas field metaData
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.deserializeIfNeeded(PackedAtlasSerializer.java:154)
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlas.metaData(PackedAtlas.java:511)
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.load(PackedAtlasSerializer.java:92)
	... 4 more
Caused by: org.openstreetmap.atlas.exception.CoreException: Could not load Field metaData from BLZ_belize-latest.osm.atlas
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.deserializeResource(PackedAtlasSerializer.java:258)
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.deserializeSingleField(PackedAtlasSerializer.java:275)
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.load(PackedAtlasSerializer.java:344)
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.deserializeIfNeeded(PackedAtlasSerializer.java:149)
	... 6 more
Caused by: java.io.StreamCorruptedException: invalid stream header: 08B98305
	at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:866)
	at java.io.ObjectInputStream.<init>(ObjectInputStream.java:358)
	at org.openstreetmap.atlas.geography.atlas.packed.PackedAtlasSerializer.deserializeResource(PackedAtlasSerializer.java:247)
	... 9 more

On the other hand, if I create the atlas file with my own Java code, it works correctly afterwards and opens the file as it should. My code for building the file is:

import org.openstreetmap.atlas.geography.atlas.Atlas;
  	private static void buildAtlas() {
        File pbfFile = new File("belize-latest.osm.pbf");
        System.out.println("Loading raw atlas from PBF...");
        RawAtlasGenerator rawAtlasGenerator = new RawAtlasGenerator(pbfFile, AtlasLoadingOption.createOptionWithNoSlicing());
        System.out.println("Building atlas...");
        Atlas atlas = rawAtlasGenerator.build();
        System.out.println("Saving atlas...");
        atlas.save(new File("map.atlas"));
    }

Version

atlas-config list says atlas-a421a45

There is also a short discussion about this problem here. At first I thought that I am doing something wrong, but maybe the issue is with the command itself.

C:\Users\user\Documents\Uni\5CS024>py -m pip install pyatlas
Collecting pyatlas
  Using cached pyatlas-5.8.9-py3-none-any.whl (52 kB)
Collecting shapely==1.6.4
  Using cached Shapely-1.6.4.tar.gz (224 kB)
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\user\AppData\Local\Programs\Python\Python38-32\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\user\\AppData\\Local\\Temp\\pip-install-xthmnf2s\\shapely\\setup.py'"'"'; __file__='"'"'C:\\Users\\user\\AppData\\Local\\Temp\\pip-install-xthmnf2s\\shapely\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\user\AppData\Local\Temp\pip-install-xthmnf2s\shapely\pip-egg-info'
         cwd: C:\Users\user\AppData\Local\Temp\pip-install-xthmnf2s\shapely\
    Complete output (9 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\user\AppData\Local\Temp\pip-install-xthmnf2s\shapely\setup.py", line 80, in <module>
        from shapely._buildcfg import geos_version_string, geos_version, \
      File "C:\Users\user\AppData\Local\Temp\pip-install-xthmnf2s\shapely\shapely\_buildcfg.py", line 200, in <module>
        lgeos = CDLL("geos_c.dll")
      File "C:\Users\user\AppData\Local\Programs\Python\Python38-32\lib\ctypes\__init__.py", line 373, in __init__
        self._handle = _dlopen(self._name, mode)
    FileNotFoundError: Could not find module 'geos_c.dll' (or one of its dependencies). Try using the full path with constructor syntax.
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I have added geos_c.dll to my PATH as "C:\OSGeo4W64\bin\geos_c.dll". Additonally, I tried:

py -m pip install shapely

which worked fine, by atlas doesn't seem to want to use it, is there a variable I need to set to change this?

The TurnRestriction class is throwing a NullPointerException on line 332.

It appears this is due to the from and too variable being refactored in #181 to use use the global variables of the same name instead of the renamed local variables (now fromMember and tooMember).

Issue related to #239

The nexus plugin failed multiple times uploading full staged repositories.
https://travis-ci.org/osmlab/atlas/builds/440233428

> Task :uploadArchives
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.8.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 1m 44s
11 actionable tasks: 1 executed, 10 up-to-date
Promote repository
> Task :closeRepository
Requested operation wasn't successful in first try. Retrying maximum 20 times with 2 seconds delay between.
Attempt 21/21 failed. WrongNumberOfRepositories was thrown with message 'Wrong number of received repositories in state 'open'. Expected 1, received 8'. Giving up. Configure longer timeout if necessary.
> Task :closeRepository FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':closeRepository'.
> Wrong number of received repositories in state 'open'. Expected 1, received 8

It seems that gradle uploadArchives, when running from a Travis VM (in GCE I believe) creates 7 or 8 partial repositories, instead of only one with all the artifacts.

I tried uploading archives from laptop and only one staging repository showed up.

Screenshots of the staging repositories:

There's many Areas of Interest (AOIs) that are modeled as relations (parks, pools, etc.). It would be useful to have a ComplexEntity type and builder to build and return what type of area it is.

I've just tried to build with
./gradlew clean build --warning-mode all

But the build fails (platform is ArchLinux):

> Configure project :
As part of making the publishing plugins stable, the 'deferred configurable' behavior of the 'publishing {}' block has been deprecated. In Gradle 5.0 the 'enableFeaturePreview('STABLE_PUBLISHING')' flag will be removed and the new behavior will become the default. Please add 'enableFeaturePreview('STABLE_PUBLISHING')' to your settings file and do a test run by publishing to a local repository. If all artifacts are published as expected, there is nothing else to do. If the published artifacts change unexpectedly, please see the migration guide for more details: https://docs.gradle.org/4.10.2/userguide/publishing_maven.html#publishing_maven:deferred_configuration.
Gradle now uses separate output directories for each JVM language, but this build assumes a single directory for all classes from a source set. This behaviour has been deprecated and is scheduled to be removed in Gradle 5.0.
	at quality_7zym5g8opbm02xm4ooe3ip1dh$_run_closure4.doCall(~/aur/atlas/src/atlas/gradle/quality.gradle:32)
	(Run with --stacktrace to get the full stack trace of this deprecation warning.)
The setTestClassesDir(File) method has been deprecated. This is scheduled to be removed in Gradle 5.0. Please use the setTestClassesDirs(FileCollection) method instead.
	at quality_7zym5g8opbm02xm4ooe3ip1dh$_run_closure4.doCall(~/aur/atlas/src/atlas/gradle/quality.gradle:32)
	(Run with --stacktrace to get the full stack trace of this deprecation warning.)

> Task :deleteGeneratedProto
Deleting generated protos

> Task :clean
> Task :extractIncludeProto
> Task :extractProto

> Task :generateProto
Using TaskInputs.file() with something that doesn't resolve to a File object has been deprecated. This is scheduled to be removed in Gradle 5.0. Use TaskInputs.files() instead.

> Task :compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.

> Task :processResources
> Task :classes
> Task :jar
> Task :javadoc
> Task :javadocJar
> Task :sourcesJar
> Task :signArchives SKIPPED
> Task :assemble
> Task :extractIncludeTestProto
> Task :extractTestProto
> Task :generateTestProto NO-SOURCE

> Task :compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

> Task :processTestResources
> Task :testClasses
> Task :extractIncludeIntegrationTestProto
> Task :extractIntegrationTestProto
> Task :generateIntegrationTestProto NO-SOURCE
> Task :compileIntegrationTestJava
> Task :processIntegrationTestResources
> Task :integrationTestClasses
> Task :checkstyleIntegrationTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':checkstyleIntegrationTest'.
> Unable to create Root Module: configLocation {~/aur/atlas/src/atlas/config/checkstyle/checkstyle.xml}, classpath {~/aur/atlas/src/atlas/build/classes/java/integrationTest:~/aur/atlas/src/atlas/build/resources/integrationTest:~/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.12/485f77901840cf4e8bf852f2abb9b723eb8ec29/slf4j-log4j12-1.7.12.jar:~/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.12/8e20852d05222dc286bf1c71d78d0531e177c317/slf4j-api-1.7.12.jar:~/.gradle/caches/modules-2/files-2.1/junit/junit/4.12/2973d150c0dc1fefe998f834810d68f278ea58ec/junit-4.12.jar:~/.gradle/caches/modules-2/files-2.1/net.sf.opencsv/opencsv/2.3/c23708cdb9e80a144db433e23344a788a1fd6599/opencsv-2.3.jar:~/.gradle/caches/modules-2/files-2.1/com.google.code.gson/gson/2.2.4/a60a5e993c98c864010053cb901b7eab25306568/gson-2.2.4.jar:~/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-math3/3.3/a85dc3aa929de625606270ae69a343cda69b7670/commons-math3-3.3.jar:~/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpclient/4.5.1/7e3cecc566df91338c6c67883b89ddd05a17db43/httpclient-4.5.1.jar:~/.gradle/caches/modules-2/files-2.1/org.geotools/gt-shapefile/17.0/99d764b3748011ba7377db185dc51e0a89b8e28e/gt-shapefile-17.0.jar:~/.gradle/caches/modules-2/files-2.1/org.geotools/gt-data/17.0/9509c9cacb05771990ea399e61e5ddabae38e670/gt-data-17.0.jar:~/.gradle/caches/modules-2/files-2.1/org.geotools/gt-main/17.0/3f2f83135a65f8bc9213025b4527a66fee422a65/gt-main-17.0.jar:~/.gradle/caches/modules-2/files-2.1/org.geotools/gt-api/17.0/b85e5f43d89fa42d5b7970a1f49b072556abd2c2/gt-api-17.0.jar:~/.gradle/caches/modules-2/files-2.1/com.vividsolutions/jts/1.13/3ccfb9b60f04d71add996a666ceb8902904fd805/jts-1.13.jar:~/.gradle/caches/modules-2/files-2.1/org.openstreetmap.osmosis/osmosis-pbf/0.44.1/91ebe9a49c06c0a7ac568852f08f822cfcf07ddc/osmosis-pbf-0.44.1.jar:~/.gradle/caches/modules-2/files-2.1/org.openstreetmap.osmosis/osmosis-xml/0.44.1/749d4cbf4c85c738e3c4af785b4017012978e4ff/osmosis-xml-0.44.1.jar:~/.gradle/caches/modules-2/files-2.1/org.openstreetmap.osmosis/osmosis-core/0.44.1/c7f1826c8b70e9d5d463edaeba2085f3ad705446/osmosis-core-0.44.1.jar:~/.gradle/caches/modules-2/files-2.1/org.openstreetmap.osmosis/osmosis-hstore-jdbc/0.44.1/196d5f334e483cc72b11f9edb8941811daf8e248/osmosis-hstore-jdbc-0.44.1.jar:~/.gradle/caches/modules-2/files-2.1/commons-io/commons-io/2.5/2852e6e05fbb95076fc091f6d1780f1f8fe35e0f/commons-io-2.5.jar:~/.gradle/caches/modules-2/files-2.1/commons-cli/commons-cli/1.3.1/1303efbc4b181e5a58bf2e967dc156a3132b97c0/commons-cli-1.3.1.jar:~/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-csv/1.2/bfb42d67ff2e16c78c3c322f4fb7c73fbed1965c/commons-csv-1.2.jar:~/.gradle/caches/modules-2/files-2.1/org.apache.maven/maven-artifact/3.5.4/9c905d5a9c144e6f0d30c9db748090e807fed60e/maven-artifact-3.5.4.jar:~/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-lang3/3.5/6c6c702c89bfff3cd9e80b04d668c5e190d588c6/commons-lang3-3.5.jar:~/.gradle/caches/modules-2/files-2.1/io.github.classgraph/classgraph/4.6.32/4d00e353e22a12ee816f795637bc517ac2f39eb0/classgraph-4.6.32.jar:~/.gradle/caches/modules-2/files-2.1/com.google.guava/guava/18.0/cce0823396aa693798f8882e64213b1772032b09/guava-18.0.jar:~/.gradle/caches/modules-2/files-2.1/org.skyscreamer/jsonassert/1.3.0/d5c250eb122496866de36dc84f98e2945845ecdd/jsonassert-1.3.0.jar:~/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-databind/2.8.8.1/fb1930cd0b1b36924717229f56ca80d785401ff8/jackson-databind-2.8.8.1.jar:~/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-core/2.8.8/d478fb6de45a7c3d2cad07c8ad70c7f0a797a020/jackson-core-2.8.8.jar:~/.gradle/caches/modules-2/files-2.1/org.openstreetmap.osmosis/osmosis-osm-binary/0.44.1/e0246684bb260f100cd0158433421838e1897e5b/osmosis-osm-binary-0.44.1.jar:~/.gradle/caches/modules-2/files-2.1/com.google.protobuf/protobuf-java/2.6.1/d9521f2aecb909835746b7a5facf612af5e890e8/protobuf-java-2.6.1.jar:~/.gradle/caches/modules-2/files-2.1/log4j/log4j/1.2.17/5af35056b4d257e4b64b9e8069c0746e8b08629f/log4j-1.2.17.jar:~/.gradle/caches/modules-2/files-2.1/org.hamcrest/hamcrest-core/1.3/42a25dc3219429f0e5d060061f71acb49bf010a0/hamcrest-core-1.3.jar:~/.gradle/caches/modules-2/files-2.1/org.apache.httpcomponents/httpcore/4.4.3/e876a79d561e5c6207b78d347e198c8c4531a5e5/httpcore-4.4.3.jar:~/.gradle/caches/modules-2/files-2.1/net.sf.jpf/jpf/1.5/dd3c81d603e0223e226ae8ccef53acdfb01052d0/jpf-1.5.jar:~/.gradle/caches/modules-2/files-2.1/commons-logging/commons-logging/1.2/4bfc12adfe4842bf07b657f0369c4cb522955686/commons-logging-1.2.jar:~/.gradle/caches/modules-2/files-2.1/commons-codec/commons-codec/1.10/4b95f4897fa13f2cd904aee711aeafc0c5295cd8/commons-codec-1.10.jar:~/.gradle/caches/modules-2/files-2.1/org.jdom/jdom/1.1.3/8bdfeb39fa929c35f5e4f0b02d34350db39a1efc/jdom-1.1.3.jar:~/.gradle/caches/modules-2/files-2.1/org.geotools/gt-referencing/17.0/411c65538e0b8110e5fd37cc659c8fb29f9d2517/gt-referencing-17.0.jar:~/.gradle/caches/modules-2/files-2.1/org.geotools/gt-metadata/17.0/db3c8bfb4132b7bdb3f10f620f076cec4216ac85/gt-metadata-17.0.jar:~/.gradle/caches/modules-2/files-2.1/org.geotools/gt-opengis/17.0/869a9f51769017a9d7fb454c8cb136d87ff29c99/gt-opengis-17.0.jar:~/.gradle/caches/modules-2/files-2.1/javax.media/jai_core/1.1.3/b179d2efb1174658483e8b41bf4ac9d2eb5de438/jai_core-1.1.3.jar:~/.gradle/caches/modules-2/files-2.1/org.codehaus.woodstox/woodstox-core-lgpl/4.4.1/367250f96efe58f97ede13e6e55e77760b85ab3a/woodstox-core-lgpl-4.4.1.jar:~/.gradle/caches/modules-2/files-2.1/xerces/xercesImpl/2.9.1/7bc7e49ddfe4fb5f193ed37ecc96c12292c8ceb6/xercesImpl-2.9.1.jar:~/.gradle/caches/modules-2/files-2.1/org.codehaus.woodstox/stax2-api/3.1.4/ac19014b1e6a7c08aad07fe114af792676b685b7/stax2-api-3.1.4.jar:~/.gradle/caches/modules-2/files-2.1/org.apache.commons/commons-compress/1.9/cc18955ff1e36d5abd39a14bfe82b19154330a34/commons-compress-1.9.jar:~/.gradle/caches/modules-2/files-2.1/org.postgresql/postgresql/9.4-1201-jdbc4/7f7a680f2aa8fe5d511c5cf5f78fa3a297a8e52b/postgresql-9.4-1201-jdbc4.jar:~/.gradle/caches/modules-2/files-2.1/org.json/json/20090211/c183aa3a2a6250293808bba12262c8920ce5a51c/json-20090211.jar:~/.gradle/caches/modules-2/files-2.1/com.fasterxml.jackson.core/jackson-annotations/2.8.0/45b426f7796b741035581a176744d91090e2e6fb/jackson-annotations-2.8.0.jar:~/.gradle/caches/modules-2/files-2.1/org.codehaus.plexus/plexus-utils/3.1.0/60eecb6f15abdb1c653ad80abaac6fe188b3feaa/plexus-utils-3.1.0.jar:~/.gradle/caches/modules-2/files-2.1/com.googlecode.efficient-java-matrix-library/core/0.26/2df89c44ad92900f025a61e228bbbcd6abca3efd/core-0.26.jar:~/.gradle/caches/modules-2/files-2.1/commons-pool/commons-pool/1.5.4/75b6e20c596ed2945a259cea26d7fadd298398e6/commons-pool-1.5.4.jar:~/.gradle/caches/modules-2/files-2.1/jgridshift/jgridshift/1.0/82c163e82b1548c93a9e866dcfd24c66c590a829/jgridshift-1.0.jar:~/.gradle/caches/modules-2/files-2.1/net.sf.geographiclib/GeographicLib-Java/1.44/7e18468fafefedd5d1934ea830c1c9683ffd0632/GeographicLib-Java-1.44.jar:~/.gradle/caches/modules-2/files-2.1/net.java.dev.jsr-275/jsr-275/1.0-beta-2/7bdd90125e67cd38bee363208e32993c0c75e9be/jsr-275-1.0-beta-2.jar:~/aur/atlas/src/atlas/build/classes/java/main:~/aur/atlas/src/atlas/build/resources/main:~/aur/atlas/src/atlas/build/classes/java/test:~/aur/atlas/src/atlas/build/resources/test}.

What's the maximum JDK version supported?

I'm not sure if I did something, but simply running the code below takes a very long time. The pbf file is about 13MB. Is that the expected performance or I'm doing something wrong?

public File getfile() {
    URL res = getClass().getClassLoader().getResource("rhode-island-latest.osm.pbf");
    File file = new File(res.getPath());
    System.out.print(file.getFile().length());
    return file;
}


public static void main(String[] args) throws URISyntaxException {
    Test t = new Test();
    File file = t.getfile();
    final Atlas rawAtlas = new RawAtlasGenerator(file).build();
    final Atlas atlas = new WaySectionProcessor(rawAtlas, AtlasLoadingOption.createOptionWithOnlySectioning()).run();
}

Adding disused:railway for handling LevelCrossingOnRailwayCheck bug.
Please refer: osmlab/atlas-checks#672

From #8 (comment)

OSM nodes are promoted to Atlas Point when they have some OSM tags. The logic determining when that is the case is very questionable (counting artificial tags and making sure it does not go over a certain limit) and easy to break by unsuspecting changes.

Dear developers,

I'm building with OpenJDK and probably because I'm not Apple jacocoTestReport makes my build fail:

> Task :jacocoTestReport FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':jacocoTestReport'.
> Unable to read execution data file /startdir/src/atlas/build/jacoco/test.exec

How may I disable it?

This test needs to be re-worked or removed entirely. Multi-threaded tests are hard to get right and this caused a Travis failure during a PRB build: https://travis-ci.org/osmlab/atlas/builds/443467043#L854

To avoid failing builds, it has been temporarily ignored until this issue is resolved.

We are getting a ClassCastException when trying to convert a MultiPolyLine to WKT.

Here is the stack trace:

Exception in thread "main" java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [Lcom.vividsolutions.jts.geom.LineString;
	at org.openstreetmap.atlas.geography.converters.jts.JtsMultiPolyLineConverter.convert(JtsMultiPolyLineConverter.java:42)
	at org.openstreetmap.atlas.geography.converters.WktMultiPolyLineConverter.convert(WktMultiPolyLineConverter.java:39)
...

This fix in src/main/java/org/openstreetmap/atlas/geography/converters/jts/JtsMultiPolyLineConverter.java is working for us:

     private static final JtsCoordinateArrayConverter COORDINATE_ARRAY_CONVERTER = new JtsCoordinateArrayConverter();
+    private static final JtsPolyLineConverter POLYLINE_CONVERTER = new JtsPolyLineConverter();
     private static final GeometryFactory FACTORY = JtsPrecisionManager.getGeometryFactory();
 
     @Override
@@ -38,8 +43,10 @@ public class JtsMultiPolyLineConverter implements TwoWayConverter<MultiPolyLine,
     @Override
     public MultiLineString convert(final MultiPolyLine multiPolyLine)
     {
-        final LineString[] lineStrings = (LineString[]) ImmutableList
-                .copyOf(multiPolyLine.iterator()).toArray();
-        return new MultiLineString(lineStrings, FACTORY);
+        final List<LineString> lineStringList =
+            StreamSupport.stream(
+                    Spliterators.spliteratorUnknownSize(multiPolyLine.iterator(), Spliterator.ORDERED), false).
+                    map(pl -> POLYLINE_CONVERTER.convert(pl)).collect(Collectors.toList());
+        return new MultiLineString(lineStringList.toArray(new LineString[lineStringList.size()]), FACTORY);
     }

Converting an OSM PBF file to an Atlas, I noticed that the edge IDs generated during country slicing do not seem to be in line with what is described in the documentation.

To illustrate this, I have created a very small example, consisting of one way: ZIP file.

This is my logic:

final CountryBoundaryMap boundaries = CountryBoundaryMap.fromPlainText(boundariesFile);
final Atlas rawAtlas = new RawAtlasGenerator(osmPbfFile).build();
final Atlas slicedAtlas = new RawAtlasCountrySlicer(AtlasLoadingOption.createOptionWithAllEnabled(boundaries)).slice(rawAtlas);
final Atlas atlas = new WaySectionProcessor(slicedAtlas, AtlasLoadingOption.createOptionWithAllEnabled(boundaries)).run();
atlas.save(atlasFile);

Expected outcome:
Edge IDs 60342899001000, 60342899002001, 60342899001002, as indicated in the README.

Actual outcome:
Edge IDs 60342899003000, 60342899001000, 60342899002000.

In atlas-check when I use atlas data ARG and TUR , I found some Relations generated from atlas have different members from the Relations in OSM, for example below:

Relation 10520585 in atlas object only has member 136811331, but Relation 10520585 which was edited over 1 year ago in OSM has 12 members.

Relation 10520342 in atlas object only has member 136811331, but Relation 10520342 which was edited over 1 year ago in OSM has 2 members.

Relation 11830619 in atlas object only has member 866599848, but Relation 11830619 which was edited 8 months ago in OSM has two members 866599848 and 860916358.

Relation 11830618 in atlas object only has member 866599848, but Relation 11830618 which was edited 8 months ago in OSM has two members 866599848 and 866711504.

Relation 11301517 in atlas object only has member 826078215, but Relation 11301517 which was edited 8 months ago in OSM has 9 members.

Relation 11301518 in atlas object only has member 826078215, but Relation 11301518 which was edited 7 months ago in OSM has 10 members.

Relation 12637000 in atlas object only has member 935676794, but Relation 12637000 which was edited 2 months ago in OSM has 12 members.

Relation 12637001 in atlas object only has member 935676794, but Relation 12637001 which was edited 2 months ago in OSM has 2 members.

Relation 12634238 in atlas object only has member 935676794, but Relation 12634238 which was edited 3 months ago in OSM has 2 members

Relation 12634237 in atlas object only has member 935676794, but Relation 12634237 which was edited 2 months ago in OSM has 8 members

1. Adding getters
   I have been working on a check, where I filter by tag values. Most of them contain getter functions, which helps avoid the NullPointerException. However, LandUse and Location Tags don't have one.
   I am suggestion adding a .get() function, which would simply extract the value of the tag.
   For instance, this would change object.get(LocationTag.KEY) != null to LocationTag.get(object).isPresent(). This is already possible for some tags, such as Waterway and Natural.

2. Modifying .isBridge() function to include "man_made=bridge":
   First idea is to just include it in the BridgeTag code.
   Second idea is to create a .isBridge() function in the ManMadeTag and the users can implement it in this way BridgeTag.isBridge() || ManMadeTag.isBridge().
   Third idea is to combine the two above. I could add a .isBridge() function to the ManMadeTag AND create a .isBridgeIncludingManMade() function to the BridgeTag which would basically return BridgeTag.isBridge() || ManMadeTag.isBridge()

Sometimes, all I need to do is make an atlas file with an OSM PBF file. Right now, I write some Java code like so:

final File pbfFile = new File(pbf);
final OsmPbfLoader loader = new OsmPbfLoader(pbfFile, MultiPolygon.MAXIMUM,
                AtlasLoadingOption.withNoFilter());
final Atlas atlas = loader.read();
final File atlasFile = new File("output.atlas");
atlas.save(atlasFile);

Instead, it would be nice to just do this from the command line and not have to write any code...

This might be already there. If so, I'm happy to add instructions to the readme.

Per discussion in PR #276, there are two ideas to try here:

1. Would it ever be possible to have subAtlas(Polygon) call subAtlas(matcher) underneath? It feels like the should really do the same thing, but subAtlas(Polygon) is a convenience method.

2. Does it make sense to have a subAtlas base class or interface? If 1 above can be done, there can be a default implementation for subAtlas(Polygon) and there could be a centralized source for logging and the packaging of the packed atlas.

I am creating an atlas check to flag any Highways that intersect with Railways that do not have a node at the intersection. I found that railway intersection nodes that exist in the OSM pbf might not exist in the atlas file. This will cause this atlas check to incorrectly flag that a node is missing when the node actually exists in the OSM pbf.

The specific case where this happens is when the highway intersects with a railway at a node that does not determine the shape of that highway and/or railway. Below is a link to a simple example of a highway that, in OSM pbf, has four nodes (the two ends and two railway intersection nodes) and in the atlas file this same polyLine has only two locations (the two ends).
OSM: https://www.openstreetmap.org/way/548339489
ATLAS: [Edge: id=548339489000000, startNode=5297396595000000, endNode=26525806000000, polyLine=LINESTRING (-0.3574737 49.1997678, -0.358776 49.1995892), [Tags: [last_edit_user_name => Julien Travert], [last_edit_changeset => 54812073], [last_edit_time => 1513855751000], [last_edit_user_id => 3629087], [iso_country_code => FRA], [highway => residential], [last_edit_version => 1], [oneway => yes]]]

What we would like to propose is that all OSM nodes that are intersections of two ways be added to the atlas files and be included in the polyLines that correspond to the ways that intersect. If that is too liberal of an algorithm then all highway and railway intersections be added to the atlas file.

Even though railways are not routable (today) we think that it probably makes sense to include all railway intersections so that routing functions can at least be warned about that fact that a railway intersects with the highway.

The TurnRestriction class needs unit tests to check that restrictions are created when the via members include Edges.

This should prevent NullPointerException: #185

This would add needed test coverage here.

Is there a way to write PBF by spark

Your project, osmlab/atlas, depends on the outdated library FastClasspathScanner in the following source files:

• src/main/java/org/openstreetmap/atlas/tags/annotations/validation/Validators.java
• src/main/java/org/openstreetmap/atlas/utilities/runtime/FlexibleCommand.java
• src/main/java/org/openstreetmap/atlas/tags/annotations/validation/TagDocumenter.java

FastClasspathScanner has been significantly reworked since the version your code depends upon:

• a significant number of bugs have been fixed
• some nontrivial API changes have been made to simplify and unify the API
• FastClasspathScanner has been renamed to ClassGraph: https://github.com/classgraph/classgraph

ClassGraph is a significantly more robust library than FastClasspathScanner, and is more future-proof. All future development work will be focused on ClassGraph, and FastClasspathScanner will see no future development.

Please consider porting your code over to the new ClassGraph API, particularly if your project is in production or has downstream dependencies:

• https://github.com/classgraph/classgraph/wiki/Porting-FastClasspathScanner-code-to-ClassGraph
• https://github.com/classgraph/classgraph/wiki/Code-examples
• https://github.com/classgraph/classgraph/wiki/ClassGraph-API

Feel free to close this bug report if this code is no longer in use. (You were sent this bug report because your project depends upon FastClasspathScanner, and has been starred by 81 users. Apologies if this bug report is not helpful.)

There are some corner cases related to SyntheticBoundaryNodeTag assignment that are handled in the legacy OsmPbfProcessor.

• Walk ways outside boundaries
  • The Edges that span out of a country and do not belong to any other country need to be added to the current country build, as long as they are connected to at least one Edge that is inside the boundary.
  • That should cover for bridges and ferries that span outside of country boundaries.
• Add synthetic boundary node tags at the end of those ways
  • All the above mentioned ways that are added end at some node. That node needs to have a synthetic boundary node tag, as that tag is not added right at the boundary.
  • Avoid the bug fixed here: #71

#373 updated Segment.intersects() to fix a bug where single point intersections would sometimes be mis-computed, due to a precision error. However, it did not update the companion method, Segment.intersection(), that returns the actual intersections. So Segment.intersects() can now return true but Segment.intersection() will return null for the same segments. This can create NullPointerErrors when the methods are used in conjunction.

The current system to load Osmosis PBF files into the Atlas format (and perform boundary slicing and way sectioning) requires multiple passes through the PBF file, which could be faster.

It would be nice to load the PBF files once only, and then do the slicing/sectioning using the Atlas APIs directly, which would provide spatial and geometry access.

The current OSM PBF to Atlas flow is optimized for sharded PBF files. The process is very memory intensive and results in OutOfMemoryError exceptions for large PBF files. There needs to be a way to support any type of PBF files, irrespective of size. Here is one possible option:

• Given a PBF file location and a sharding tree, shard the PBF file, produce an Atlas for each shard and produce either sharded Atlas output or as a single Atlas file (multi-atlas the sharded atlases and clone into a PackedAtlas)

• If no sharding tree is provided, fall back to a slippy tile zoom level and flat sharding case, then follow the same output strategy as outlined above.

This is loosely related to issue #88. An example of a reported use-case can be found here.

The method for calculating the width of a rectangle is flawed in the case of Rectangles as wide as Rectangle.MAX(Longitude -180 to 180). With the change of Longitude.MAX being changed to +180, width is calculated 180-(-180) = 360 which gets truncated in Angle class construction to be in the range [0,360). This truncated angle is returned.

As a consequence, Rectangle.MAX.surface() == 0 as well.

From #4

It would be nice to have support for allMatch in StreamIterable