Coder Social home page Coder Social logo

osm_easy_api's Introduction

osm_easy_api

pypi downloads python versions

Me on OpenStreetMap

Python package for parsing osm diffs and communicating with the OpenStreetMap api.

What's the point of this package?

This package was created to provide an easy way to create automated scripts and programs that use diff and/or osm api. The main advantage is the classes (data_classes) that provide data of elements (node, way, relation, OsmChange, etc.) in a readable way and the possibility to use them in diff and api without worrying about missing data or dictionaries. You can easily find nodes in diff, add a tag to them and send the corrected version to osm.

Installation

Works on python >= 3.10. (Due to new typehints standard)

Install osm_easy_api from PyPi:

pip install osm_easy_api

Documentation

You can view documentation on github-pages.

Documentation is build using pdoc. To run docs on your machine use preferred command: pdoc --docformat google --no-show-source osm_easy_api !osm_easy_api.utils.

OAuth 2.0

Due to the deprecation of HTTP Basic Auth you need an access token to use most api endpoints. To obtain an access token we recommend using https://tools.interactivemaps.xyz/token/.

Examples

DIFF

Print trees

from osm_easy_api.diff import Diff, Frequency
from osm_easy_api.data_classes import Node

# Download diff from last hour.
d = Diff(Frequency.HOUR)

# Get Meta namedtuple for diff metadata and generator that parse diff file.
meta, gen = d.get(tags="natural")

# Print all created, modified and deleted Nodes with natural=tree tag.
for action, element in gen:
    if type(element) == Node and element.tags.get("natural") == "tree":
        print(action, element.id)

Print incorrectly tagged single tress

from osm_easy_api.diff import Diff, Frequency
from osm_easy_api.data_classes import Action, Node

d = Diff(Frequency.DAY)

meta, gen = d.get(tags="natural")

for action, element in gen:
    if type(element) == Node:
        if action == Action.CREATE or action == Action.MODIFY:
            if element.tags.get("natural") == "wood":
                print(element)

Example output:

Node(id = 10208486717, visible = None, version = 1, changeset_id = 129216075, timestamp = 2022-11-22T00:16:44Z, user_id = 17471721, tags = {'leaf_type': 'broadleaved', 'natural': 'wood'}, latitude = 48.6522286, longitude = 12.583809, )

API

Add missing wikidata tag

from osm_easy_api.api import Api
from osm_easy_api.data_classes import Node, Tags

api = Api("https://master.apis.dev.openstreetmap.org", ACCESS_TOKEN)

node = api.elements.get(Node, 4296460336) # We are getting Node with id 4296460336 where we want to add a new tag to
node.tags.add("wikidata", "Qexample") # Add a new tag to node.

my_changeset = api.changeset.create("Add missing wikidata tag", Tags({"automatic": "yes"})) # Create new changeset with description and tag
api.elements.update(node, my_changeset) # Send new version of a node to osm
api.changeset.close(my_changeset) # Close changeset.

Notes

Note that the following codes do the same thing

from osm_easy_api.diff import Diff, Frequency

d = Diff(Frequency.DAY)

meta, gen = d.get()

for action, element in gen:
    if element.tags.get("shop") == "convenience":
        print(element)
from osm_easy_api.diff import Diff, Frequency
from osm_easy_api.data_classes import Tags

d = Diff(Frequency.DAY)

meta, gen = d.get(tags=Tags({"shop": "convenience"}))

for action, element in gen:
        print(element)

but the second seems to be faster.

Also you can use OsmChange object if you don't want to use generator

from osm_easy_api.diff import Diff, Frequency
from osm_easy_api.data_classes import Action, Node

d = Diff(Frequency.MINUTE)

osmChange = d.get(generator=False)

deleted_nodes = osmChange.get(Node, Action.DELETE)
for node in deleted_nodes:
    print(node.id)

but it can consume large amounts of ram and use of this method is not recommended for large diff's.

Tests

You will need to install test-requirements.txt. You can use tox. To run tests manually use python -m unittest discover.

osm_easy_api's People

Contributors

dependabot[bot] avatar docentyt avatar matkoniecz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

Forkers

matkoniecz

osm_easy_api's Issues

Maybe show xml on parse failure?

  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/api/endpoints/elements.py", line 165, in history
    for event, elem in generator:
  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/api/api.py", line 65, in _raw_stream_parser
    for event, element in iterator:
  File "/usr/lib/python3.10/xml/etree/ElementTree.py", line 1253, in iterator
    yield from pullparser.read_events()
  File "/usr/lib/python3.10/xml/etree/ElementTree.py", line 1324, in read_events
    raise event
  File "/usr/lib/python3.10/xml/etree/ElementTree.py", line 1296, in feed
    self._parser.feed(data)
xml.etree.ElementTree.ParseError: syntax error: line 1, column 0

I strongly suspect that OSM API returned some error for whatever reason. Would be nice if osm_easy_api would show its contents before crashing.

Support ancient anonymous edits

This tool works really nice, but it has some problems when dealing OSM prehistorical oddities.

Trying to fetch history of https://www.openstreetmap.org/node/25733488/history results in

  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/api/endpoints/elements.py", line 167, in history
    objects_list.append(_element_to_osm_object(elem))
  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/diff/diff_parser.py", line 126, in _element_to_osm_object
    node = _create_node_from_attributes(element.attrib)
  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/diff/diff_parser.py", line 88, in _create_node_from_attributes
    user_id =       int(    attributes["uid"]       ),
KeyError: 'uid'

note, users can be also deleted - it may cause a similar issue

No comment_tag.text in tag 'date'"

api.notes.get_bbox(left = 54.5643516 , bottom= 32.6475314 , right= 54.5643516 , top= 32.6475314 , limit = 10000, closed_days = 0)

crashes with

  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/api/endpoints/notes.py", line 115, in get_bbox                                                                        
    return self._xml_to_note(generator)                                                                                                                                                      
  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/api/endpoints/notes.py", line 40, in _xml_to_note                                                                     
    assert comment_tag.text, "[ERROR::API::ENDPOINTS::NOTE::_xml_to_note] No comment_tag.text in tag 'date'"
AssertionError: [ERROR::API::ENDPOINTS::NOTE::_xml_to_note] No comment_tag.text in tag 'date'

it worked fine recently

TypeError: quote_from_bytes() expected bytes

  File "/home/mateusz/.local/lib/python3.10/site-packages/osm_easy_api/api/endpoints/changeset_discussion.py", line 23, in comment
    response = self.outer._request(self.outer._RequestMethods.POST, self.outer._url.changeset_discussion["comment"].format(id=changeset_id, text=urllib.parse.quote(text)), self.outer._Requirement.YES, auto_status_code_handling=False)
  File "/usr/lib/python3.10/urllib/parse.py", line 881, in quote
    return quote_from_bytes(string, safe)
  File "/usr/lib/python3.10/urllib/parse.py", line 906, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
TypeError: quote_from_bytes() expected bytes

Would it be possible to move _create_node_from_attributes to a public API?

https://github.com/docentYT/osm_easy_api/blob/main/src/osm_easy_api/diff/diff_parser.py#LL79C2-L79C2

This would be useful tool for deserializing cached API responses.

As pickling cannot be safely used in this case, I am using

def deserialize_node(attributes):
    # see https://github.com/docentYT/osm_easy_api/issues/3
    # for request to move _create_node_from_attributes to
    # a public API
    node = diff_parser._create_node_from_attributes({
        'id' :            int(    attributes["id"]        ),
        'visible' :       str(attributes["visible"]).lower(), # 'true' is needed :(
        'version' :       int(    attributes["version"]   ),
        'timestamp' :     str(    attributes["timestamp"] ),
        'uid' :       int(    attributes["user_id"]       ), # inconsistent name :(
        'changeset' :  int(    attributes["changeset_id"] ), # inconsistent name :(
        'lat' :      str(    attributes.get("latitude")   ), # inconsistent name :(
        'lon' :     str(    attributes.get("longitude")   ), # inconsistent name :(
    })
    node.tags = attributes["tags"] # not passed directly
    return node

def serialize_node(node):
    return {
        'type': 'node',
        'id': node.id,
        'visible': node.visible,
        'version': node.version,
        'changeset_id': node.changeset_id,
        'timestamp': node.timestamp,
        'user_id': node.user_id,
        'tags': node.tags,
        'latitude': node.latitude,
        'longitude': node.longitude,
    }

to save responses in a database.

osm_object_primitive.to_dict() does not process relations' members

Given e is a relation, invoke e.to_dict() gives:

e.to_dict()
{'id': 12270672, 'visible': True, 'version': 2, 'changeset_id': 122818285, 'timestamp': '2022-06-24T20:42:22Z', 'user_id': 3421839, 'tags': {'name': '29 路', 'network': '肇庆公交', 'network:wikidata': 'Q111736572', 'network:zh': '肇庆公交', 'ref': '29', 'route_master': 'bus', 'type': 'route_master'}, 'members': [Member(element=Relation(id=12270671, visible=None, version=None, changeset_id=None, timestamp=None, user_id=None, tags={}, members=[]), role=''), Member(element=Relation(id=12270670, visible=None, version=None, changeset_id=None, timestamp=None, user_id=None, tags={}, members=[]), role='')], 'type': 'Relation'}

Notice that e's members is still in Member type, which is not python primitive. That is to say, serialization is not complete.
This prevents deserialization with ast.literal_eval from string.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.