dragoon87 / openaddresses

Pelias import pipeline for OpenAddresses.

This repository is part of the Pelias project. Pelias is an open-source, open-data geocoder built by Mapzen that also powers Mapzen Search. Our official user documentation is here.

Pelias OpenAddresses importer

Overview

The OpenAddresses importer is used to process data from OpenAddresses for import into the Pelias geocoder.

Requirements

Node.js 4 or higher is required.

Installation

git clone https://github.com/pelias/openaddresses
cd openaddresses
npm install

Data Download

Use the imports.openaddresses.files configuration option to limit the download to just the OpenAddresses files of interest. Refer to the OpenAddresses data listing for file names.

see the 'Configuration' section below for a more detailed example of how to use imports.openaddresses.files

npm run download

Usage

# show full command line options
node import.js --help

# run an import
npm start

Admin Lookup

OpenAddresses records do not contain information about which city, state (or other region like province), or country that they belong to. Pelias has the ability to compute these values from Who's on First data. For more info on how admin lookup works, see the documentation for pelias/wof-admin-lookup. By default, adminLookup is enabled. To disable, set imports.adminLookup.enabled to false in Pelias config.

Note: Admin lookup requires loading around 5GB of data into memory.

Configuration

This importer can be configured in pelias-config, in the imports.openaddresses hash. A sample configuration file might look like:

{
  "esclient": {
    "hosts": [
      {
        "env": "development",
        "protocol": "http",
        "host": "localhost",
        "port": 9200
      }
    ]
  },
  "logger": {
    "level": "debug"
  },
  "imports": {
    "whosonfirst": {
      "datapath": "/mnt/data/whosonfirst/",
      "importPostalcodes": false,
      "importVenues": false
    },
    "openaddresses": {
      "datapath": "/mnt/data/openaddresses/",
      "files": [ "us/ny/city_of_new_york.csv" ]
    }
  }
}

The following properties are recognized:

This importer is configured using the pelias-config module. The following configuration options are supported by this importer.

key	required	default	description
datapath	yes		The absolute path of the directory containing OpenAddresses files. Must be specified if no directory is given as a command-line argument.
files	no		An array of the names of the files to download/import. If specified, only these files will be downloaded and imported, rather than all .csv files in the given directory. If the array is empty, all files will be downloaded and imported. Refer to the OpenAddresses data listing for file names.
deduplicate	no	false	Boolean flag to enable deduplication (deprecated. See pelias/address-deduplicator for more info).