seanhandley / h3_ruby

Ruby Bindings for H3, a hexagonal geospatial indexing system

H3 Ruby

Ruby-to-C bindings for Uber's H3 library.

Please consult the H3 documentation for a full explanation of terminology and concepts.

Supported H3 Versions

The semantic versioning of this gem matches the versioning of the H3 C library. E.g. version 3.5.x of this gem is targeted for version 3.5.y of H3 C lib where x and y are independent patch levels.

Naming Conventions

We have changed camel-case method names to snake-case, as per the Ruby convention.

In addition, some methods using the get verb have been renamed i.e. getH3UnidirectionalEdgesFromHexagon becomes unidirectional_edges_from_hexagon.

We have also suffixed predicate methods with a question mark, as per the Ruby convention, and removed h3Is from the name i.e. h3IsPentagon becomes pentagon?

Getting Started

This gem uses FFI to link directly into the H3 library (written in C).

The H3 library is packaged with the gem and is built as a native extension. H3 is not installed system-wide, so it will not interfere with any other versions you may have installed previously.

Before installing the gem, please install the build dependencies for your system as instructed here: https://github.com/uber/h3#install-build-time-dependencies

Installing

You can install the gem directly from RubyGems.org using

gem install h3

or add it to your Gemfile

# Gemfile
gem "h3", "~> 3.7"

Usage

Require the gem in your code

require "h3"

Call H3 functions via the H3 namespace

H3.from_geo_coordinates([53.959130, -1.079230], 8).to_s(16)
=> "8819429a9dfffff"
H3.valid?("8819429a9dfffff".to_i(16))
=> true
H3.pentagon?("8819429a9dfffff".to_i(16))
=> false
H3.to_boundary("8819429a9dfffff".to_i(16))
=> [[53.962987505331384, -1.079984346847996], [53.9618315234061, -1.0870313428985856], [53.95744798515881, -1.0882421079017874], [53.95422067486053, -1.082406760751464], [53.955376670617454, -1.0753609232787642], [53.95975996282198, -1.074149274503605]]

Documentation

Please read the Gem Documentation for a full reference of available methods.

Development

The development environment requires the H3 library to be compiled from source before tests can be executed.

This is done automatically by the test suite. However, Rake tasks are provided to handle building H3 in a more fine-grained manner.

Building H3

rake build

You can remove the compiled H3 library with rake clean, or rebuild it with rake rebuild.

Running Tests

The test suite exercises all the H3 functions.

rake spec

Be aware that errors may be encountered if you have a locally cached H3 binary that's older than the version targeted. Try rake rebuild and re-run rake spec if this occurs.

Contributing

Pull requests and issues are welcome! Please read the Contributing Guide for more info.

h3_ruby's Issues

Set up Ruby version Matrix on Travis

Ensure gem works for multiple Rubies. JRuby, MRI, Rubinius. Current versions and some older versions too.

NOTE: It may be necessary to modify the ffi_lib call to fall back to the installed lib in case the native extension can't be built (this will be the case on non-MRI Rubies).

Run Rubocop over the whole source locally and make fixes

Release v0.0.1 Preparations

To Do

• Finish wrapping remaining functions.
• Remove rake-compiler dep once all functions are in FFI DSL.
• Declare deps in .gemspec
• Set gemspec to find files with git ls-files.
• Memory leak/segfault safeguarding. Ensure specs try to Do Bad Things.

Make changes in accordance with Code Climate Issues

https://codeclimate.com/repos/5c18b7f49bc79a02a4000d81/issues

Reduce issues to 0.

Document max_hex_ring_size

The method max_hex_ring_size has a mysterious implementation:

k == 0 ? 1 : 6*k

While this method is private, maintainers would thank us if we explain where does that formula come from in a code comment.

Allow hex_ranges to take a flag `grouped` defaults to true

This flag indicates that the output should be grouped or not

Use the correct FFI C types

Some of the H3 functions accept a size_t. FFI has a :size_t type we can use, so we should use it for parity with the C lib.

There are also :buffer_in, :buffer_out, and :buffer_in_out types available. We should use them where appropriate.

See https://github.com/ffi/ffi/wiki/Types

Re-examine the behaviour of hex_ranges

We're currently doing some manual manipulation of the data returned by hex_ranges i.e. grouping them by distance. This is largely because other bindings (namely python and go) take this approach.

The C lib just returns an array without grouping the contents.

Before we release, we should agree what approach we wish to take.

Sort out version handling for h3 lib

Introduce FFI and start rewriting

FFI has many benefits that will reduce likelihood of leaks/crashes and make the library simpler to maintain over time.

Set up CI

Investigate possible uses of rgeo gem and rgeo-kml

https://github.com/rgeo/rgeo
https://github.com/ldonnet/rgeo-kml

Deprecate methods named with `.h3_`

Methods like .h3_pentagon? should become .pentagon? etc

A deprecation warning should be shown when using the older version.

Documentation

https://blog.github.com/2016-08-22-publish-your-project-documentation-with-github-pages/

Enforce uniqueness for functions that expect a set

Some functions expect a set i.e. a C array with no duplicates. Without this check, the behaviour can be undefined according to the docs so it's incumbent on us to ensure we removes duplicates ourselves first.

Remaining functions to wrap

ALL DONE 😁

Add ability to automatically install h3 C

It's possible to install h3 from source as part of the gem installation. We can (and probably should) script this for convenience.

The Python bindings already do this https://github.com/uber/h3-py/blob/master/.install.sh

Rearrange code/specs according to function groups

See here for API reference function groups: https://uber.github.io/h3/#/documentation/api-reference/indexing

• Indexing
• Inspection
• Traversal
• Hierarchy
• Regions
• Unidirectional Edges
• Miscellaneous

The gem's API needn't change, but it would be helpful to split things up so we don't have one large implementation file and one large spec.

Update the gem in line with h3 3.4.4

https://github.com/uber/h3/releases/tag/v3.4.4

Register with rubygems.org

Update for 3.3

https://github.com/uber/h3/releases/tag/v3.3.0

Add custom types with FFI

https://www.varvet.com/blog/advanced-topics-in-ruby-ffi/#writing-our-own-data-types

This could be helpful in preventing some edge case misbehaviours. For instance, a lot of functions take a resolution argument as an integer but only accept values 0-15 (or produce undefined behaviour). e.g.

irb(main):035:0> H3.h3_to_children(517700169958293503, 100000000)
Traceback (most recent call last):
        5: from /Users/sean/.rbenv/versions/2.5.0/bin/irb:11:in `<main>'
        4: from (irb):35
        3: from /Users/sean/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/h3-3.2.0/lib/h3/hierarchy.rb:51:in `h3_to_children'
        2: from /Users/sean/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/h3-3.2.0/lib/h3/hierarchy.rb:51:in `new'
        1: from /Users/sean/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/h3-3.2.0/lib/h3/hierarchy.rb:51:in `initialize'
NoMemoryError (too large allocation size)

In this case, we get told memory can't be allocated - which is ok.

irb(main):036:0> H3.h3_to_children(517700169958293503, 1000000000)
Traceback (most recent call last):
        4: from /Users/sean/.rbenv/versions/2.5.0/bin/irb:11:in `<main>'
        3: from (irb):36
        2: from /Users/sean/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/h3-3.2.0/lib/h3/hierarchy.rb:52:in `h3_to_children'
        1: from /Users/sean/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/h3-3.2.0/lib/h3/hierarchy.rb:52:in `h3_to_children'
SystemStackError (stack level too deep)

Here we blow the stack, which is recoverable

irb(main):037:0> H3.h3_to_children(517700169958293503, 10000000000)
Traceback (most recent call last):
        4: from /Users/sean/.rbenv/versions/2.5.0/bin/irb:11:in `<main>'
        3: from (irb):37
        2: from /Users/sean/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/h3-3.2.0/lib/h3/hierarchy.rb:50:in `h3_to_children'
        1: from /Users/sean/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/h3-3.2.0/lib/h3/hierarchy.rb:50:in `max_h3_to_children_size'
RangeError (integer 10000000000 too big to convert to `int')

Here FFI raises RangeError because the value is too large for a C integer.

irb(main):039:0> H3.h3_to_children(613196570357137407, 1000)
Illegal instruction: 4

Now the real problem! We can crash the Ruby process like this 😱

Setting up some custom types would give us a nice mechanism to validate this kind of data and prevent such nasty behaviour.

Allow h3 indexes to be integers or strings

Make it possible to use hex strings and integers interchangeably when calling public methods that take them.

Need to consider whether we should always return h3 indexes as integers.

Update for 3.4

https://github.com/uber/h3/releases/tag/v3.4.0

Investigate Managed Struct

Pointers are GC'd automatically in FFI but structs may not be.

See https://www.rubydoc.info/github/ffi/ffi/FFI/ManagedStruct and https://www.varvet.com/blog/advanced-topics-in-ruby-ffi/

Also check the FFI wiki.

Add missing tests

Coveralls shows missing test coverage here: https://coveralls.io/builds/20760354/source?filename=lib/h3/geo_json.rb#L162

and here: https://coveralls.io/builds/20760354/source?filename=lib/h3/regions.rb#L158

Memory leak/segfault safeguarding

Ensure specs try to Do Bad Things.

Move camel case functions to a private module

e.g. hexRing is called by wrapping snake case function hex_ring - if we move hexRing into a private module then it prevents users from accidentally calling it.

Document hex_ranges

hex_ranges returns an array or ranges. Each range comes in turn as an array of the concentric hex Rings outwards. The C lib does not have this, so we need to explain what our binding does.