geoid – Efficient storage and queriying of geographic data
1 Storing geoids in a SQLite database
2 API Documentation
first-valid-geoid
last-valid-geoid
sentinel-geoid
valid-geoid?
lat-lng->geoid
geoid->lat-lng
geoid-level
geoid-stride
enclosing-geoid
split-geoid
leaf-geoid?
leaf-span
contains-geoid?
leaf-corners
leaf-outline
lat-lng-rect
random-geoid
geoid->sqlite-integer
sqlite-integer->geoid
7.8

geoid – Efficient storage and queriying of geographic data

Alex Harsányi

 (require geoid) package: geoid

The geoid library allows representing latidude/longitude coordinates using 64-bit integers. The integer values are ordered to preseve locality, which means that these IDs can be stored in databases and used to query and efficiently retrieve locations which are in a geographic region.

This library is inspired by the S2 Geometry library, in the sense that the integer IDs are determined in the same way. Until this libary has its own introduction section, you can read S2 Cell Hierarchy to understand how geoids work. What this library calls geoids, are called cells by the S2 libary.

Also, while inspired by the S2 library, this Racket module is an independent implementation that does not generate compatible IDs and does not aim to have the same API and functionality as that library.

1 Storing geoids in a SQLite database

Geoids use all 64 bits, and more than half of them have the highest bit set. SQLite will store numbers as signed 64 bits and convert unsigned 64 bit numbers to 64 bit floating points.

To store geoids in a SQLLite database, you need to subtract 263 from it, to convert it to a signed number. This will preserve the ordering properties of the geoid. Just remember to add back 263 back to it before using a geoid retrived from the database this way.

The geoid->sqlite-integer and sqlite-integer->geoid functions can be used to convert to and from SQLite representation.

2 API Documentation

Return the first and last valid integers that represent geoids. All integers between these two numbers are valid geoids, but they can be at different levels. Note that 0 is not a valid geoid.

See geoid-stride to determine the amount to add to a geoid to obtain the next geoid at the same level.

Returns an integer which is one bigger than the largest valid geoid (as returned by last-valid-geoid). The returned value is not a valid geoid.

This can be used to create half-open geoid ranges, for example by leaf-span.

procedure

(valid-geoid? geoid)  boolean?

  geoid : exact-integer?
Return true if geoid is a valid geoid. This is the same as testing that the geoid is between first-valid-geoid and last-valid-geoid, but shorter to type.

procedure

(lat-lng->geoid lat lng)  exact-integer?

  lat : real?
  lng : real?
(geoid->lat-lng geoid)  
real? real?
  geoid : exact-integer?
Convert a geographic coordinate (latidude, longitude) to and from a geoid. The returned geoid will be at the highest level of precision (level 0).

The conversion from latitude/longitude to geoid and back has a small amount of error, emprirical testing showed this to be less than 7 millimeters, which is sufficient for the type of applications intended for this libary. Note that this error is not constant accross the globe, and 7 millimeters is the maximum error seen.

procedure

(geoid-level geoid)  (integer-in 0 30)

  geoid : exact-integer?
Return the level of this geoid – geoids can represent areas at increasing level of detail. The highest resolution is at level 0, and going up, geoids become 4 times bigger at each level. Level 30 is the top level, where the entire Earth surface is divided into 6 faces.

Geoids produced by lat-lng->geoid are at level 0 and you can use enclosing-geoid to obtain a geoid at a higher level.

procedure

(geoid-stride geoid)  exact-integer?

  geoid : exact-integer?
Return the integer step that must be added to a geoid to obtain another geoid at the same level. This can be used to generate valid geoids in sequence. Note that the geoids at the highest resolution (level 2) have a stride of 2, so you cannot simply incremenet geoids.

procedure

(enclosing-geoid geoid level)  exact-integer?

  geoid : exact-integer?
  level : (integer-in 0 30)
Return the geoid at level that contains the specified geoid. This function can be used to obtain the geoid at a lower resolution which contains a given point or geoid.

Return the four geoids at the lower level into which the current geoid can be split. An error is raised, if the supplied geoid is a leaf geoid, at level 0.

The returned geoids are not in any particular order.

procedure

(leaf-geoid? geoid)  boolean?

  geoid : exact-integer?
Return #t if geoid is a geoid at level 0 (highest level of detail). This is equivalent to checking if geoid-level returns 0, but faster.

procedure

(leaf-span geoid)  
exact-integer? exact-integer?
  geoid : exact-integer?
Returns the half-open geoid range that are valid geoids contained in geoid. The first value is the smallest leaf geoid which is inside this geoid, the second value is either the smallest leaf geoid which is not part of geoid, or the sentinel-geoid, whichever is smaller.

All geoids which are inside this geoid, regardless of level, are contained in the returned number range, so this range can be used to check if any geoid is inside this one.

The leaf span returned by this function can be used to search for geoids in an SQL query, however, if you do that, make sure you adjust them, as noted in the SQLite section above.

procedure

(contains-geoid? this-geoid other-geoid)  boolean?

  this-geoid : exact-integer?
  other-geoid : exact-integer?
Return true if the other-geoid is geographically inside this-geoid.

This a convenient function, but if you need to check lots of geoids, this will be slower than obtainging the leaf-span of this-geoid and checking of other geoids are inside the returned range.

Return the four leaf geoids which represent the corners of this geoid. The corners are returned in couter-clockwise order, but there is no guarante of which one is first (i.e. there is no guarantee that the list will start with the top-left corner)

procedure

(leaf-outline geoid    
  [#:steps steps    
  #:closed? closed?])  (listof exact-integer?)
  geoid : exact-integer?
  steps : (and/c integer? positive?) = 10
  closed? : boolean? = #t
Return a sequence of leaf geoids representing the outline of geoid. This can be used to obtain the outline of a geoid to display on a map.

steps specifies the number of geoids ot put on each side of the rectangle, while closed? specifies if the first geoid should be duplicated as the last element in the list, to close the loop.

As with leaf-corners, geoids are placed in counter-clockwise order, but there is no guarantee about the start corner of the loop.

procedure

(lat-lng-rect geoid)  
real? real? real? real?
  geoid : exact-integer?
Return the latitude / longitude rectangle which encloses the geoid. The four values returned are: minimum latitude, minimum longitude, maximum latitude, maximum longitude. The bounds are slightly extended, to ensure all leaf geoids are inside the bounding box.

The bounding box encloses the geoid minimally, but geoids and bounding boxes don’t overlap exactly, so the bounding box will always be bigger than then geoid and there will be other geoids which are inside the bounding box, but not in the geoid.

procedure

(random-geoid level #:parent parent)  exact-integer?

  level : exact-integer?
  parent : (or/c exact-integer #f)
Generate a random geoid at the specified level. If parent is specified, the level has to be lower (mode detailed) than the parent and a geoid which is inside the parent will be generated.

This function is intended to generate geoids for unit tests.

Convert a geoid into an integer suitable for storage into a SQLite database, or convert an integer stored in the database back into a geoid.

Geoids are unsigned 64 bit values, and SQLite will only store signed values. Unsigned 64 bit values which are greater than the maximum signed 64 bit value will be converted to floating point numbers, loosing precision. This means that geoids cannot be stored directly into a SQLite database.

These pair of functions subtract 263 from the geoid (or add that value back) to make sure the value is stored correctly. The ordering of the geoids is preserved, so they can still be used to index geograpic information.