Merge branch 'feature/add-pelias-geocoder' into 'master'
Feature/add pelias geocoder See merge request framasoft/mobilizon!324
This commit is contained in:
Thomas Citharel
commit
83fcf2b62e
@ -140,6 +140,9 @@ config :mobilizon, Mobilizon.Service.Geospatial.MapQuest,
|
||||
config :mobilizon, Mobilizon.Service.Geospatial.Mimirsbrunn,
|
||||
endpoint: System.get_env("GEOSPATIAL_MIMIRSBRUNN_ENDPOINT") || nil
|
||||
|
||||
config :mobilizon, Mobilizon.Service.Geospatial.Pelias,
|
||||
endpoint: System.get_env("GEOSPATIAL_PELIAS_ENDPOINT") || nil
|
||||
|
||||
config :mobilizon, Oban,
|
||||
repo: Mobilizon.Storage.Repo,
|
||||
prune: {:maxlen, 10_000},
|
||||
|
||||
78
docs/administration/configure/geocoders.md
Normal file
78
docs/administration/configure/geocoders.md
Normal file
@ -0,0 +1,78 @@
|
||||
# Geocoders
|
||||
|
||||
Geocoding is the ability to match an input **string representing a location - such as an address - to geographical coordinates**.
|
||||
Reverse geocoding is logically the opposite, matching **geographical coordinates to names of places**.
|
||||
|
||||
This is needed to set correct address for events, and more easily find events with geographical data, for instance if you want to discover events happening near your current position.
|
||||
|
||||
However, providing a geocoding service is quite expensive, especially if you want to cover the whole Earth.
|
||||
|
||||
!!!note
|
||||
To give an idea of what hardware is required to self-host a geocoding service, we successfully used Addok, Pelias and Mimirsbrunn on a 8 core/16GB RAM machine without any issues **on French data**.
|
||||
|
||||
## List of supported geocoders
|
||||
|
||||
This is the list of all geocoders supported by Mobilizon. The current default one is [Nominatim](#nominatim) and uses the official OpenStreetMap instance.
|
||||
|
||||
!!! bug
|
||||
Changing geocoder through `.env` configuration isn't currently supported by Mobilizon.
|
||||
Instead you need to edit the following line in `config.prod.exs`:
|
||||
```elixir
|
||||
config :mobilizon, Mobilizon.Service.Geospatial, service: Mobilizon.Service.Geospatial.Nominatim
|
||||
```
|
||||
And change `Nominatim` to one of the supported geocoders. This change might be overwritten when updating Mobilizon.
|
||||
|
||||
### Nominatim
|
||||
|
||||
[Nominatim](https://wiki.openstreetmap.org/wiki/Nominatim) is a GPL-2.0 licenced tool to search data by name and address. It's written in C and PHP and uses PostgreSQL.
|
||||
It's the current default search tool on the [OpenStreetMap homepage](https://www.openstreetmap.org).
|
||||
|
||||
!!! warning
|
||||
When using the official Nominatim OpenStreetMap instance (default endpoint for this geocoder if not configured otherwise), you need to read and accept the [Usage Policy](https://operations.osmfoundation.org/policies/nominatim).
|
||||
|
||||
Several companies provide hosted instances of Nominatim that you can query via an API, for example see [MapQuest Open Initiative](https://developer.mapquest.com/documentation/open/nominatim-search).
|
||||
|
||||
### Addok
|
||||
|
||||
[Addok](https://github.com/addok/addok) is a WTFPL licenced search engine for address (and only address). It's written in Python and uses Redis.
|
||||
It's used by French government for [adresse.data.gouv.fr](https://adresse.data.gouv.fr).
|
||||
|
||||
!!! warning
|
||||
When using France's Addok instance at `api-adresse.data.gouv.fr` (default endpoint for this geocoder if not configured otherwise), you need to read and accept the [GCU](https://adresse.data.gouv.fr/cgu) (in French).
|
||||
|
||||
### Photon
|
||||
|
||||
[Photon](https://photon.komoot.de/) is an Apache 2.0 licenced search engine written in Java and powered by ElasticSearch.
|
||||
|
||||
!!! warning
|
||||
The terms of use for the official instance (default endpoint for this geocoder if not configured otherwise) are simply the following:
|
||||
> You can use the API for your project, but please be fair - extensive usage will be throttled. We do not guarantee for the availability and usage might be subject of change in the future.
|
||||
|
||||
### Pelias
|
||||
|
||||
[Pelias](https://github.com/pelias/pelias) is a MIT licensed geocoder composed of several services written in NodeJS. It's powered by ElasticSearch.
|
||||
|
||||
There's [Geocode Earth](https://geocode.earth/) SAAS that provides a Pelias API.
|
||||
They offer discounts for Open-Source projects. [See the pricing](https://geocode.earth/).
|
||||
|
||||
### Mimirsbrunn
|
||||
|
||||
[Mimirsbrunn](https://github.com/CanalTP/mimirsbrunn) is an AGPL-3.0 licensed geocoding written in Rust and powered by ElasticSearch.
|
||||
|
||||
Mimirsbrunn is used by [Qwant Maps](https://www.qwant.com/maps) and [Navitia](https://www.navitia.io).
|
||||
|
||||
### Google Maps
|
||||
|
||||
[Google Maps](https://developers.google.com/maps/documentation/geocoding/intro) is a proprietary service that provides APIs for geocoding.
|
||||
|
||||
They don't have a free plan, but offer credit when creating a new account. [See the pricing](https://cloud.google.com/maps-platform/pricing/).
|
||||
|
||||
### MapQuest
|
||||
|
||||
[MapQuest](https://developer.mapquest.com/documentation/open/geocoding-api/) is a proprietary service that provides APIs for geocoding.
|
||||
|
||||
They offer a free plan. [See the pricing](https://developer.mapquest.com/plans).
|
||||
|
||||
### More geocoding services
|
||||
|
||||
Geocoding implementations are simple modules that need to implement the [`Mobilizon.Service.Geospatial.Provider` behaviour](https://framasoft.frama.io/mobilizon/backend/Mobilizon.Service.Geospatial.Provider.html), so feel free to write your own!
|
||||
@ -25,7 +25,8 @@
|
||||
<template slot="empty">
|
||||
<span v-if="isFetching">{{ $t('Searching…') }}</span>
|
||||
<div v-else class="is-enabled">
|
||||
<span>{{ $t('No results for "{queryText}". You can try another search term or drag and drop the marker on the map', { queryText }) }}</span>
|
||||
<span>{{ $t('No results for "{queryText}"') }}</span>
|
||||
<span>{{ $t('You can try another search term or drag and drop the marker on the map', { queryText }) }}</span>
|
||||
<!-- <p class="control" @click="openNewAddressModal">-->
|
||||
<!-- <button type="button" class="button is-primary">{{ $t('Add') }}</button>-->
|
||||
<!-- </p>-->
|
||||
|
||||
@ -71,7 +71,6 @@
|
||||
"Didn't receive the instructions ?": "Bestätigung nicht erhalten?",
|
||||
"Display name": "Namen einzeigen",
|
||||
"Display participation price": "Teilnahmegebühr anzeigen",
|
||||
"Displayed name": "Angezeigter Name",
|
||||
"Draft": "Entwurf",
|
||||
"Drafts": "Entwürfe",
|
||||
"Edit": "Bearbeiten",
|
||||
@ -162,7 +161,6 @@
|
||||
"No group found": "Keine Gruppe gefunden",
|
||||
"No groups found": "Keine Gruppen gefunden",
|
||||
"No results for \"{queryText}\"": "Keine Ergebnisse für \"{queryText}\"",
|
||||
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "Keine Ergebinsse für \"{queryText}\". Du kannst es erneut mit anderen Begriffen versuchen, oder mit Drag&Drop den Marker auf die Karte setzen",
|
||||
"No user account with this email was found. Maybe you made a typo?": "Kein Account mit dieser E-Mail gefunden. Vielleicht hast Du dich vertippt?",
|
||||
"Number of places": "Anzahl der Plätze",
|
||||
"OK": "OK",
|
||||
|
||||
@ -71,7 +71,6 @@
|
||||
"Didn't receive the instructions ?": "Didn't receive the instructions ?",
|
||||
"Display name": "Display name",
|
||||
"Display participation price": "Display participation price",
|
||||
"Displayed name": "Displayed name",
|
||||
"Draft": "Draft",
|
||||
"Drafts": "Drafts",
|
||||
"Edit": "Edit",
|
||||
@ -161,7 +160,7 @@
|
||||
"No events found": "No events found",
|
||||
"No group found": "No group found",
|
||||
"No groups found": "No groups found",
|
||||
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map",
|
||||
"No results for \"{queryText}\"": "No results for \"{queryText}\"",
|
||||
"No user account with this email was found. Maybe you made a typo?": "No user account with this email was found. Maybe you made a typo?",
|
||||
"Number of places": "Number of places",
|
||||
"OK": "OK",
|
||||
@ -172,6 +171,7 @@
|
||||
"On {date}": "On {date}",
|
||||
"One person is going": "No one is going | One person is going | {approved} persons are going",
|
||||
"Only accessible through link and search (private)": "Only accessible through link and search (private)",
|
||||
"Only alphanumeric characters and underscores are supported.": "Only alphanumeric characters and underscores are supported.",
|
||||
"Opened reports": "Opened reports",
|
||||
"Organized by {name}": "Organized by {name}",
|
||||
"Organized": "Organized",
|
||||
@ -298,6 +298,7 @@
|
||||
"You are already a participant of this event.": "You are already a participant of this event.",
|
||||
"You are already logged-in.": "You are already logged-in.",
|
||||
"You can add tags by hitting the Enter key or by adding a comma": "You can add tags by hitting the Enter key or by adding a comma",
|
||||
"You can try another search term or drag and drop the marker on the map": "You can try another search term or drag and drop the marker on the map",
|
||||
"You can't remove your last identity.": "You can't remove your last identity.",
|
||||
"You have been disconnected": "You have been disconnected",
|
||||
"You have cancelled your participation": "You have cancelled your participation",
|
||||
|
||||
@ -71,7 +71,6 @@
|
||||
"Didn't receive the instructions ?": "Vous n'avez pas reçu les instructions ?",
|
||||
"Display name": "Nom affiché",
|
||||
"Display participation price": "Afficher un prix de participation",
|
||||
"Displayed name": "Nom affiché",
|
||||
"Draft": "Brouillon",
|
||||
"Drafts": "Brouillons",
|
||||
"Edit": "Éditer",
|
||||
@ -162,7 +161,6 @@
|
||||
"No group found": "Aucun groupe trouvé",
|
||||
"No groups found": "Aucun groupe trouvé",
|
||||
"No results for \"{queryText}\"": "Pas de résultats pour « {queryText} »",
|
||||
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "Pas de résultats pour « {queryText} ». Vous pouvez essayer avec d'autres termes de recherche ou bien glisser et déposer le marqueur sur la carte",
|
||||
"No user account with this email was found. Maybe you made a typo?": "Aucun compte utilisateur trouvé pour cet email. Peut-être avez-vous fait une faute de frappe ?",
|
||||
"Number of places": "Nombre de places",
|
||||
"OK": "OK",
|
||||
@ -173,6 +171,7 @@
|
||||
"On {date}": "Le {date}",
|
||||
"One person is going": "Personne n'y va | Une personne y va | {approved} personnes y vont",
|
||||
"Only accessible through link and search (private)": "Uniquement accessibles par lien et la recherche (privé)",
|
||||
"Only alphanumeric characters and underscores are supported.": "Seuls les caractères alphanumériques et les tirets bas sont acceptés.",
|
||||
"Opened reports": "Signalements ouverts",
|
||||
"Organized by {name}": "Organisé par {name}",
|
||||
"Organized": "Organisés",
|
||||
@ -300,6 +299,7 @@
|
||||
"You are already a participant of this event.": "Vous participez déjà à cet événement.",
|
||||
"You are already logged-in.": "Vous êtes déjà connecté.",
|
||||
"You can add tags by hitting the Enter key or by adding a comma": "Vous pouvez ajouter des tags en appuyant sur la touche Entrée ou bien en ajoutant une virgule",
|
||||
"You can try another search term or drag and drop the marker on the map": "Vous pouvez essayer avec d'autres termes de recherche ou bien glisser et déposer le marqueur sur la carte",
|
||||
"You can't remove your last identity.": "Vous ne pouvez pas supprimer votre dernière identité.",
|
||||
"You have been disconnected": "Vous avez été déconnecté⋅e",
|
||||
"You have cancelled your participation": "Vous avez annulé votre participation",
|
||||
|
||||
@ -71,7 +71,6 @@
|
||||
"Didn't receive the instructions ?": "Hebt u de instructies niet ontvangen?",
|
||||
"Display name": "Getoonde naam",
|
||||
"Display participation price": "Prijs voor deelname tonen",
|
||||
"Displayed name": "Getoonde naam",
|
||||
"Draft": "Concept",
|
||||
"Drafts": "Concepten",
|
||||
"Edit": "Bewerken",
|
||||
|
||||
@ -76,7 +76,6 @@
|
||||
"Disallow promoting on Mobilizon": "Refusar la promocion sus Mobilizon",
|
||||
"Display name": "Nom mostrat",
|
||||
"Display participation price": "Far veire un prètz de participacion",
|
||||
"Displayed name": "Nom mostrat",
|
||||
"Do you want to participate in {title}?": "Volètz participar a {title} ?",
|
||||
"Draft": "Borrolhon",
|
||||
"Drafts": "Borrolhons",
|
||||
@ -184,7 +183,6 @@
|
||||
"No groups found": "Cap de grop pas trobat",
|
||||
"No participants yet": "Cap de participant pel moment",
|
||||
"No results for \"{queryText}\"": "Cap de resultats per « {queryText} »",
|
||||
"No results for \"{queryText}\". You can try another search term or drag and drop the marker on the map": "Cap de resultat per « {queryText} ». Podètz ensajar un autre tèrme de recèrca o botar lo marcador sus la mapa",
|
||||
"No user account with this email was found. Maybe you made a typo?": "Pas de compte utilizaire pas trobat amb aquesta adreça. Benlèu qu’avètz fach una deca ?",
|
||||
"Number of places": "Nombre de plaças",
|
||||
"OK": "OK",
|
||||
|
||||
@ -69,7 +69,6 @@
|
||||
"Didn't receive the instructions ?": "Nie otrzymałeś(-aś) instrukcji?",
|
||||
"Display name": "Wyświetlana nazwa",
|
||||
"Display participation price": "Wyświetlaj cenę udziału",
|
||||
"Displayed name": "Wyświetlana nazwa",
|
||||
"Draft": "Szkic",
|
||||
"Drafts": "Szkice",
|
||||
"Edit": "Edytuj",
|
||||
|
||||
@ -71,7 +71,6 @@
|
||||
"Didn't receive the instructions ?": "Fick inte instruktionerna?",
|
||||
"Display name": "Visa namn",
|
||||
"Display participation price": "Visa pris för deltagande",
|
||||
"Displayed name": "Visat namn",
|
||||
"Draft": "Utkast",
|
||||
"Drafts": "Utkast",
|
||||
"Edit": "Redigera",
|
||||
|
||||
@ -81,8 +81,10 @@ export class Address implements IAddress {
|
||||
}
|
||||
} else if (this.locality && this.locality.trim()) {
|
||||
alternativeName = `${this.locality}, ${this.region}, ${this.country}`;
|
||||
} else {
|
||||
} else if (this.region && this.region.trim()) {
|
||||
alternativeName = `${this.region}, ${this.country}`;
|
||||
} else if (this.country && this.country.trim()) {
|
||||
alternativeName = this.country;
|
||||
}
|
||||
poiIcon = this.iconForPOI;
|
||||
break;
|
||||
|
||||
128
lib/service/geospatial/pelias.ex
Normal file
128
lib/service/geospatial/pelias.ex
Normal file
@ -0,0 +1,128 @@
|
||||
defmodule Mobilizon.Service.Geospatial.Pelias do
|
||||
@moduledoc """
|
||||
[Pelias](https://pelias.io) backend.
|
||||
|
||||
Doesn't provide type of POI.
|
||||
"""
|
||||
|
||||
alias Mobilizon.Addresses.Address
|
||||
alias Mobilizon.Service.Geospatial.Provider
|
||||
alias Mobilizon.Config
|
||||
|
||||
require Logger
|
||||
|
||||
@behaviour Provider
|
||||
|
||||
@endpoint Application.get_env(:mobilizon, __MODULE__) |> get_in([:endpoint])
|
||||
|
||||
@impl Provider
|
||||
@doc """
|
||||
Pelias implementation for `c:Mobilizon.Service.Geospatial.Provider.geocode/3`.
|
||||
"""
|
||||
@spec geocode(number(), number(), keyword()) :: list(Address.t())
|
||||
def geocode(lon, lat, options \\ []) do
|
||||
user_agent = Keyword.get(options, :user_agent, Config.instance_user_agent())
|
||||
headers = [{"User-Agent", user_agent}]
|
||||
url = build_url(:geocode, %{lon: lon, lat: lat}, options)
|
||||
Logger.debug("Asking Pelias for reverse geocoding with #{url}")
|
||||
|
||||
with {:ok, %HTTPoison.Response{status_code: 200, body: body}} <-
|
||||
HTTPoison.get(url, headers),
|
||||
{:ok, %{"features" => features}} <- Poison.decode(body) do
|
||||
process_data(features)
|
||||
end
|
||||
end
|
||||
|
||||
@impl Provider
|
||||
@doc """
|
||||
Pelias implementation for `c:Mobilizon.Service.Geospatial.Provider.search/2`.
|
||||
"""
|
||||
@spec search(String.t(), keyword()) :: list(Address.t())
|
||||
def search(q, options \\ []) do
|
||||
user_agent = Keyword.get(options, :user_agent, Config.instance_user_agent())
|
||||
headers = [{"User-Agent", user_agent}]
|
||||
url = build_url(:search, %{q: q}, options)
|
||||
Logger.debug("Asking Pelias for addresses with #{url}")
|
||||
|
||||
with {:ok, %HTTPoison.Response{status_code: 200, body: body}} <-
|
||||
HTTPoison.get(url, headers),
|
||||
{:ok, %{"features" => features}} <- Poison.decode(body) do
|
||||
process_data(features)
|
||||
end
|
||||
end
|
||||
|
||||
@spec build_url(atom(), map(), list()) :: String.t()
|
||||
defp build_url(method, args, options) do
|
||||
limit = Keyword.get(options, :limit, 10)
|
||||
lang = Keyword.get(options, :lang, "en")
|
||||
coords = Keyword.get(options, :coords, nil)
|
||||
endpoint = Keyword.get(options, :endpoint, @endpoint)
|
||||
country_code = Keyword.get(options, :country_code)
|
||||
|
||||
url =
|
||||
case method do
|
||||
:search ->
|
||||
url =
|
||||
"#{endpoint}/v1/autocomplete?text=#{URI.encode(args.q)}&lang=#{lang}&size=#{limit}"
|
||||
|
||||
if is_nil(coords),
|
||||
do: url,
|
||||
else: url <> "&focus.point.lat=#{coords.lat}&focus.point.lon=#{coords.lon}"
|
||||
|
||||
:geocode ->
|
||||
"#{endpoint}/v1/reverse?point.lon=#{args.lon}&point.lat=#{args.lat}"
|
||||
end
|
||||
|
||||
if is_nil(country_code), do: url, else: "#{url}&boundary.country=#{country_code}"
|
||||
end
|
||||
|
||||
defp process_data(features) do
|
||||
features
|
||||
|> Enum.map(fn %{
|
||||
"geometry" => %{"coordinates" => coordinates},
|
||||
"properties" => properties
|
||||
} ->
|
||||
address = process_address(properties)
|
||||
%Address{address | geom: Provider.coordinates(coordinates)}
|
||||
end)
|
||||
end
|
||||
|
||||
defp process_address(properties) do
|
||||
%Address{
|
||||
country: Map.get(properties, "country"),
|
||||
locality: Map.get(properties, "locality"),
|
||||
region: Map.get(properties, "region"),
|
||||
description: Map.get(properties, "name"),
|
||||
postal_code: Map.get(properties, "postalcode"),
|
||||
street: street_address(properties),
|
||||
origin_id: "pelias:#{Map.get(properties, "id")}",
|
||||
type: get_type(properties)
|
||||
}
|
||||
end
|
||||
|
||||
defp street_address(properties) do
|
||||
if Map.has_key?(properties, "housenumber") do
|
||||
"#{Map.get(properties, "housenumber")} #{Map.get(properties, "street")}"
|
||||
else
|
||||
Map.get(properties, "street")
|
||||
end
|
||||
end
|
||||
|
||||
@administrative_layers [
|
||||
"neighbourhood",
|
||||
"borough",
|
||||
"localadmin",
|
||||
"locality",
|
||||
"county",
|
||||
"macrocounty",
|
||||
"region",
|
||||
"macroregion",
|
||||
"dependency"
|
||||
]
|
||||
|
||||
defp get_type(%{"layer" => layer}) when layer in @administrative_layers, do: "administrative"
|
||||
defp get_type(%{"layer" => "address"}), do: "house"
|
||||
defp get_type(%{"layer" => "street"}), do: "street"
|
||||
defp get_type(%{"layer" => "venue"}), do: "venue"
|
||||
defp get_type(%{"layer" => _}), do: nil
|
||||
end
|
||||
@ -9,11 +9,13 @@ defmodule Mobilizon.Service.Geospatial.Provider do
|
||||
* `Mobilizon.Service.Geospatial.Addok` [🔗](https://github.com/addok/addok)
|
||||
* `Mobilizon.Service.Geospatial.MapQuest` [🔗](https://developer.mapquest.com/documentation/open/)
|
||||
* `Mobilizon.Service.Geospatial.GoogleMaps` [🔗](https://developers.google.com/maps/documentation/geocoding/intro)
|
||||
* `Mobilizon.Service.Geospatial.Mimirsbrunn` [🔗](https://github.com/CanalTP/mimirsbrunn)
|
||||
* `Mobilizon.Service.Geospatial.Pelias` [🔗](https://pelias.io)
|
||||
|
||||
|
||||
## Shared options
|
||||
|
||||
* `:user_agent` User-Agent string to send to the backend. Defaults to `"Mobilizon"`
|
||||
* `:user_agent` User-Agent string to send to the backend. Defaults to `"Mobilizon"` or `Mobilizon.Config.instance_user_agent/0`
|
||||
* `:lang` Lang in which to prefer results. Used as a request parameter or
|
||||
through an `Accept-Language` HTTP header. Defaults to `"en"`.
|
||||
* `:country_code` An ISO 3166 country code. String or `nil`
|
||||
@ -31,7 +33,10 @@ defmodule Mobilizon.Service.Geospatial.Provider do
|
||||
|
||||
## Options
|
||||
|
||||
Most backends implement all of [the shared options](#module-shared-options).
|
||||
In addition to [the shared options](#module-shared-options), `c:geocode/3` also
|
||||
accepts the following options:
|
||||
|
||||
* `zoom` Level of detail required for the address. Default: 15
|
||||
|
||||
## Examples
|
||||
|
||||
|
||||
12
mkdocs.yml
12
mkdocs.yml
@ -33,15 +33,3 @@ theme:
|
||||
icon: 'calendar_today'
|
||||
feature:
|
||||
tabs: true
|
||||
#
|
||||
#nav:
|
||||
# - Home: 'index.md'
|
||||
# - About: 'about.md'
|
||||
# - Administration:
|
||||
# - Install: 'administration/install.md'
|
||||
# - Dependencies: 'administration/dependencies.md'
|
||||
# - Docker: 'administration/docker.md'
|
||||
# - Contribute:
|
||||
# - Contribute: 'contribute.md'
|
||||
# - Development: 'contribute/development.md'
|
||||
# - Styleguide:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user