README

Minimalist, self-hosted geocoding using datasets from national registry.

Installation

You can install the package via Composer:

composer require yossuf/laravel-geocoding

You may publish all of the package's resources at once:

php artisan vendor:publish --tag="laravel-geocoding"

Or, you may publish each resource individually:

Publishing the Configuration File

php artisan vendor:publish --tag="laravel-geocoding-config"

The published config/geocoding.php covers the search bounds and the maintenance jobs, each with an environment variable:

return [
    // The most results a search may ask for and the longest query it takes.
    // A search outside either bound throws an InvalidArgumentException.
    'geocode' => [
        'max_limit' => (int) env('GEOCODING_MAX_LIMIT', 100),
        'max_query_length' => (int) env('GEOCODING_MAX_QUERY_LENGTH', 255),
    ],

    // How many records one queued maintenance job handles, such as the
    // addresses recomputed after a city or street is renamed.
    'maintenance' => [
        'chunk_size' => (int) env('GEOCODING_MAINTENANCE_CHUNK_SIZE', 1000),
    ],
];

GEOCODING_MAX_LIMIT=100
GEOCODING_MAX_QUERY_LENGTH=255
GEOCODING_MAINTENANCE_CHUNK_SIZE=1000

Running the Migrations

The package does not load its migrations itself, so publish them before migrating:

php artisan vendor:publish --tag="laravel-geocoding-migrations"
php artisan migrate

This creates the GEOCODING_COUNTRY, GEOCODING_STATE, GEOCODING_CITY, GEOCODING_DISTRICT, GEOCODING_STREET and GEOCODING_ADDRESS tables.

The CSV imports below run on yossuf/laravel-import, which is installed alongside. Publish its migrations as well before migrating to create its IMPORT tracking table:

php artisan vendor:publish --tag="laravel-import-migrations"
php artisan migrate

Publish its config with --tag="laravel-import-config" to change the engine's defaults.

The imports also need Laravel's job_batches table, part of the default create_jobs_table migration or created by php artisan make:queue-batches-table, and a cache store that supports locks: redis, memcached, database, dynamodb or file. The array store's locks last for one process only, so it suits tests, not queue workers.

Usage

1. Model the address hierarchy

The package models addresses as Country -> City -> Street -> Address, with State as an optional level between Country and City and District as an optional level between City and Street:

use Yossuf\Laravel\Geocoding\Models\Country;

$country = Country::create([
    'name' => 'Slovakia',
    'native_name' => 'Slovensko',
    'iso_code_alpha2' => 'SK',
    'iso_code_alpha3' => 'SVK',
    'capital' => 'Bratislava',
    'phone_prefix' => '421',
]);

$city = $country->cities()->create(['name' => 'Bratislava']);
$street = $city->streets()->create(['name' => 'Hlavná']);

$street->addresses()->create([
    'house_number' => '10',
    'street_number' => '12A',
    'postal_code' => '811 01',
]);

A name is unique within its parent: one state name per country, one city name per state, one district name per city and one street name per district. The database enforces it, which is what keeps two imports running at the same time from creating the same city twice. A city without a state or a street without a district sits outside that guarantee, because databases treat each null as distinct.

The formatted address, Hlavná 10/12A, 811 01 Bratislava for the example above, is generated whenever an address is saved and is what search matches against. It is not mass assignable, and a value set directly is rebuilt on save.

Renaming a Street or City queues a DispatchRecomputeFormattedAddress job, which batches RecomputeFormattedAddress jobs of GEOCODING_MAINTENANCE_CHUNK_SIZE addresses each (1000 by default) to recompute the formatted text of every address beneath the renamed row. A city can hold thousands of addresses, which is why this runs on your queue instead of blocking the rename. Renaming the same row again before the queue picks the job up queues nothing new, since the job reads the names current when it runs. Run a queue worker (php artisan queue:work) in production, or set QUEUE_CONNECTION=sync to recompute inline.

Changing a Country's iso_code_alpha2 queues a ReindexCountryAddresses job when Meilisearch is active, since the index carries the code and the addresses do not. It hands the country's addresses to Scout in chunks of the same size and writes nothing to the database. Under the database fallback it queues nothing.

A row cannot be deleted while anything beneath it still points at it: the foreign keys are restricted, so deleting a street with addresses, or a country with states or cities, fails with a QueryException. Delete or reassign the children first. Nothing cascades, so the search index stays in step with the addresses.

2. Import countries

The package ships one Artisan command, geocoding:import, whose first argument names what to import. Import countries from a CSV file:

php artisan geocoding:import countries /path/to/countries.csv

The file must use the header row of the countries dataset:

"Code","Code3","Name","Native","Phone","Continent","Capital","Currency","Languages","ISO3166-1"
"SK","SVK","Slovakia","Slovensko","421","Europe","Bratislava","EUR","sk","1"

Headers are snake cased and empty values become null. Each row is mapped onto a country as Code -> iso_code_alpha2, Code3 -> iso_code_alpha3, Name -> name, Native -> native_name, Capital -> capital and Phone -> phone_prefix. The remaining columns are ignored.

Every row is validated before it is written: Name and Code are required, and the other mapped columns must be present but may be empty. A row that fails validation is skipped and counted, and the rest of the file still imports.

Tracking Progress

The command hands the file to the import engine, which queues it and returns at once, so a queue worker (php artisan queue:work) has to be running. Every queued run creates a tracking record (Yossuf\Laravel\Import\Models\ImportRecord) and prints its id. Poll a fresh copy of it for the import's state (Waiting, Processing, Success or Failed), its row counts and progress(), or listen for the engine's ImportCompleted and ImportFailed events, which carry the settled record. The jobs go to the queue named by IMPORT_QUEUE_CONNECTION and IMPORT_QUEUE. Pass --sync to import inline instead and print the result:

php artisan geocoding:import countries /path/to/countries.csv --sync

Chunks and Bulk Statements

One job streams the file and reads it in chunks. Each chunk is queued as an import job of its own, which writes its rows in bulk upsert statements. The imports read 1000 rows per chunk and write 100 rows per statement, and both sizes can be set per run:

php artisan geocoding:import countries /path/to/countries.csv --chunk-size=1000 --bulk-size=250

Each statement is upserted on iso_code_alpha2, so the command is safe to re-run: an existing country has its name, native_name, iso_code_alpha3, capital and phone_prefix refreshed from the file, keeps its created_at, and a new one is inserted.

Failures

A failing statement fails the import. The tracking record ends up Failed with the error message (a database failure keeps the driver's message only, never the statement with its values), and the chunks written before it stay committed. A --sync run instead executes inside the batch's transaction, so it is rolled back as a whole, and the command reports the failure and exits with a non-zero code.

Concurrent Imports

Concurrent imports are guarded on both ends. The command is isolatable, so --isolated makes it a no-op while another geocoding:import run, of either import, is running. That is what keeps two scheduled --sync imports from overlapping, since a queued run holds the lock only until the import is dispatched:

php artisan geocoding:import countries /path/to/countries.csv --sync --isolated

And the command queues nothing while a tracking record for the same file is still Waiting or Processing, so running it again before the first import has settled is a no-op. A record a lost job leaves in either state blocks the file until you delete it or mark it Failed.

3. Import addresses

Addresses are imported per country from a local CSV file, such as a regional export of the Slovak address register:

php artisan geocoding:import addresses /path/to/addresses.csv --parent=SK

The --parent option names the model the rows belong to. What key it takes is up to the import; for addresses it is the country's ISO 3166-1 alpha-2 code, in either case. Import the country with geocoding:import countries first. The command stops without queueing anything when the option is missing, is not a 2-letter code, names a country that has not been imported, or is given to an import that takes no parent, such as countries.

The Register Export

Nothing is downloaded. The command resolves the path to an absolute one and queues it, and the queue worker opens it, so it has to point at storage the worker can reach. The path is trusted: build it yourself, for example from a Storage disk, never from a filename a request supplied. The file is parsed as CSV whatever its extension.

The file is expected to be semicolon delimited and must use the header row of the address register export:

IDENTIFIKATOR;KRAJ;OKRES;OBEC;CAST_OBCE;ULICA;SUPISNE_CISLO;ORIENTACNE_CISLO_CELE;PSC;ADRBOD_X;ADRBOD_Y
4320536;Banskobystrický;Banská Bystrica;Badín;;Banská;150;2;97632;19,121502;48,6661543

Each row is mapped onto the address hierarchy as:

CSV column	Model
`OKRES`	`State.name`
`OBEC`	`City.name`
`CAST_OBCE`	`District.name` (skipped when empty)
`ULICA`	`Street.name` (falls back to `OBEC` when empty)
`SUPISNE_CISLO`	`Address.house_number`
`ORIENTACNE_CISLO_CELE`	`Address.street_number`
`PSC`	`Address.postal_code`
`ADRBOD_Y`	`Address.latitude`
`ADRBOD_X`	`Address.longitude`
`IDENTIFIKATOR`	`Address.external_ref`

KRAJ is ignored, though the column still has to be present. Coordinates use a decimal comma in the dataset and are converted on the way in. ADRBOD_X is the longitude and ADRBOD_Y the latitude, despite the names. A village with no street name is addressed by its municipality, so a blank ULICA becomes a street named after OBEC and formats as Badín 150, 97632 Badín.

IDENTIFIKATOR, OBEC and SUPISNE_CISLO are required. The remaining mapped columns must be present but may be empty. A row that fails validation is skipped and counted, and the rest of the file still imports.

Queueing

The import is queued the same way as the countries import, and it matters more here because a single regional file holds hundreds of thousands of rows. The command prints the tracking record id and returns, and a queue worker streams the file in chunks of --chunk-size rows (1000 by default), each imported by its own job in upsert statements of --bulk-size rows (100 by default). Pass --sync to import inline instead and print the result:

php artisan geocoding:import addresses /path/to/addresses.csv --parent=SK --sync

States, cities, districts and streets are resolved once per chunk and cached for the rest of it. Each statement is upserted on external_ref, so re-importing a newer export of the same region updates the addresses in place instead of duplicating them. A failing statement fails the import and leaves the chunks written before it committed, while a --sync run is rolled back as a whole. Either way the tracking record ends up Failed with the error message.

Because addresses are written in bulk, Scout is not notified as rows land. Run php artisan scout:import "Yossuf\Laravel\Geocoding\Models\Address" after an import when you search through Meilisearch.

Concurrent Imports

The guards described under Concurrent Imports for the countries import apply here as well: --isolated keeps two runs from overlapping, and queueing the same export again before the first import has settled is a no-op. Different regional files keep importing in parallel:

php artisan geocoding:import addresses /path/to/addresses.csv --parent=SK --isolated

4. Geocode an address

Use the Geocoding facade to resolve a formatted address in the form STREET HOUSE_NUMBER, CITY or STREET HOUSE_NUMBER, POSTAL_CODE CITY. HOUSE_NUMBER may be a single Slovak house number (10) or a house/street number pair (10/12A):

use Yossuf\Laravel\Geocoding\Facades\Geocoding;

$results = Geocoding::geocode('Hlavná 10/12A, 811 01 Bratislava');

// Limit how many results come back (default 10):
$results = Geocoding::geocode('Hlavná 10/12A, 811 01 Bratislava', limit: 25);

// Restrict matches to a single country by its ISO 3166-1 alpha-2 code:
$results = Geocoding::geocode('Hlavná 10/12A, 811 01 Bratislava', countryCode: 'SK');

geocode() returns an Eloquent collection of Address models with their street, district, city, state and country eager loaded.

The query may be at most geocode.max_query_length characters (255 by default), the limit must be between 1 and geocode.max_limit (100 by default) and the country code must be two letters. Anything else throws an InvalidArgumentException, so validate request input against the configured bounds before passing it on:

$validated = $request->validate([
    'query' => ['required', 'string', 'max:'.config('geocoding.geocode.max_query_length')],
    'limit' => ['sometimes', 'integer', 'between:1,'.config('geocoding.geocode.max_limit')],
    'country' => ['sometimes', 'string', 'size:2'],
]);

Searching is typo-tolerant when Laravel Scout is configured with the meilisearch driver (SCOUT_DRIVER=meilisearch), in which case matching is delegated to Meilisearch. Otherwise the package falls back to a fulltext search against the formatted column: NATURAL LANGUAGE MODE on the mysql and mariadb drivers, a to_tsvector match on pgsql, and a plain substring match on other drivers such as SQLite.

To filter by country under Meilisearch, add country_code to the filterableAttributes of the GEOCODING_ADDRESS index (with your scout.prefix, if any) in config/scout.php and run php artisan scout:sync-index-settings first. Until then Meilisearch rejects a search with a countryCode as invalid_search_filter, and Scout raises that as an exception.

Every result carries a confidence(): the ranking score Meilisearch reports for the match, a float between 0 and 1. The database fallback has no equivalent, so confidence() is null there:

foreach (Geocoding::geocode('Hlavná 10, Bratislava') as $address) {
    $address->confidence(); // 0.9873 under Meilisearch, null otherwise
}

5. Autocomplete an address

geocode() eager loads the whole hierarchy of every result. When you only read the address row itself, such as its formatted text for a suggestion list, use autocomplete() instead. It runs the same search with the same arguments and skips loading the hierarchy:

$suggestions = Geocoding::autocomplete('Hlavná', limit: 5, countryCode: 'SK')
    ->pluck('formatted');

6. Register an import of your own

The geocoding:import command runs the imports registered on the Geocoding service, the two above out of the box. Register more from the boot method of one of your service providers, the way $this->commands() registers Artisan commands:

use Yossuf\Laravel\Geocoding\Facades\Geocoding;

public function boot(): void
{
    Geocoding::imports([
        StreetImport::class,
    ]);
}

The import implements Yossuf\Laravel\Geocoding\Contracts\GeocodingImport, the engine's ModelImport contract plus a static name() the command selects it by. One whose rows belong to a parent model, as the addresses belong to a country, implements ImportsIntoParent as well to take the --parent option. Registering an import under the name of a package import replaces it.

License

Laravel Geocoding is open-sourced software licensed under the Apache-2.0 license.

yossuf / laravel-geocoding

Package info

Security

Statistics

README

Installation

Publishing the Configuration File

Running the Migrations

Usage

1. Model the address hierarchy

2. Import countries

Tracking Progress

Chunks and Bulk Statements

Failures

Concurrent Imports

3. Import addresses

The Register Export

Queueing

Concurrent Imports

4. Geocode an address

5. Autocomplete an address

6. Register an import of your own

License