b2pweb / bdf-prime-indexer
Indexer plugin for Prime
Requires
- php: ~7.4 | ~8.0.0 | ~8.1.0 | ~8.2.0 | ~8.3.0
- b2pweb/bdf-collections: ^1.1.4
- b2pweb/bdf-prime: ~2.0
- elasticsearch/elasticsearch: ~7.0|~8.0
Requires (Dev)
- b2pweb/bdf-prime-bundle: ~1.0
- phpunit/phpunit: ~9.0
- squizlabs/php_codesniffer: ~3.0
- symfony/console: ~4.0|~5.0
- symfony/framework-bundle: ~4.0|~5.0
- symfony/messenger: ~4.0|~5.0
- symfony/yaml: ~4.0|~5.0
- vimeo/psalm: ~4.7
This package is auto-updated.
Last update: 2024-10-15 09:46:14 UTC
README
Indexing entities through prime, and request from Elasticsearch index.
Installation
Install with composer :
composer require b2pweb/bdf-prime-indexer
Register into config/bundles.php
:
<?php return [ // ... Bdf\Prime\Indexer\Bundle\PrimeIndexerBundle::class => ['all' => true], Bdf\PrimeBundle\PrimeBundle::class => ['all' => true], ];
Configure indexes into config/packages/prime_indexer.yaml
:
prime_indexer: elasticsearch: # Define elasticsearch hosts hosts: ['127.0.0.1:9222'] # Define indexes in form [Entity class]: [Index configuration class] # This is not mandatory if autoconfiguration is enabled indexes: App\Entities\City: App\Entities\CityIndex App\Entities\User: App\Entities\UserIndex
Usage
Declaring index
For declaring an index, you should first declare the configuration :
<?php class CityIndex implements ElasticsearchIndexConfigurationInterface { // Declare the index name public function index(): string { return 'test_cities'; } // Get the mapped entity type public function entity(): string { return City::class; } // Build properties public function properties(PropertiesBuilder $builder): void { $builder ->string('name') ->integer('population') ->string('zipCode') ->string('country')->notAnalyzed() ->boolean('enabled') ; } // The id accessor public function id(): ?PropertyAccessorInterface { return new SimplePropertyAccessor('id'); } // Declare analyzers public function analyzers(): array { return [ 'default' => [ 'type' => 'custom', 'tokenizer' => 'standard', 'filter' => ['lowercase', 'asciifolding'], ], ]; } // Scopes public function scopes(): array { return [ // "default" scope is always applied to the query 'default' => function (ElasticsearchQuery $query) { $query ->wrap( (new FunctionScoreQuery()) ->addFunction('field_value_factor', [ 'field' => 'population', 'factor' => 1, 'modifier' => 'log1p' ]) ->scoreMode('multiply') ) ->filter('enabled', true) ; }, // Other scope : can be used as custom filter on query // Or using $index->myScope() 'matchName' => function (ElasticsearchQuery $query, string $name) { $query ->where(new Match('name', $name)) ->orWhere( (new QueryString($name.'%')) ->and() ->defaultField('name') ->analyzeWildcard() ->useLikeSyntax() ) ; } ]; } }
Some extra configuration can be added by implementing interfaces :
CustomEntitiesConfigurationInterface
: For define the entities loading methodShouldBeIndexedConfigurationInterface
: For define predicate which check if an entity should be indexed or not
After that, the index can be added to the "prime_indexer.indexes" configuration, or let the autoconfiguration do the job.
Querying the index
The query system use Prime interfaces, so usage is almost the same :
<?php // Get the City index $index = $container->get(\Bdf\Prime\Indexer\IndexFactory::class)->for(City::class); // Get the query $query = $index->query(); $query ->where('country', 'FR') // Simple where works as expected ->where('name', ':like', 'P%') // "like" operator is supported ->orWhere(new QueryString('my complete query')) // Operator object can be used for more powerful filters ; // Get all cities who match with filters $query->all(); // First returns the first matching element, wrapped into an Optional $query->first()->get(); // Get the raw result of the elasticsearch query $query->execute(); // Use scope directly $index->matchName('Paris')->all(); // Same as above, but with scope as filter $index->query()->where('matchName', 'Paris')->all();
Updating the index
Update operations can be done on the index manually :
<?php // Get the City index $index = $container->get(\Bdf\Prime\Indexer\IndexFactory::class)->for(City::class); // Create the index, and insert all cities from database $index->create(City::walk()); $paris = new City([ 'name' => 'Paris', 'population' => 2201578, 'country' => 'FR', 'zipCode' => '75000' ]); // Indexing the city $index->add($paris); // The "id" property is filled after insertion echo $paris->id(); // Make sure that index is up to date // !!! Do not use on production !!! $index->refresh(); $index->contains($paris); // true // Update one attribute $paris->setPopulation(2201984); $index->update($paris, ['population']); // Remove the entity $index->remove($paris); $index->contains($paris); // false // Drop index $index->drop();
With CLI
Create index, and indexing entities :
bin/console.php prime:indexer:create App\Entities\City
A progress bar will be displayed for follow the indexing progress.
Note: The full qualified class name of the entity must be used as argument.
For manage Elasticsearch index :
bin/console.php elasticsearch:show
bin/console.php elasticsearch:delete test_cities
Testing
Because testing is one of more important things, an utility class is added for this :
Note : The index name will be prefixed by "test_" to ensure that it will not impact the real index.
<?php class MyTest extends \Bdf\PHPUnit\WebTestCase { /** * @var TestingIndexer */ private $indexTester; protected function setUp(): void { parent::setUp(); $this->indexTester = new TestingIndexer($this->app); $this->indexTester->index(City::class); // Declare the city index } protected function tearDown() : void { parent::tearDown(); $this->indexTester->destroy(); } public function test_city_index() { // Push entities to index $this->indexTester->push([ new City(...), new City(...), new City(...), ]); // Remove from index $this->indexTester->remove(new City(...)); // Querying to the index $query = $this->indexTester->index(City::class)->query(); } }
Interactions and differences with Prime
- Prime is not required to be registered for use index system. Some entities can be into an index, but not in database.
- Unlike Prime, the mapping is index-oriented and not model-oriented :
- The PropertiesBuilder define the index properties, and maps to the model ones
- Computed properties are permitted (i.e. properties not stored into the entity)
- Query filters columns are not mapped, and use the indexed ones
- Queries use streams (from b2pweb/bdf-collections), so first() returns an OptionalInterface, and transformation are done on the stream