nekman / es-pagination
Deep pagination for the Elasticsearch client
Installs: 2 564
Dependents: 0
Suggesters: 0
Security: 0
Stars: 1
Watchers: 2
Forks: 1
Open Issues: 0
Requires
- php: ^7.4||^8.0
- elasticsearch/elasticsearch: ^7.17||^8.0
Requires (Dev)
- friendsofphp/php-cs-fixer: ^2.16
- php-coveralls/php-coveralls: ^2.4
- phpunit/phpunit: ^9.4
README
A library to deep paginate an Elasticsearch search operation. There are three ways to paginate:
Which one to use depends on the context, read more in the Elasticsearch documentation.
The library will get pageSize
amount of hits in memory at the same time, which means a lower amount will result in less memory used but more requests to Elasticsearch (and the opposite). Never will it fully exhaust
an index before returning the results.
Usage
The first step is to construct an $elasticsearchClient
(instance of Elasticsearch\Client
) which you can read more about in the Elasticsearch official PHP driver.
Scroll
use Nekman\EsPagination\CursorFactories\EsScrollCursorFactory; $cursorFactory = new EsScrollCursorFactory( $elasticsearchClient, $pageSize = 1000, $scrollDuration = "1m" ); $params = [ /* * Same params as a normal Elasticsearch search operation. * See Elasticsearch documentation for more information. */ ]; $cursor = $cursorFactory->hits($params); foreach ($cursor as $hit) { echo "Hit {$hit['_id']}"; }
From
use Nekman\EsPagination\CursorFactories\EsFromCursorFactory; $cursorFactory = new EsFromCursorFactory( $elasticsearchClient, $pageSize = 1000 ); $params = [ /* * Same params as a normal Elasticsearch search operation. * See Elasticsearch documentation for more information. */ ]; $cursor = $cursorFactory->hits($params); foreach ($cursor as $hit) { echo "Hit {$hit['_id']}"; }
Search after
use Nekman\EsPagination\CursorFactories\EsSearchAfterCursorFactory; $cursorFactory = new EsSearchAfterCursorFactory( $elasticsearchClient, $pageSize = 1000 ); $params = [ /* * Same params as a normal Elasticsearch search operation. * See Elasticsearch documentation for more information. */ ]; $cursor = $cursorFactory->hits($params); foreach ($cursor as $hit) { echo "Hit {$hit['_id']}"; }
Point in time (PIT)
Elasticsearch pit (point in time) is a lightweight view into the state of the data as it existed when initiated. Create a cursor factory and decorate it with PIT:
use \Nekman\EsPagination\CursorFactories\EsPitCursorFactory; $cursorFactory = /* Create cursor factory, see above */; $pitCursorFactory = new EsPitCursorFactory( $cursorFactory, $elasticsearchFactory, $pitKeepAlive = "1m" ); $params = [ /* * Same params as a normal Elasticsearch search operation. * See Elasticsearch documentation for more information. */ ]; $cursor = $cursorFactory->hits($params); foreach ($cursor as $hit) { echo "Hit {$hit['_id']}"; }
Versioning
This project complies with Semantic Versioning.
Changelog
For a complete list of changes, and how to migrate between major versions, see releases page.