inzh/elasticsearch-high-level-dump

Tool for create or restore high level dump for cluster elasticsearch with search/scroll method

v1.0.0 2022-03-07 00:00 UTC

Requires

• php: >=5.6
• elasticsearch/elasticsearch: >=1.4.1

CC-BY-4.0

• Jean-Raffi Nazareth

• elasticsearch
• bin

README

A tool for creating and restoring dump on cluster Elasticsearch with search/scroll method.

Installation

You can install this tool in your PHP project using composer:

composer require inzh/elasticsearch-high-level-dump

Usage

⚠️ All indexs is read with search/scrool method, and written with bulk method, use with caution for preserve data integrity.

Bash:

You can use this tool in cli context.

For dump process :

./vendor/bin/edump

For restore process :

./vendor/bin/erestore 

All options :

1. The --gzip option Use GZip compression for restore or dump.

2. The -b, --buffer option Set buffer for read on write, big buffer need more memory: 1000

3. The -o, --output option Set output path file or stream, default on standart output: /var/dir/file

4. The -i, --input option Set input path file or stream, default on standart input: /var/dir/file

5. The -es-host option Set Elasticsearch service hostname or ip: localhost

6. The -es-port option Set Elasticsearch service port: 9200

Exemple :

./vendor/bin/edump --gzip -b 1000 -es-host localhost -es-port 9200 > output.json.gz
cat output.json.gz | ./vendor/bin/erestore --gzip -b 1000 -es-host localhost -es-port 9200

./vendor/bin/edump --gzip -b 1000 -es-host localhost -es-port 9200 -o output.json.gz
./vendor/bin/erestore --gzip -b 1000 -es-host localhost -es-port 9200 -i output.json.gz

Development:

You can use directly process class in your code.

For dump process :

use inzh\elasticsearch\dump\HighLevelDump;

$client = $yourInstanceOfClient; // Your elasticsearch client
$options = new \stdClass; // .... See all options in next section

$process = new HighLevelDump($client, $options);
$process->dump();

For restore process :

use inzh\elasticsearch\dump\HighLevelRestore;

$client = $yourInstanceOfClient; // Your elasticsearch client
$options = new \stdClass; // .... See all options in next section

$process = new HighLevelRestore($client, $options);
$process->restore();

All options :

// All options parameters
$options = new \stdClass;
$options->buffer = 1000;
$options->gzip = false;
$options->output = "php://stdout"; // Stream or filePath
$options->input = "php://stdin"; // Stream or filePath
$options->host = "localhost";
$options->port = 9200;

© 2011-2022 [InZH] Studio.