High-throughput non-concurrent NoSQL key-value file-based database with a memory-backed index.

1.0.7 2021-12-21 13:06 UTC

This package is auto-updated.

Last update: 2022-01-21 13:18:26 UTC


IndexedFile is a DB system allowing to very quickly process and store big amount of data by leveraging memory as an index while keeping all the other data on disk with high-throughput read and write engine.

To simplify snippets below, its assumed that the following namespaces are in use:

use \Simpletools\Db\IndexedFile;

Initialise Temp DB

To start a new temp DB which will get removed when the script terminates:

$indexedFile = new IndexedFile\File();

Initialise from existing DB

To start DB which might have already exists or which should persist after script terminate:

$indexedFile = new IndexedFile\File('/path/to/my/db.jsonl');

Setup a custom IndexStore

You can write your own IndexStore which implements IndexStoreInterface and preset it with the following static method:

//the default IndexStore

Storing data

Inserting/Replacing data by key


Storing data if not exists

Ignoring insert if key already exists


Reading data

Reading by key

$value = $indexedFile->read('key');

Iterating through all entries

Iterating through all entires

foreach($indexedFile->iterate() as $key => $value)

Upserting data

Updating/Inserting your data

        return $row;
        return [
            'counter'   => 0

Removing data

Removing a key


Truncating database

Removing all entires


or when booting up

$indexedFile = new IndexedFile\File('/path/to/my/db.jsonl',true);


Declare text sort

$indexedFile->sort('title','ASC', 'string');

Declare integer sort

$indexedFile->sort('count','DESC', 'int');

includeSortStats flag adds to the indexed object _sort meta field with position in sort and percent of total (integer only)

$indexedFile->sort('count','DESC', 'int', true);

Own input and output files

$indexedFile->sort('title','ASC', 'string',true, __DIR__.'/unsorted.csv', __DIR__.'/sorted.csv');

Example of use for sorting by price and list with an iterator

$indexedFile->sort('price','DESC', 'int');

    'title' => $data['title'],
    'price' => $data['price']


foreach ($indexedFile->sortIterate() as $groupId => $row)
  echo 'Position: '. $row->_sort->position.' Percent: '.$row->_sort->position.' Index key: '. $groupId .' Price: '. $row->price."\n";