slava-vishnyakov / map-reduce
MapReduce implementation
1.0.3
2018-04-17 18:05 UTC
Requires (Dev)
- phpunit/phpunit: ^6
README
To install:
composer require slava-vishnyakov/map-reduce
use \SlavaVishnyakov\MapReduce\MapReduceMemory; $m = new MapReduceMemory(); // or MapReduceProcess $m->send('a', 1); $m->send('b', 1); $m->send('a', 2); foreach($m->iter() as $key => $groups) { // yields $key = 'a', $groups = [1,2] $key = 'b', $groups = [1]
Also can be done via next() calls:
$m = new MapReduceMemory(); // or MapReduceProcess $m->send('a', 1); $m->send('b', 1); $m->send('a', 2); $m->send('a', new stdClass); $m->next() ==> ['a', [1, 2, new stdClass]] $m->next() ==> ['b', [1]] $m->next() ==> null
There are two implementations MapReduceMemory
and MapReduceProcess
.
The first does all sorting in memory, the second is for memory-hungry workloads, uses /usr/bin/sort
to
process basically unlimited amount of data.