gendoria/cruftflake

Unique ID generator based on Twitter Snowflake, but in PHP with a simple ZeroMQ interface.

2.2.2 2016-05-31 14:20 UTC

Requires

• php: >=5.3.9
• ext-zmq: *
• psr/log: ~1.0
• symfony/console: >=2.5

Requires (Dev)

• doctrine/dbal: =2.3.0
• phpunit/dbunit: ~1.4.0
• symfony/console: =2.5

Suggests

• doctrine/dbal: Allows configuration using database through doctrine DBAL.

MIT

• Gendoria
• Tomasz Struczyński
• Dave Gardner
• Davegardnerisme

• cruftflake

README

Unique ID generator based on Twitter Snowflake, but in PHP with a simple ZeroMQ interface (rather than Thrift).

This is a rewrite of dvomedia/cruftflake. It organizes the code in modules and adds several interfaces, allowing easier extension of other server and configuration mechanisms.

Implementation

The implementation copies Twitter - generating 64 bit IDs.

• time - 41 bits
• configured machine ID - 10 bits
• sequence number - 12 bits

Has a custom epoch that means it can generate IDs until 2081-09-06 (not the same epoch as Snowflake).

ZooKeeper for config coordination

We use ZooKeeper to store which machine IDs are in use. When a new node starts up for the first time it must be able to contact the ZooKeeper cluster and create a new node. It will look at all the existing nodes and then (if it can't find its own Mac Address) attempt to claim a free one.

I was using Ephemeral nodes for this - similar(ish) to a lock pattern but this had the issue that the node needed to remain connected to ZK throughout its lifetime -- this way it doesn't.

The downside is that potentially the 1024 possible machine IDs will "fill up" and need to be manually pruned.

Running

Installation via composer:

	{
    	"require": {
	        "gendoria/cruftflake": "*"
		}
	}

There are several example scripts provided for playing about with. Both require previous composer update.

1. The generator (the server)

       ./examples/server.php

2. A client, that will generate N IDs and dump to STDOUT

   ./examples/client.php -n 100

3. A client, that will ask server for generator status

   ./examples/status.php

For client examples to work, server example has to be be running.

Dependencies

• ZeroMQ
• ZooKeeper (if you want to use ZooKeeper centralized configuration)
• Doctrine DBAL >= 2.3.0 (if you want Doctrine DBAL centralized configuration)

Composer requires php-zmq module installed.