README

Created as part of inspishop e-commerce platform by inspirum team.

Memcached cache store implementation for Laravel framework optimized to work with Mcrouter.

Static cache for tags to reduces the number of queries to the Memcached server
Support for Mcrouter prefix routing
Optimized to be used in Kubernetes cluster with Memcached server on each node to achieve the lowest latency

System requirements

Installation

composer require inspirum/mcrouter

This package supports Laravel 5.3 or later (including Laravel 6/7/8).

For Laravel 5.4 and below it necessary to register the service provider in config/app.php.

'providers' => [
  // ...
  Inspirum\Mcrouter\Providers\McrouterServiceProvider::class,
]

On newer versions Laravel will automatically register via Package Discovery.

Config Files

In order to edit the default configuration you may execute:

php artisan vendor:publish --provider="Inspirum\Mcrouter\Providers\McrouterServiceProvider"

After that, config/mcrouter.php will be created.

<?php

return [
  'shared_prefix' => '/default/shr/',
  'prefixes'      => [
    '/default/a/',
    '/default/b/',
    '/default/c/',
  ],
];

Or you can used environment variables:

CACHE_MCROUTER_SHARED_PREFIX='/default/shr/'
CACHE_MCROUTER_PREFIXES='/default/a/,/default/b/,/default/c/'

Usage example

Cache tags are automatically prefixed with Mcrouter shared prefix.

CACHE_PREFIX='__prefix__'
CACHE_MCROUTER_SHARED_PREFIX='/default/shr/'

cache()->tags(['bop', 'zap'])->get('foo');

get /default/shr/__prefix__:tag:bop:key
get /default/shr/__prefix__:tag:zap:key
get __prefix__:foo

Package support additional prefixes which can be used on Mcrouter routing prefix.

CACHE_PREFIX='__prefix__'
CACHE_MCROUTER_PREFIXES='/default/a/,/default/b/'

cache()->get('/default/a/foo');
cache()->get('/default/b/foo');
cache()->get('/default/c/foo');

get /default/a/__prefix__:foo
get /default/b/__prefix__:foo
get __prefix__:/default/c/foo

Cache tags static cache

cache()->tags(['bop', 'zap'])->get('bar1');
cache()->tags(['bop'])->get('bar2');
cache()->tags(['bop', 'zap', 'foo'])->get('bar3');

get tag:bop:key
get tag:zap:key
get bar1
get bar2
get tag:foo:key
get bar3

Mcrouter configuration

This configuration example is for multiple Memcached servers, one of which is local, such as a typical Kubernetes cluster. We only want to use the local server (on the same node as pod), if possible, to achieve the lowest latency, but to invalidate the cache key on each server.

Tagged cache flush method (cache()->tags(['bop', 'zap'])->flush()) do not use delete operation on Memcached server but update tag cached values instead.

All operations with shared prefix (/default/shr/) and all delete operations are send to each nodes with AllFastestRoute handle, rest of the operations are send only to local server(s) with PoolRoute handle.

Instead of AllFastestRoute you can use AllSyncRoute or AllAsyncRoute handle.

{
  "pools": {
    "local": {
      "servers": [
        "127.0.0.1:11211"
      ]
    },
    "nodes": {
      "servers": [
        "10.80.10.1:11211",
        "10.80.10.2:11211",
        "10.80.10.3:11211"
      ]
    }
  },
  "routes": [
    {
      "aliases": [
        "/default/local/"
      ],
      "route": {
        "type": "OperationSelectorRoute",
        "default_policy": "PoolRoute|local",
        "operation_policies": {
          "delete": "AllFastestRoute|Pool|nodes"
        }
      }
    },
    {
      "aliases": [
        "/default/shr/"
      ],
      "route": "AllFastestRoute|Pool|nodes"
    }
  ]
}

Kubernetes example

Example of Memcached with Mcrouter used on Kuberentes cluster with three nodes (10.80.10.1, 10.80.10.2, 10.80.10.3).

Using DaemonSet resource ensures that Memcached and Mcrouter will be available on each server (on ports 11211 and 11212).

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: memcached
spec:
  template:
    spec:
      containers:
        - name: memcached
          image: memcached:1.5-alpine
          command: ["memcached"]
          ports:
            - name: memcache
              containerPort: 11211
              hostPort: 11211

apiVersion: v1
kind: ConfigMap
metadata:
  name: mcrouter
data:
  config.json: |-
    {
      "pools": {
        "local": {
          "servers": [
            "$HOST_IP:11211"
          ]
        },
        "nodes": {
          "servers": [
            "10.80.10.1:11211",
            "10.80.10.2:11211",
            "10.80.10.3:11211"
          ]
        }
      },
      "routes": [
        {
          "aliases": [
            "/default/local/"
          ],
          "route": {
            "type": "OperationSelectorRoute",
            "default_policy": "PoolRoute|local",
            "operation_policies": {
              "delete": "AllFastestRoute|Pool|nodes"
            }
          }
        },
        {
          "aliases": [
            "/default/shr/"
          ],
          "route": "AllFastestRoute|Pool|nodes"
        }
      ]
    }

kind: DaemonSet
metadata:
  name: mcrouter
spec:
  template:
    spec:
      volumes:
        - name: config
          emptyDir: {}
        - name: config-stub
          configMap:
            name: mcrouter
      initContainers:
        - name: config-init
          image: alpine:latest
          imagePullPolicy: Always
          command: ['sh', '-c', 'cp /tmp/mcrouter/config.json /etc/mcrouter/config.json && sed -i "s|\$HOST_IP|${HOST_IP}|g" /etc/mcrouter/config.json']
          env:
            - name: HOST_IP
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
          volumeMounts:
            - name: config-stub
              mountPath: /tmp/mcrouter
            - name: config
              mountPath: /etc/mcrouter
      containers:
        - name: mcrouter
          image: mcrouter/mcrouter:latest
          imagePullPolicy: IfNotPresent
          command: ["mcrouter"]
          args:
            - --port=11212
            - --config-file=/etc/mcrouter/config.json
            - --route-prefix=/default/local/
            - --send-invalid-route-to-default
          volumeMounts:
            - name: config
              mountPath: /etc/mcrouter
          ports:
            - name: mcrouter
              containerPort: 11212
              hostPort: 11212

You can use status.hostIP to inject current node IP to pod to use to connect to local Memcached/Mcrouter server.

kind: Deployment
metadata:
  name: example
spec:
  template:
    spec:
      containers:
        - name: example
          image: alpine:latest
          env:
            - name: MCROUTER_HOST
              valueFrom:
                fieldRef:
                  fieldPath: status.hostIP
            - name: MCROUTER_PORT
              value: "11212"

Testing

To run unit tests, run:

composer test:test

To show coverage, run:

composer test:coverage

Contributing

Please see CONTRIBUTING and CODE_OF_CONDUCT for details.

Security

If you discover any security related issues, please email tomas.novotny@inspirum.cz instead of using the issue tracker.

Credits

License

The MIT License (MIT). Please see License File for more information.

inspirum / mcrouter

Maintainers

Details