README

SilverStripe module to store assets in S3 rather than on the local filesystem.

composer require silverstripe/s3

Warning

This module does not currently implement any kind of bucket policy for protected assets. It is up to you to implement this yourself using AWS bucket policy.

Caution

This replaces the built-in local asset store that comes with SilverStripe with one based on S3. Any files that had previously been uploaded to an existing asset store will be unavailable (though they won't be lost - just run composer remove silverstripe/s3 to remove the module and restore access).

Environment setup

The module requires a few environment variables to be set.

AWS_REGION: The AWS region your S3 bucket is hosted in (e.g. eu-central-1)
AWS_BUCKET_NAME: The name of the S3 bucket to store assets in.

If running outside of an EC2 instance it will be necessary to specify an API key and secret.

AWS_ACCESS_KEY_ID: Your AWS access key that has access to the bucket you want to access
AWS_SECRET_ACCESS_KEY: Your AWS secret corresponding to the access key

Example YML Config when running outside of EC2:

---
Only:
    envvarset: AWS_BUCKET_NAME
After:
    - "#assetsflysystem"
---
SilverStripe\Core\Injector\Injector:
    Aws\S3\S3Client:
        constructor:
            configuration:
                region: "`AWS_REGION`"
                version: latest
                credentials:
                    key: "`AWS_ACCESS_KEY_ID`"
                    secret: "`AWS_SECRET_ACCESS_KEY`"

(Optional) CDN Implementation

If you're serving assets from S3, it's recommended that you utilize CloudFront. This improves performance and security over exposing from S3 directly.

Once you've set up your CloudFront distribution, ensure that assets are reachable within the assets directory of the cdn (for example; https://cdn.example.com/assets/Uploads/file.jpg) and set the following environment variable:

AWS_PUBLIC_CDN_PREFIX: Your CloudFront distribution domain that has access to the bucket you want to access

For example, adding this to your .env:

AWS_PUBLIC_CDN_PREFIX='https://cdn.example.com/'

will change your URLs from something like:

https://s3.ap-southeast-2.amazonaws.com/mycdn/public/example/live/assets/Uploads/file.jpg

to something like:

https://cdn.example.com/assets/Uploads/file.jpg

You can override the default /assets/ path by declaring the PublicCDNAdapter constructor, with the parameter for the cdnAssetsDir set to a string of your folder name. In your app/_config/assets.yml file add the following:

---
Name: app#silverstripes3-cdn
Only:
    envvarset: AWS_PUBLIC_CDN_PREFIX
After:
    - "#assetsflysystem"
    - "#silverstripes3-flysystem"
---
SilverStripe\Core\Injector\Injector:
    SilverStripe\Assets\Flysystem\PublicAdapter:
        class: SilverStripe\S3\Adapter\PublicCDNAdapter
        constructor:
            s3Client: '%$Aws\S3\S3Client'
            bucket: "`AWS_BUCKET_NAME`"
            prefix: "`AWS_PUBLIC_BUCKET_PREFIX`"
            visibility: null
            mimeTypeDetector: null
            cdnPrefix: "`AWS_PUBLIC_CDN_PREFIX`"
            options: []
            cdnAssetsDir: "cms-assets" # example of a custom assets folder name

Editing images with a CDN

The image editor added in Silverstripe CMS 6.3 replaces an image under its existing filename. As the URL would not change, your CDN would keep serving the cached image from before the edit.

When AWS_PUBLIC_CDN_PREFIX is set, this module does not overwrite a published image. The edited image is added to your bucket under a new versioned filename and used instead, so bird.jpg becomes bird-v2.jpg.

If you choose to back up the original image, the backup keeps the name bird.jpg and uses the original file where it is, so no copy is made and its URL keeps working. Without a backup, the original file is removed when you publish the edit.

Configuration

Assets are classed as either 'public' or 'protected' by SilverStripe. Public assets can be freely downloaded, whereas protected assets (e.g. assets not yet published) shouldn't be directly accessed.

The module supports this by streaming the contents of protected files down to the browser via the web server (as opposed to linking to S3 directly) by default. To ensure that protected assets can't be accessed, ensure you setup an appropriate bucket policy (see below for an untested example).

Configuring S3

The 'protected' S3 asset store should be protected using standard AWS IAM policies that disallow all access to anonymous users, but still allow the action s3:GetObject for both public and protected files. Protected files will be streamed from AWS, so they do not need to be accessed by users directly. Therefore, something similar to the following bucket policy may be useful.

Make sure you replace <bucket-name> below with the appropriate values.

Note: The below policy has not been extensively tested - feedback welcome.

{
    "Policy": {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "AddPerm",
                "Effect": "Allow",
                "Principal": "*",
                "Action": "s3:GetObject",
                "Resource": "arn:aws:s3:::<bucket-name>/public/*"
            }
        ]
    }
}

If you are utilizing a CloudFront distribution for your public assets, you will have the option of securing your S3 bucket against all public access while still allowing access to your public files via your CloudFront distribution and access to your protected files via signed URLs.

For developers

Read Setting up a local sandbox for developing the Silverstripe S3 module if you wish to do some local development.

Performance

This module comes with a basic in-memory cache for calls to S3. It is highly recommended to add an additional layer of caching to achieve the best results.

See https://docs.silverstripe.org/en/5/developer_guides/performance/caching/ for more information.

Name: silverstripes3-flysystem-memcached
After:
    - "#silverstripes3-flysystem"
---
SilverStripe\Core\Injector\Injector:
    MemcachedClient:
        class: "Memcached"
        calls:
            - [addServer, ["localhost", 11211]]
    MemcachedCacheFactory:
        class: 'SilverStripe\Core\Cache\MemcachedCacheFactory'
        constructor:
            client: "%$MemcachedClient"
    SilverStripe\Core\Cache\CacheFactory: "%$MemcachedCacheFactory"

File hashes and image dimensions

Silverstripe checks the SHA1 hash of a file when resolving its URL, and reads the dimensions of an image when rendering it. With an empty cache that would mean downloading the file from S3, so this module stores both as metadata on each object as it is written (sha1, plus width and height for images) and reads them back with a HEAD request instead.

Files uploaded before this was introduced are given their metadata the first time their hash or dimensions are needed: the object is downloaded once, as it always was, and then copied onto itself with the metadata added, keeping its content type, cache control and visibility. Nothing needs to be run for this. If requests shouldn't write to the bucket, turn it off, and such files are simply downloaded to be hashed or measured as before:

SilverStripe\S3\Adapter\CachedAwsS3V3Adapter:
  lazy_metadata_backfill: false

To update every file up front instead of as they are used, run the backfill task.

# See what would change
vendor/bin/sake tasks:S3BackfillMetadata --dry-run

# Try it on one folder first
vendor/bin/sake tasks:S3BackfillMetadata --path=Uploads --limit=10

# Everything
vendor/bin/sake tasks:S3BackfillMetadata

The task can be stopped and re-run, as objects which already have a hash are skipped. Use --force to recalculate them. On a versioned bucket each updated object gains a new version.

Photos which EXIF says are on their side (typically portrait phone photos) are stored with the size in the file plus an orientation value, and the width and height are swapped when read if the image manager has autoOrientation on. This needs the PHP exif extension; without it JPEGs are not given dimensions and are measured from the image as before.

Files given metadata by an earlier version of this feature only gain their dimensions when the task is run with --force.

Uninstalling

Run composer remove silverstripe/s3 to remove the module.

silverstripe / s3

Package info

Security

Statistics