popphp/kettle-storage

1.0.0 2017-06-03 16:33 UTC

README

kettle-storage is a simple file storage REST API. It allows client applications to manages files and folders, as well as their permissions. It also uses the kettle-auth authentication API to authenticate users with the kettle-storage API.

The Kettle Namespace is a set of tools built using components from the Pop PHP Framework.

Requirements

The client application that is interfacing with the kettle-storage API would also interface with the kettle-auth API to authenticate a user and obtain a token. That token can then be passed to the kettle-storage API on each request and it will validate the token on each request with the kettle-auth API server.

Top

Installation

kettle-storage comes with the database schemas for MySQL, PostgreSQL and SQLite. Load the preferred schema into your database, and copy the file app/config/app.web.orig.php to app/config/app.web.php and load the corresponding database credentials into the new config file. Also, enter the URL of the authentication server that is running the kettle-auth authentication API.

For the examples below, the PHP CLI server is used at http://localhost:8000/. Please note, it is strongly recommended that you use https in production.

Top

Configuration

Some basic configuration settings are available within the app/config/app.web.php file. You can set the following configurations:

  • max_filesize - Sets the max filesize allowed. Various formats supported (10MB, 150K, 50000, etc.)
  • disallowed_types - An array or comma-separated string of file formats that are not allowed ('php,js,css')
  • overwrite - Set to false to automatically rename any conflicting file name. Set to true to overwrite it.
  • default_mime_type - Default mime type to fallback on.
  • mime_types - Mime types to register with file extensions.

Top

Files

Uploading Files

Uploading files to the kettle-storage API requires using a JSON payload with a base64 encoded data stream of the file.

curl -i -X PUT --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  --header "Content-Type: application/json" -d @files.json http://localhost:8000/files

Below is an example of the bare minimum of what is required, a name field and a data field:

{
    "name": "hello.txt",
    "data": "SGVsbG8gV29ybGQK"
}

You can also upload multiple files in one payload by using a files array:

{
    "files" : [
        {
            "name": "hello1.txt",
            "data": "SGVsbG8gV29ybGQK"
        },
        {
            "name": "hello2.txt",
            "data": "SGVsbG8gV29ybGQK"
        }
    ]
}

Top

Listing Files

To list files and their associated data directly from the application, you can do this:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/files
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 20:27:34 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 935
Content-Type: application/json

{
    "files": [
        {
            "id": 6,
            "uri": "\/hello.txt",
            "folder_id": null,
            "filename": "hello.txt",
            "filesize": 12,
            "description": null,
            "created_by": null,
            "created": "2017-05-26 15:27:09",
            "modified_by": null,
            "modified": null,
            "permissions": {
                "role_ids": [],
                "user_ids": []
            }
        }
    ]
}

To list a single file and its data, you can do this:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/files/6
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 20:28:58 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 323
Content-Type: application/json

{
    "id": 6,
    "uri": "\/hello.txt",
    "folder_id": null,
    "filename": "hello.txt",
    "filesize": 12,
    "description": null,
    "created_by": null,
    "created": "2017-05-26 15:27:09",
    "modified_by": null,
    "modified": null,
    "permissions": {
        "role_ids": [],
        "user_ids": []
    }
}

Top

Fetching a File

To fetch an actual file to serve, you can just use it's URI:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/hello.txt
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 20:29:41 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Content-Length: 12
Content-Type: text/plain;charset=UTF-8

Hello World

Top

Updating a File

Updating a file only requires a JSON payload if you will be replacing the file data contents. Otherwise, you can just pass a normal data string. For example, to just rename a file:

curl -i -X POST --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  -d"name=hello2.txt" http://storage.localhost/files/6
HTTP/1.1 200 Updated
Date: Fri, 26 May 2017 20:32:26 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 342
Content-Type: application/json

{
    "id": 6,
    "uri": "\/hello2.txt",
    "folder_id": null,
    "filename": "hello2.txt",
    "filesize": 12,
    "description": null,
    "created_by": null,
    "created": "2017-05-26 15:27:09",
    "modified_by": null,
    "modified": "2017-05-26 15:32:26",
    "permissions": {
        "role_ids": [],
        "user_ids": []
    }
}

Top

Deleting Files

You can delete a single file like this:

curl -i -X DELETE --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/files/6

Or, delete multiple files at once like this:

curl -i -X DELETE --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  -d"file_ids[]=6&file_ids[]=7" http://storage.localhost/files

Both requests will return:

HTTP/1.1 204 Deleted
Date: Fri, 26 May 2017 20:33:35 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Type: application/json

Top

Folders

Creating a Folder

curl -i -X PUT --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  -d"name=foo" http://storage.localhost/folders
HTTP/1.1 201 Created
Date: Fri, 26 May 2017 20:47:09 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 274
Content-Type: application/json

{
    "uri": "\/foo",
    "parent_id": null,
    "name": "foo",
    "description": null,
    "created_by": null,
    "created": "2017-05-26 15:47:09",
    "modified_by": null,
    "modified": null,
    "permissions": {
        "role_ids": [],
        "user_ids": []
    }
}

You can then upload directly to that folder by sending a request with the file name containing the folder:

{
    "name": "foo/hello.txt",
    "data": "SGVsbG8gV29ybGQK"
}

Creating a Folder on the Fly

Using the above example, you can create a folder (and its sub-folders) on the fly. If the folder structure foo/bar does not exist when you upload a file, it will be created for you:

{
    "name": "foo/bar/hello.txt",
    "data": "SGVsbG8gV29ybGQK"
}

Top

Listing Folders

To list folders and their associated data directly from the application, you can do this:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/folders
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 20:51:21 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 434
Content-Type: application/json

{
    "folders": [
        {
            "id": 7,
            "uri": "\/foo",
            "parent_id": null,
            "name": "foo",
            "description": null,
            "created_by": null,
            "created": "2017-05-26 15:47:09",
            "modified_by": null,
            "modified": null,
            "permissions": {
                "role_ids": [],
                "user_ids": []
            }
        }
    ]
}

To list a single folder and its data, you can do this:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/folders/7
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 20:51:49 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 323
Content-Type: application/json

{
    "id": 7,
    "uri": "\/foo",
    "parent_id": null,
    "name": "foo",
    "description": null,
    "created_by": null,
    "created": "2017-05-26 15:47:09",
    "modified_by": null,
    "modified": null,
    "permissions": {
        "role_ids": [],
        "user_ids": []
    },
    "files": [],
    "folders": []
}

Top

Fetching a Folder

To fetch an actual folder and its file listings, you can just use it's URI:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/foo
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 20:58:31 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 478
Content-Type: application/json

{
    "files": [
        {
            "id": 7,
            "uri": "\/foo\/hello.txt",
            "folder_id": 7,
            "filename": "hello.txt",
            "filesize": 12,
            "description": null,
            "created_by": null,
            "created": "2017-05-26 15:53:37",
            "modified_by": null,
            "modified": null,
            "permissions": {
                "role_ids": [],
                "user_ids": []
            }
        }
    ]
}

Top

Updating a Folder

Updating a folder will cascade all of the updates across any sub-folders or files within the folder or its sub-folders:

curl -i -X POST --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  -d"name=foo2" http://storage.localhost/folders/7
HTTP/1.1 200 Updated
Date: Fri, 26 May 2017 21:00:53 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Length: 293
Content-Type: application/json

{
    "uri": "\/foo2",
    "parent_id": null,
    "name": "foo2",
    "description": null,
    "created_by": null,
    "created": "2017-05-26 15:47:09",
    "modified_by": null,
    "modified": "2017-05-26 16:00:53",
    "permissions": {
        "role_ids": [],
        "user_ids": []
    }
}

For example, /foo/hello.txt will now result in a 404 Not Found:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/foo/hello.txt
HTTP/1.1 404 Not Found
Date: Fri, 26 May 2017 21:01:57 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Content-Length: 0
Content-Type: application/json

But, the new URI will work correctly:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/foo2/hello.txt
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 21:03:08 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Content-Length: 12
Content-Type: text/plain;charset=UTF-8

Hello World

Top

Deleting Folders

Deleting a folder will delete the all other files and folders contained within the folder. You can delete a single folder like this:

curl -i -X DELETE --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  http://storage.localhost/folders/7

Or, delete multiple folders at once like this:

curl -i -X DELETE --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  -d"folder_ids[]=7&folder_ids[]=8" http://storage.localhost/folders

Both requests will return:

HTTP/1.1 204 Deleted
Date: Fri, 26 May 2017 21:04:15 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: Authorization, Content-Type, X-Token-Expires, X-Permission
Access-Control-Allow-Methods: HEAD, OPTIONS, GET, PUT, POST, PATCH, DELETE
Content-Type: application/json

Top

Permissions

You can have an additional level of access control by utilizing permissions. When managing both files and folders, you can pass and store the arrays user_ids and role_ids that contain numeric IDs that are associated with certain users and roles. If those values are stored with a file or a folder, then that resource when only be available to those users or roles.

Looking at the earlier example of uploading a file:

curl -i -X PUT --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  --header "Content-Type: application/json" -d @files.json http://localhost:8000/files

But we add the arrays user_ids and role_ids to the JSON data:

{
    "name": "hello.txt",
    "data": "SGVsbG8gV29ybGQK",
    "user_ids" : [1, 2],
    "role_ids" : [1]
}

Now when the file is stored, only users with the IDs 1 and 2 or with the role ID of 1 can access the file. And you would access it using the X-Permission header with a base64 encoded string. The structure of the value would be:

$permission = base64_encode('role_id:1'); // cm9sZV9pZDox

or

$permission = base64_encode('user_id:2'); // dXNlcl9pZDoy

Using either one of the bash64 strings would work in this case:

curl -i -X GET --header "Authorization: Bearer 54c0d5884470eeed3e7cd3d8bf66c4021d929a1c" \
  --header "X-Permission: cm9sZV9pZDox
  http://storage.localhost/hello.txt
HTTP/1.1 200 OK
Date: Fri, 26 May 2017 20:29:41 GMT
Server: Apache/2.4.18 (Ubuntu)
X-Powered-By: PHP/7.0.8
Content-Length: 12
Content-Type: text/plain;charset=UTF-8

Hello World

Failing to use the X-Permission header or using the wrong value will result in a 403 Forbidden response.

Top