keboola/storage-api-client

Keboola Storage API PHP CLient

Installs: 50 090

Dependents: 5

Suggesters: 0

Stars: 5

Watchers: 7

Forks: 3

Open Issues: 7

4.10.0 2016-05-18 14:59 UTC

README

Latest Stable Version License Total Downloads Build Status

Simple PHP wrapper library for Keboola Storage REST API

Installation

Library is available as composer package. To start using composer in your project follow these steps:

Install composer

curl -s http://getcomposer.org/installer | php
mv ./composer.phar ~/bin/composer # or /usr/local/bin/composer

Create composer.json file in your project root folder:

{
    "require": {
        "php" : ">=5.4.0",
        "keboola/storage-api-client": "2.12.*"
    }
}

Install package:

composer install

Add autoloader in your bootstrap script:

require 'vendor/autoload.php';

Read more in Composer documentation

Usage examples

Table write:

require 'vendor/autoload.php';

use Keboola\StorageApi\Client,
    Keboola\Csv\CsvFile;

$client = new Client([
  'token' => 'YOUR_TOKEN',
]);
$csvFile = new CsvFile(__DIR__ . '/my.csv', ',', '"');
$client->writeTableAsync('in.c-main.my-table', $csvFile);

Table export to file:

require 'vendor/autoload.php';

use Keboola\StorageApi\Client,
  Keboola\StorageApi\TableExporter;

$client = new Client(['token' => 'YOUR_TOKEN',]);

$exporter = new TableExporter($client);
$exporter->exportTable('in.c-main.my-table', './in.c-main.my-table.csv', []);

Tests

Warning: Never run this tests on production project with real data, always create project for testing purposes!!!

The main purpose of these test is "black box" test driven development of Keboola Connection. These test guards the API implementation.

Tests are divided into multiple test suites:

  • common - tests all storage backend unrelated features like configurations, events, file uploads, tokens management
  • backend-mysql - tests project with mysql storage backend
  • backend-redshift - tests project with redshift storage backend

Common test suite

This test suite expects following environment variables set:

You can export variables manually or you can create and fill file set-env.sh as copy of attached set-env.template.sh.

Than you can run tests:

source ./set-env.sh && php ./vendor/bin/phpunit --testsuite common

Mysql backend test suite

This test suite expects following environment variables set:

  • STORAGE_API_URL - URL of Keboola Storage API (https://connection.keboola.com/)
  • STORAGE_API_TOKEN - Storage API token associated to user (Admin master token) with all permissions. Project must have mysql set as default backend.

You can export variables manually or you can create and fill file set-env.mysql.sh as copy of attached set-env.mysql.template.sh.

Than you can run tests:

source ./set-env.mysql.sh && php ./vendor/bin/phpunit --testsuite backend-mysql

Redshift backend test suite

Reshift tests require a cluster connected to Storage API and credentials. When you have a project with enabled Redshift, create 2 Redshift buckets (using Storage API Console):

  • in.c-api-tests
  • out.c-api-tests

These credentials are used in tests to simulate "copy from table" transformations behaviour. Then you can create your Redshift user. Connect to your Redshift database sapi_YOURPROJECTID and run queries:

CREATE USER test_user PASSWORD '***';
GRANT ALL PRIVILEGES ON DATABASE sapi_YOURPROJECTID TO test_user;
GRANT ALL PRIVILEGES ON SCHEMA "in.c-api-tests-redshift" TO test_user;
GRANT ALL PRIVILEGES ON SCHEMA "out.c-api-tests-redshift" TO test_user;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA "in.c-api-tests-redshift" TO test_user;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA "out.c-api-tests-redshift" TO test_user;

This test suite expects following environment variables set:

  • STORAGE_API_URL - URL of Keboola Storage API (https://connection.keboola.com/)
  • STORAGE_API_TOKEN - Storage API token associated to user (Admin master token) with all permissions. Project must have Redshift set as default backend.
  • REDSHIFT_HOSTNAME - hostname of storage backend Redshift cluster
  • REDSHIFT_USER - previously created user
  • REDSHIFT_PASSWORD - previously created passowrd

You can export variables manually or you can create and fill file set-env.mysql.sh as copy of attached set-env.mysql.template.sh.

Than you can run tests:

source ./set-env.redshift.sh && php ./vendor/bin/phpunit --testsuite backend-redshift

Versioning

semver.org is followed.

Release History

See the CHANGELOG.