keboola / db-extractor-common
Common library from Keboola Database Extractors
Installs: 7 186
Dependents: 0
Suggesters: 0
Security: 0
Stars: 0
Watchers: 13
Forks: 0
Open Issues: 25
Requires
- php: >=7.4
- ext-json: *
- ext-pdo: *
- keboola/common-exceptions: ^1.1
- keboola/csv: ^2.2
- keboola/db-extractor-adapter: ^1.0.3
- keboola/db-extractor-config: ^1.4
- keboola/db-extractor-ssh-tunnel: ^1.1
- keboola/db-extractor-table-format: ^3.0
- keboola/php-component: ^8.1
- keboola/php-datatypes: ^4.7
- keboola/php-utils: ^2.3||^3.0||^4.0
- keboola/retry: ^0.5
- monolog/monolog: ^2.0
- nette/utils: ^3.0
- phpunit/phpunit: ^9.1
- pimple/pimple: ^3.3
- psr/log: ^1.1
Requires (Dev)
- ihsw/toxiproxy-php-client: ^2.0
- keboola/coding-standard: >=9.0.0
- keboola/php-temp: ^2.0
- php-parallel-lint/php-parallel-lint: ^1.2
- phpstan/phpstan: ^0.12.14
- symfony/debug: ^4.3
- dev-master
- 14.0.4
- 14.0.3
- 14.0.2
- 14.0.1
- 14.0.0
- 13.3.0
- 13.2.4
- 13.2.3
- 13.2.2
- 13.2.1
- 13.2.0
- 13.1.0
- 13.0.0
- 12.2.0
- 12.1.2
- 12.1.1
- 12.1.0
- 12.0.1
- 12.0.0
- 11.0.0
- 10.0.1
- 10.0.0
- 9.4.1
- 9.4.0
- 9.3.2
- 9.3.1
- 9.3.0
- 9.2.1
- 9.2.0
- 9.1.11
- 9.1.10
- 9.1.9
- 9.1.8
- 9.1.7
- 9.1.6
- 9.1.5
- 9.1.4
- 9.1.3
- 9.1.2
- 9.1.1
- 9.1.0
- 9.0.0
- 8.1.0
- 8.0.0
- 7.0.0
- 6.2.0
- 6.1.0
- 6.0.0
- 5.3.2
- 5.3.1
- 5.3.0
- 5.2.3
- 5.2.2
- 5.2.1
- 5.2.0
- 5.1.0
- 5.0.2
- 5.0.1
- 5.0.0
- 4.1.1
- 4.1.0
- 4.0.6
- 4.0.5
- 4.0.4
- 4.0.3
- 4.0.2
- 4.0.1
- 4.0.0
- 3.2.2
- 3.2.1
- 3.2.0
- 3.1.3
- 3.1.1
- 3.1.0
- 3.0.4
- 3.0.3
- 3.0.2
- 3.0.1
- 3.0.0
- 2.0.7
- 2.0.6
- 2.0.5
- 2.0.4
- 2.0.3
- 2.0.2
- 2.0.1
- 2.0.0
- 1.4.0
- 1.3.0
- 1.2.5
- 1.2.4
- 1.2.3
- 1.2.2
- 1.2.1
- 1.2.0
- 1.1.1
- 1.1.0
- 1.0.0
- 0.0.4
- 0.0.3
- 0.0.2
- 0.0.1
- dev-ondra-setup-logger
- dev-webrouse-254-db-adapter
- dev-piv-ignore-extra-action-keys
- dev-tf-testing-rf
- dev-david-value-objects
- dev-david-v10
- dev-piv-callable-retry
- dev-ujovlado-notifications
- dev-piv-retry-connection
- dev-piv-tunnel-c
- dev-backup-david-value-objects-2019-01-23
- dev-backup-v10-before-rebase-2019-01-23
- dev-piv-retry-sqlcodes
- dev-piv-update-php-datatypes
- dev-piv-empty-table-fix
- dev-php55
This package is auto-updated.
Last update: 2021-01-07 12:32:12 UTC
README
Common classes for creating vendor specific database extractors.
Extractors using DBC
Development and running tests
docker-compose build
docker-compose run --rm tests # runs the tests
Usage
Add the library to your component's composer:
php composer.phar require db-extractor-common
composer.json
{
"require": "db-extractor-common": ^8.0
}
Usage
Create entrypoint script file run.php
like this one for Mysql extractor:
https://github.com/keboola/db-extractor-mysql/blob/master/src/run.php
Note that as of version 7, configuration rows are supported so it is not necessary to support .yml configs or table array configurations.
The $config is loaded from the config.json
file. You have to provide values for the data_dir
and extractor_class
keys.
extractor_class
is the main class of derived extractor, it should extend Keboola\DbExtractor\Extractor\Extractor
.
You will need to implement the following methods:
createConnection(array $params)
testConnection()
simpleQuery(array $table, array $columns = array()): string
getTables(?array $tables = null): array;
Note that to support identifier sanitation, the getTables method should return a sanitizedName
property for each
column. This sanitizedName
should be created using Keboola\php-utils\sanitizeColumnName
If you want to implement incremental fetching, you must implement
validateIncrementalFetching(array $table, string $columnName, ?int $limit = null): void
Please see the sample in the Common
class: https://github.com/keboola/db-extractor-common/blob/master/src/Keboola/DbExtractor/Extractor/Common.php#L52
The namespace of your extractor class shoud be Keboola\DbExtractor\Extractor
and the name of the class should corespond to DB vendor name i.e. PgSQL, Oracle, Impala, Firebrid, DB2 and so on.
Please check the existing implementations above for help getting started.