FAO toolkit, to manage FAO services such as SMTA api, DOI api and WIEWS code and organization data fetching
Intended use for Gene banks
FAO-toolkit is a toolkit to utilize FAO services from command line. Three module commands are att the moment implemented. The first from the fork of Easy smta toolkit, an HTTP POST utility, to register your institute's SMTAs ( Standard Material Transfer Agreements). The second command is for look up information from the WIEWS project. At the moment you can get organisation information by WIEWS Institute code. The third and latest the DOI module, which have the implemented commands; register and update.
To make SMTA posting/registrating work, you need to have the following:
- An account at FAO's Planttreaty site.
- Two database tables or dataviews with SMTA and SMTA material data.
- Make the configuration in this application's config file.
- run ./yii (or bin/fao-toolkit.php) smta/register command. The command uses parameters to limit which SMTAs you are intended to post/register.
To register and/or updating DOIs, you need to have the following:
- An account at FAO's Planttreaty site (It is really the same account as above. If you have an Easysmta account. You will also have an account to registry DOI's. Anyway contact the Office of International Treaty on Plant Genetic Resources for Food and Agriculture, if you represent a national Gene bank or similar.)
- Two or more database tables or dataviews with GLIS data (https://www.fao.org/3/bt113e/bt113e.pdf). Read fao-params-dist.php for more information
- Make the configuration in this application's config file.
- run ./yii (or bin/fao-toolkit.php) doi/register command. The command uses parameters to limit which SMTAs you are intended to post/register.
In order to install this library via composer run the following command in the console:
composer require endlessdreams/fao-toolkit ^2.0
or add the package manually to your composer.json file in the require section:
composer global require endlessdreams/fao-toolkit ^2.0
This will install fao-toolkit into the ~/.composer/vendor/ directory and, most importantly, the fao-toolkit CLI tool are installed into ~/.composer/vendor/bin/.
Simply add this directory to your PATH in your ~/.bash_profile (or ~/.bashrc) like this:
and fao-toolkit is now available on your command line.
To keep your tools up to date, you simply do this:
composer global update composer global install
To remove a package, you edit ~/.composer/composer.json and then run composer global update.
Major configuration file called fao-params-dist.php is located in sub folder to fao-toolkit application config folder. The sub folder can be any of:
- prod for production. This is the one you normally use.
- dev for development to a fake server.
- test which is used for the codeception tests
Do not change fao-params-dist.php file. Make a copy instead with the name fao-params-local.php
In fao-params-dist.php file you will see the following structure like:
return [ 'default' => getenv('FAO_TOOLKIT_DEFAULT_INSTITUTE') ?: 'default_institute', 'options' => [ 'force_migration' => false, ], 'database' => [ getenv('FAO_TOOLKIT_DEFAULT_INSTITUTE') ?: 'default_institute' => [ 'default' => 'smta', 'smta' => [ 'driver' => getenv('EASYSMTA_DB_DRIVER'), 'host' => getenv('EASYSMTA_DB_HOSTNAME'), 'database_name' => getenv('EASYSMTA_DB_DATABASE'), 'port' => getenv('EASYSMTA_DB_PORT') ?: null, 'options' => getenv('EASYSMTA_DB_OPTIONS_JSON') ?: null, 'username' => getenv('EASYSMTA_DB_USERNAME'), 'password' => getenv('EASYSMTA_DB_PASSWORD'), ], ], ], ...
To set database and FAO HTTPS API credentials in the config file or exporting environment variables in /etc/environment.
The examples are written as if your working directory is bin or the application is globally installed.
To test generate xml to terminal.
fao-toolkit smta/register --dry-run
To test generate all smta xml to 2018-01-01. (No https credentials needed here, only to your database)
fao-toolkit smta/register --dry-run --to=2018-01-01
To test post all SMTA's until 2018-01-01.
fao-toolkit smta/register --to=2018-01-01 -t
To test post all SMTA's from 2017-01-01 until 2018-01-01.
fao-toolkit smta/register --from=2017-01-01 --to=2018-01-01 -t
To actual post all SMTA's until 2018-01-01. (No testing)
fao-toolkit smta/register --to=2018-01-01
If several gene banks are hosted on the same server, you have to explicit refer to which gene bank / institute you will register SMTA's. Then you use the run-as parameter with the WIEWS-code.
fao-toolkit smta/register --to=2018-01-01 --run-as=SWE054
If the run-as institute uses accession numbers with different prefixes and only one set should be registered. (Note: this is for a MS SQL server, otherwise Regular expressions will be used.)
fao-toolkit smta/register --to=2018-01-01 --run-as=SWE054 --filter="NGB %"
To get information of WIEWS institute code registered organisation
fao-toolkit wiews/fetch NOR051
To get information of WIEWS institute code registered organisation and selecting output format like JSON.
fao-toolkit wiews/fetch --format=json -- NOR051
In the same way, you will be able to register DOI's Here is an example how to test register all accessions of institute SWE054 with accession prefix NGB. Create a special error output file and general output to another file. Finally you measure the time consumption of all of this. For Windows users check out 'Measure-Command'.
time fao-toolkit doi/register -t --run-as=SWE054 --filter='NGB %' --output-error-file=~/testerroroutput.txt > ~/testoutput.txt
To run Codeception tests
composer run serve 127.0.0.1:8080 > ./runtime/yii.log 2>&1 & bin/codecept run
ir. R (Roel) Hoekstra and dr.ir. TJL (Theo) van Hintum at Centrum voor Genetische Bronnen Nederland for letting me look at their dos cmd toolkit.
Mr Marco Marsella at FAO, Climate, Biodiversity, Land and Water Department, IT for great support.
This command line program is licensed under AGPL.
LICENSE for more information.