limetecbiotechnologies / behat-logger-extension
log the behat execution into json result
Requires
- php: ^8.0
- ext-json: *
- behat/behat: ^3.3
- behat/mink-extension: ^2.2
- behat/mink-selenium2-driver: ^1.3
- limetecbiotechnologies/testrail-php-api: ^1.1
- stecman/symfony-console-completion: ^0.8.0
- symfony/finder: ^3.3|^4.2
- symfony/framework-bundle: ^3.3|^4.2
- symfony/http-kernel: ^3.3|^4.2
Requires (Dev)
- phpunit/phpunit: ^5.7|^6.0|^9.5
- dev-master
- v3.1.0
- v3.0.5
- v3.0.4
- v3.0.3
- v3.0.2
- v3.0.1
- v3.0.0
- v2.1.1
- v2.1.0
- v2.0.0
- v1.6.0
- v1.5.2
- v1.5.1
- v1.5.0
- v1.4.3
- v1.4.2
- v1.4.1
- v1.4.0
- v1.3.0
- v1.2.1
- v1.2.0
- v1.1.0
- v1.0.3
- v1.0.2
- v1.0.1
- v1.0.0
- v0.3.1
- v0.3.0
- v0.2.1
- v0.2.0
- v0.1.1
- v0.1.0
- v0.0.21
- v0.0.20
- v0.0.19
- v0.0.18
- v0.0.17
- v0.0.16
- v0.0.15
- v0.0.14
- v0.0.13
- v0.0.12
- v0.0.11
- v0.0.10
- v0.0.9
- v0.0.8
- v0.0.7
- v0.0.6
- v0.0.5
- v0.0.4
- v0.0.3
- v0.0.2
- v0.0.1
- v0.0.0
- dev-path-fix
- dev-file_exists
- dev-check-exists
- dev-master_2
- dev-master_1.6
This package is auto-updated.
Last update: 2024-10-10 13:47:56 UTC
README
this package provides an extension for behat to log the test results in an json-file. this package also provide commands to validate and merge this json files.
Installation
INFORMATION: if you want to use the behat logger extension for your project, please read the "PHP Integration" section! The installation section installs only the cli-commands as standalone application
For users is now a docker-image available. For developers see PHP Integration
Migration from 1.x to 2.x
since the major release 2, the identification of tests for synchronization has changed. In version 1 the title of the scenario was used for synchronization. in this release, an id will used, given by an behat-tag. every test requires an unique identifier tag. for example "@testrail-case-1,@testrail-case-2, e.t.c." update your dependency to the last 1.x version and add the following properties to your .testrail.yml
api: ... identifier_tag_field: yourNewIdentifierField
execute the testrail:push:cases command to add all the ids to your cases in testrail. Now you can update to 2.x (but some .testrail.yml properties has changed!!!)
docker image
docker run -v /path/to/project/:/behat/ seretos/behat-logger-cli behat-logger-cli list
add the docker-container to your docker-compose.yml
version: '3.7' services: features-push: image: seretos/behat-logger-cli command: push volumes: - ./:/behat/ environment: TESTRAIL_SERVER: http://testrail:80 TESTRAIL_USER: yourTestrailLogin TESTRAIL_PASSWORD: yourTestrailPassword TESTRAIL_PROJECT: yourTestrailProject TESTRAIL_SUITE: yourTestrailSuite features-validate: image: seretos/behat-logger-cli command: validate volumes: - ./:/behat/
PHP Integration
add the package to your project as below
composer require seretos/behat-logger-extension --dev vendor/bin/behat-logger-cli --help
activate the logger in your behat.yml:
default: formatters: logger: ~ extensions: seretos\BehatLoggerExtension\BehatLoggerExtension: output_path: '%paths.base%/build/behat'
OPTIONAL: if you use an symfony application, you can add this extension (seretos\BehatLoggerExtension\BehatLoggerExtensionBundle) to your Symfony Kernel and integrate the commands in your cli
Command line usage
combine different result jsons into one file and one suite:
behat-logger-cli combine:logs [suite-name] --output=/output/path/ --regex=results/firefox*
if different jsons contain a testresult for one test with the same environment, this command throws an exception
convert all found feature-files to one json-file without results:
behat-logger-cli feature:to:log [suite-name] --output=/output/path/ --regex=features/
check that all scenarios in the log file has an unique title:
This command is deprecated please use validate:scenario:id in future!
behat-logger-cli validate:scenario:title [log-file.json]
check that all schenarios in the log file has an unique id tag:
behat-logger-cli validate:scenario:id [log-file.json] --identifier_tag_regex="/^testrail-case-([0-9]*)$/"
check that all tests are executed in the given environments:
# check that all browserless tests are executed behat-logger-cli validate:execution actual.json expected.json --tags=~javascript --environments=unknown # check that all browser tests are executed in firefox and chrome behat-logger-cli validate:execution actual.json expected.json --tags=javascript --environments=firefox --environments=chrome
send a json-result to testrail and create sections and cases
behat-logger-cli testrail:push:cases testRailSuiteName actual.json
send a json-result to testrail and create environment configurations:
behat-logger-cli testrail:push:configs actual.json
send a json-result to testrail and create results
behat-logger-cli testrail:push:results testRailSuiteName actual.json testResultName --milestone=v2.8.0
the commands testrail:push:cases and testrail:push:results requires an .testrail.yml in the current work directory with the following informations:
api: server: https://yourTestrail.testrail.io/ user: yourUser@mail.de password: yourPassword project: youtProject template: Test Case (Steps) type: Automated title_field: custom_preconds group_field: custom_automation_type identifier_field: custom_identifier identifier_regex: /^testrail-case-([0-9]*)$/ # set field an priorities on specific tags fields: /^.*$/: custom_automation_type: Behat priorities: /^priority_low$/: Low
log format
first of all, the json file contains the behat suite. if the log-writer can not detect the suite name, they use a suite named "default"
{ "suites": [ { "name": "default", "features": { ... } } ] }
the suite contains a list of features:
"features": { "features\/featurefile.feature": { "title": "feature title", "filename": "features\/featurefile.feature", "description": null, "language": "en", "scenarios": { ... } }, ... }
and a feature contains scenarios with steps and results
"scenarios": { "scenariotitle": { "title": "scenariotitle", "tags": ["behattag1","behattag2"], "steps": [ { "line": 0, "text": "the user 'test' exists", "keyword": "Given", "arguments": [] }, { "line": 1, "text": "i logged in as 'test'", "keyword": "And", "arguments": [] } ], "results": [ ... ] }, ... }
and last but not least, contains the features results. the environment property is the browser name. on guette the environment name is setted to "unknown"
"results": [ "firefox": { "environment": "firefox", "duration": "1.00", "message": null, "stepResults": [ { "line": 0, "passed": true, "screenshot": null, "message": null }, { "line": 1, "passed": true, "screenshot": null, "message": null } ] } ]