akdr/behat-github-annotations

Extension for enabling Github action annotations for Behat

dev-main 2025-08-19 21:12 UTC

Requires (Dev)

• php: >=8.2
• behat/behat: ^v3.10.0
• carthage-software/mago: ^0.26.1
• mockery/mockery: ^1.6
• phpunit/phpunit: ^11.0
• symfony/process: ^v7.3.0

MIT

• Daniel Westerling

README

Extension for enabling GitHub Action annotations for Behat test failures.

Automatically converts failed Behat steps into GitHub annotation format, providing inline code review feedback directly in your GitHub Actions workflow.

Features

• 🎯 Automatic Error Annotation: Failed Behat steps automatically appear as GitHub annotations
• 📍 Precise Location: Annotations point to exact file and line where step failed
• 🔍 Rich Context: Includes feature, scenario, and step information
• 📊 Verbosity Control: Configurable detail levels including stack traces
• ⚙️ Easy Integration: Simple configuration with existing Behat setups
• 🚀 GitHub Actions Ready: Works seamlessly in CI/CD pipelines

Example Output

When a Behat step fails, this extension generates GitHub annotations like:

::error file=features/user_login.feature,line=12,title=Failed: I enter invalid credentials (in Invalid Login: User Authentication)::Invalid username or password provided

This appears in your GitHub Actions workflow as inline annotations on the failing lines.

Installation

Install via Composer:

composer require --dev akdr/behat-github-annotations

Requirements

• PHP 8.2 or higher
• Behat 3.10 or higher
• GitHub Actions environment (for annotation display)
• Mago 0.26 (for linting)

Quick Start

1. Basic Configuration

Add to your behat.yml:

default:
  formatters:
    github: ~
  extensions:
    Akdr\BehatGithubAnnotations\GitHubAnnotationsExtension: ~

2. GitHub Actions Integration

Update your .github/workflows/test.yml:

name: Tests
on: [push, pull_request]

jobs:
  behat-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup PHP
        uses: shivammathur/setup-php@v2
        with:
          php-version: '8.4'
          extensions: mbstring
          
      - name: Install Dependencies
        run: composer install --no-dev --prefer-dist
        
      - name: Run Behat Tests
        run: ./vendor/bin/behat --format=github

3. Run Tests

# Local testing
./vendor/bin/behat --format=github

# In GitHub Actions (automatic annotation display)
./vendor/bin/behat --format=github

Configuration

Basic Configuration

# behat.yml
default:
  formatters:
    github: ~
  extensions:
    Akdr\BehatGithubAnnotations\GitHubAnnotationsExtension: ~

Advanced Configuration

# behat.yml
default:
  formatters:
    github:
      parameters:
        annotation_type: error        # error, warning, notice
        include_trace: false          # Include stack traces
        file_relative_path: true      # Use relative file paths
  extensions:
    Akdr\BehatGithubAnnotations\GitHubAnnotationsExtension: ~

Output to File

# behat.yml  
default:
  formatters:
    github:
      output: annotations.txt        # Save annotations to file

Verbosity Levels

# Basic output
./vendor/bin/behat --format=github

# Verbose output (enhanced error details)
./vendor/bin/behat --format=github -v

# Very verbose output (includes stack traces)
./vendor/bin/behat --format=github -vv

Examples and Integration Guides

📁 Complete Examples

This repository includes comprehensive examples in the examples/ directory:

• Example Configuration - Complete Behat configuration with multiple profiles
• Sample Features - Real-world test scenarios demonstrating the formatter
• GitHub Actions Workflows - Production-ready CI/CD examples
• Step-by-Step Guide - Detailed integration instructions

🚀 Quick Integration

Copy the example configuration to get started quickly:

# Copy example configuration
cp examples/behat.yml your-project/behat.yml

# Copy GitHub Actions workflow  
cp examples/.github/workflows/basic-behat.yml .github/workflows/tests.yml

Integration Patterns

With Multiple Formatters

# behat.yml - Use alongside other formatters
default:
  formatters:
    pretty: ~                       # Standard console output
    junit:                          # JUnit XML for CI
      filename: test-results.xml
    github: ~                       # GitHub annotations

With Different Environments

# behat.yml
default:
  # Default configuration for local development
  
ci:
  # CI-specific configuration
  formatters:
    github: ~
  extensions:
    Akdr\BehatGithubAnnotations\GitHubAnnotationsExtension: ~

GitHub Actions Matrix Testing

See examples/.github/workflows/advanced-testing.yml for a complete workflow that includes:

• Multi-PHP version testing
• Test artifact collection
• Coverage reporting
• Deployment simulation

Troubleshooting

Common Issues

Annotations Not Appearing

Problem: GitHub annotations not visible in Actions

Solutions:

• Ensure you're using --format=github in your command
• Verify the extension is properly loaded in behat.yml
• Check that tests are actually failing (annotations only show for failures)

Deprecation Warnings

Problem: PHP deprecation warnings in output

Explanation: These warnings come from Behat core when running on PHP 8.4, not from this extension

Solutions:

• Warnings don't affect functionality
• Consider suppressing warnings in CI: php -d error_reporting="E_ALL & ~E_DEPRECATED"

File Path Issues

Problem: Incorrect file paths in annotations

Solutions:

• Use file_relative_path: true in configuration
• Ensure working directory is set correctly in CI
• Check file permissions and accessibility

Debug Mode

Enable debug output to troubleshoot issues:

# Debug mode with maximum verbosity
./vendor/bin/behat --format=github -vvv --debug

Validation

Verify extension is loaded correctly:

# List available formatters (should include 'github')
./vendor/bin/behat --help

# Test configuration
./vendor/bin/behat --config-reference

API Reference

Configuration Parameters

Parameter	Type	Default	Description
annotation_type	string	error	GitHub annotation type (error, warning, notice)
include_trace	boolean	false	Always include stack traces regardless of verbosity
file_relative_path	boolean	true	Use relative file paths in annotations

GitHub Annotation Format

This extension generates annotations in the format:

::type file=path,line=number,title=title::message

Components:

• type: Annotation type (error, warning, notice)
• file: File path where the failure occurred
• line: Line number of the failed step
• title: Descriptive title including step, scenario, and feature context
• message: Error message with optional stack trace

Escaping Rules

Special characters are URL-encoded according to GitHub's requirements:

• % → %25
• \r → %0D
• \n → %0A
• , → %2C

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for details.

Development Setup

# Clone the repository
git clone https://github.com/your-org/behat-github-annotations.git
cd behat-github-annotations

# Install dependencies
composer install

# Run tests
./vendor/bin/behat

# Run with the extension itself
./vendor/bin/behat --format=github

Testing

The extension includes comprehensive tests:

# Run all tests
./vendor/bin/behat

# Run specific test features
./vendor/bin/behat tests/Behat/Features/github_annotations_formatter.feature

# Test different scenarios
./vendor/bin/behat tests/Behat/Features/background_failures.feature
./vendor/bin/behat tests/Behat/Features/multiple_failures.feature

License

This project is licensed under the MIT License - see the LICENSE file for details.

Related Projects

• Behat - Behavior-driven development framework
• GitHub Actions - CI/CD platform

⭐ If this project helped you, please consider giving it a star on GitHub!