README

A PHP library to create and manage robots.txt files, built on top of the oihana-php-commands library.

✨ Features

Create or remove a project's robots.txt from the CLI
Optional custom path via -f|--file (absolute or relative)
Clear console support via -c|--clear
Config-driven defaults (config.toml)

A robots.txt file is a simple text file placed on a website’s root directory to give instructions to web crawlers (like search engine bots) about which pages or sections of the site should or shouldn’t be crawled and indexed. It helps control search engine access and manage site visibility.

📦 Installation

Requires PHP 8.4+

Install via Composer:

composer require oihana/php-robots

🚀 Usage

The package provides a Symfony Console command named command:robots with actions:

create: Generate a robots.txt file
remove: Delete a robots.txt file

Examples:

Create a robots.txt using defaults

bin/console command:robots create

Create a robots.txt at a custom path

bin/console command:robots create --path /var/www/my-website/htdocs/robots.txt

bin/console command:robots create --p /var/www/my-website/htdocs/robots.txt

Remove the default robots.txt

bin/console command:robots remove

Remove a robots.txt at a custom path

bin/console command:robots remove --path /var/www/my-website/htdocs/robots.txt

Clear the console before running

bin/console command:robots create --clear

Options

Option	ShortCut	Description
--clear	-c	Clear the console before running
--path	-p	The directory path of the 'robots.txt' file

Notes:

If --path is a relative path, it is resolved against the current working directory.
On creation/removal, parent directory existence and permissions are validated.

⚙️ Configuration

You can set defaults in config/config.toml under the [robots] section:

[robots]
path        = "/path/to/your/project/htdocs"
overwrite   = true
permissions = 0o644
owner       = "www-data"
group       = "www-data"
content     = '''
User-agent: *
Disallow: /
'''

🧪 Programmatic usage

You can instantiate and configure the command in PHP if needed:

use DI\Container;
use oihana\robots\commands\RobotsCommand;

$container = new Container();
$command = new RobotsCommand
(
    null,        // let kernel resolve the name
    $container,
    [
        'robots' => [
            'path'    => '/var/www/my-website/htdocs',
            'content' => "User-agent: *\nDisallow: /private/"
        ]
    ]
);

🔚 Exit codes

0 Success
1 Failure (invalid action, IO failure, etc.)

❓ Troubleshooting

Ensure the parent directory for the robots.txt is writable (especially when using a custom --path option).
When using relative paths with --path, they are resolved from the current working directory (pwd).
For more details, see the inline documentation in RobotsCommand.php.

✅ Running Unit Tests

To run all tests:

composer test

🧾 License

This project is licensed under the Mozilla Public License 2.0 (MPL-2.0).

👤 About the author

Author: Marc ALCARAZ (aka eKameleon)
Mail: marc@ooop.fr
Website: http://www.ooop.fr

oihana / php-robots

Maintainers

Details