jaybizzle / crawler-detect

v1.4.2 2026-10-20 18:03 UTC

Requires

• php: >=7.2.0

Requires (Dev)

• phpunit/phpunit: ^8.5.52|^9.4

MIT

• Mark Beech

• crawler
• crawler detect
• crawler detector
• php crawler detect
• crawlerdetect

Published on Packagist.org on 2026-10-20 18:04 UTC

README

crawlerdetect.io

About

CrawlerDetect is a PHP library for detecting bots, crawlers and spiders via the User-Agent and HTTP_FROM headers. It currently recognises thousands of user agents and is updated regularly.

Installation

composer require jaybizzle/crawler-detect

Usage

use Jaybizzle\CrawlerDetect\CrawlerDetect;

$CrawlerDetect = new CrawlerDetect;

// Check the user agent of the current visitor
if ($CrawlerDetect->isCrawler()) {
    // true if a crawler user agent was detected
}

// Pass a user agent as a string
if ($CrawlerDetect->isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')) {
    // true if a crawler user agent was detected
}

// Output the part of the user agent that matched (if any)
echo $CrawlerDetect->getMatches();

Passing headers from a request object

With no arguments, CrawlerDetect reads $_SERVER. If your headers come from somewhere else — a PSR-7 request, Symfony's HeaderBag, Swoole, or a Lambda event — pass them in directly. Both real header names (User-Agent) and PHP's SAPI names (HTTP_USER_AGENT) are understood, and values may be strings or arrays of strings.

// PSR-7 (Slim, Mezzio, Laminas, League)
$CrawlerDetect = new CrawlerDetect($request->getHeaders());

// Symfony HttpFoundation
$CrawlerDetect = new CrawlerDetect($request->headers->all());

// Swoole
$CrawlerDetect = new CrawlerDetect($request->header);

if ($CrawlerDetect->isCrawler()) {
    // ...
}

Prefer this over isCrawler($request->getHeaderLine('User-Agent')). Some crawlers — Googlebot in particular — send a genuine browser User-Agent and identify themselves in another header such as From or Sec-CH-UA. Passing the full set lets CrawlerDetect check all of them; passing a single string can only ever check one.

Contributing

If you find a bot, spider or crawler that CrawlerDetect fails to detect, please open a pull request that:

• adds the regex pattern to the $data array in src/Fixtures/Crawlers.php
• adds the failing user agent string to tests/data/user_agent/crawlers.txt

The raw/Crawlers.json and raw/Crawlers.txt files are regenerated automatically by export.php after merge — no need to touch them.

If you're not able to submit a PR, open an issue with the user agent string and we'll take it from there.

Ports & Integrations

CrawlerDetect has been ported to a number of other languages and frameworks. If you maintain a port not listed here, please open a PR.

Platform	Project
Laravel	Laravel-Crawler-Detect
Node.js / ES6	es6-crawler-detect
Python	crawlerdetect
.NET / .NET Core	NetCrawlerDetect
Ruby	crawler_detect
Go	crawlerdetect

These ports are no longer maintained and do not receive new crawler signatures:

Platform	Project	Last updated
Symfony 2 / 3 / 4	CrawlerDetectBundle	2021
Yii2	yii2-crawler-detect	2016
JVM (Java, Scala, Kotlin)	CrawlerDetect	2022

Security

Please don't report security issues in the public issue tracker. See SECURITY.md for how to report one privately.

Credits

Parts of this library are based on the excellent MobileDetect.

License

Released under the MIT License.