README

Solr-based traffic analytics and bot forensics for Apache, nginx and Caddy.

Loghound reads the access logs you already have, correlates them with a small JavaScript beacon, and tells you two things that nothing else tells you honestly.

See it in action: 19 screenshots of the live panel, each with a short explanation, on the Loghound page at opensolr.com. Full documentation: opensolr.com/loghound-docs.

New since 1.2: SEO Tools comparing any two periods like for like · attack patterns you define per host, with 48 shipped defaults · non-browser User-Agents and refused attacks scored as bots · every rule list exported to and imported from CSV · a chart above every ranked table, compact on a phone. Details in the changelog.

Why this exists

There are exactly two reasons. Everything else in here — the charts, the geo map, the facets — is table stakes that a dozen tools already do well.

1. It is a serious bot detector, not a User-Agent blocklist

Every analytics tool "filters bots". What they actually do is match a list of strings like Googlebot and AhrefsBot against the User-Agent header, which catches exactly the crawlers that were polite enough to identify themselves and nothing else.

Modern scraping does not look like that. It looks like headless Chrome on rotating residential proxies: a real browser engine, executing your JavaScript, fetching your CSS and images, presenting a perfectly ordinary Windows Chrome User-Agent, arriving from a different consumer ISP every request. Every existing analytics product counts that as a human visit, because from the inside of any single plane of observation, it is indistinguishable from one.

Loghound observes three planes and cross-checks them. A scraper has to defeat all three simultaneously, and defeating the JavaScript plane without also defeating the behavioural plane is exactly what gives it away.

2. It measures real time-on-site

When Clicky, GA4 or Matomo tell you "average time on page: 4m 12s", they are usually reporting how long a tab existed. Not how long it was visible. Not how long anyone was looking at it. A tab left open in a background window for an hour is an hour of "engagement" in almost every analytics product on the market.

Loghound reports four separate numbers and never conflates them:

Number	What it actually means
`log_span_ms`	Last request minus first request. This is what log-only tools call "time on site". It is a lower bound and it misses the last page of every session entirely.
`wall_ms`	The page was open this long. This is the number everyone else reports.
`visible_ms`	The tab was visible and the window was focused.
`engaged_ms`	Someone was actually interacting — within 30 seconds of a real scroll, click, keypress or pointer movement, while visible. The honest number.

Because the beacon flushes on pagehide with navigator.sendBeacon(), Loghound measures the last page of the session — which log-based tools structurally cannot see, because there is no subsequent request to measure against.

Screenshots

See all 19 screenshots on the Loghound page →

Every screen is captured from the reference installation reading opensolr.com's own traffic, with a short explanation under each one: the four time-on-site clocks, sessions split into five populations, top pages, country by verdict, the live stream, recent visitors with the facet rail, the map, channels, bounce rate, the hour-by-weekday heatmap, SEO Tools period against period, why each session was scored, bot class by network type, crawler impersonation, attack probes over time, fingerprint clusters (the hero view: 721 addresses, one fingerprint), latency over time, slowest paths and status codes.

The three-plane model

The verdict matrix is the core idea:

Plane 1 (the log)	Plane 3 (the beacon)	Verdict
HTML 200 served	no beacon ever arrived	`bot` — non-JS client
HTML 200 served	beacon arrived, headless signals positive	`bot` — headless automation
HTML 200 served	beacon arrived, zero interaction, single page, left in under 10s	`unknown` — 55 points, deliberately short of a verdict
HTML 200 served	beacon + interaction + plausible timing distribution	`human`

GoAccess and AWStats only ever see plane 1. Clicky, GA4, Plausible and Matomo-JS only ever see plane 3 — and they trust it. A client that runs JavaScript is a human, as far as they are concerned. Loghound sees all three and cross-checks them.

Quickstart

git clone https://github.com/phpcip/loghound.git
cd loghound

# Look before you leap. This changes nothing.
sudo ./install/install.sh --dry-run

sudo ./install/install.sh

Two installers, one result

Setup happens either in a browser or in a shell, and neither is a lesser version of the other. Both drive the same code in src/Setup/ — the same log detection, the same format grading, the same index provisioning, the same password rules — so a configuration written by one is indistinguishable from one written by the other, and you can start in the browser and finish over SSH.


In a browser — the normal path	Open the site and you land on the installer: a system check with the exact command for anything that is not passing, then three screens — access logs, storage, sign-in. docs/INSTALL-WEB.md
In a shell	`bin/loghound-setup` over SSH. This is what `install/install.sh` hands over to, and what unattended installs use. docs/INSTALL.md

install.sh hands over to the shell wizard by default; --skip-setup stops once the machine is ready so you can finish in the browser.

Anything not ready to serve lands on the browser installer — no configuration, half a configuration, or a configuration with no way to sign in. Loghound never answers a request by printing a configuration error and stopping. Once the configuration is complete every installer route is dead.

Installs to /opt/loghound by default; --prefix=/srv/loghound (or anywhere else) works and every generated artefact follows it — the vhost docroot, the FPM pool, the systemd units, the deny rules. If the prefix is a git working copy, --upgrade is a fast-forward pull in place.

After every upgrade, run bin/loghound-schema. New code can write a field your indexes do not declare yet, and the only dynamic field in either schema maps everything unknown to ignored — so Solr accepts the document, throws that value away, and reports no error at all. The command compares what this release writes against the schemas your indexes are actually running, --apply pushes the configsets to fix it, and its exit codes (0 up to date, 3 out of date, 2 could not tell) are meant for a deployment script. The panel's Settings page shows the same verdict and says how old it is.

The installer:

checks PHP ≥ 8.1 and the five extensions it needs, and refuses clearly if any is missing;
creates the loghound system user and adds it to adm for read access to /var/log;
lays out /opt/loghound with the code owned by root and only var/ writable;
installs and enables the hardened systemd units and the two timers;
writes a dedicated PHP-FPM pool and a vhost for whichever web server you already run, validates both before enabling them, and rolls back if validation fails;
verifies the TLS certificate actually chains to a trusted root before deploying it (a Let's Encrypt staging certificate is refused — it has valid dates and the right hostname, and produces unable to get local issuer certificate in every browser);
hands over to the setup wizard, starts the daemon, and waits for the first documents to arrive before telling you it worked.

It never modifies an existing vhost, pool, cron entry or service, never overwrites a file it did not write, and uses reload rather than restart so other sites on the box are undisturbed. --dry-run prints every action and changes nothing.

sudo ./install/uninstall.sh reverses it — a thin wrapper around install.sh --uninstall, so there is one implementation rather than two that drift. It stops and removes the units and both timers, removes the vhost only after proving the web server still validates without it, removes the FPM pool, overwrites the credentials before unlinking them, and drops the service user's adm membership before trying userdel. It offers, as a separate question defaulting to no, to delete the two Opensolr indexes this installation provisioned — names derived from solr.install_id, matched against ^loghound_[a-f0-9]{8}_(hits|sessions)$, and cross-checked against your account's own index list. If ownership cannot be proven it deletes nothing and prints the names instead; nothing else in your account is ever touched. Afterwards the account is listed again and each name is reported gone or still present, because a control plane that accepted a delete has made a weaker claim than the index being gone — and you are about to stop being billed for it. --dry-run walks the whole teardown and changes nothing, and the run ends with a list of what it deliberately left behind — including the beacon <script> tag, which only you can remove.

The teardown is eleven numbered steps, printed as ==> [3/11] Deleting the Loghound indexes, and the panel runs the same eleven under the same names: Settings → Remove Loghound entirely does the five that need no root and shows the other six with the reason and the command. See docs/INSTALL.md.

Your log files are opened read-only and are never written to, truncated, rotated, renamed or deleted. Every source path is opened 'rb' and no other mode string appears near one anywhere in the codebase; there is a test in the suite that asserts a source file's size is unchanged after a full read. Whatever else is already reading those files — fail2ban, a log shipper, your own scripts — is unaffected, and so is logrotate.

Ingest is a systemd daemon, not a cron job. Cron's floor is sixty seconds, and the point of the tailer is that a request shows up in the panel about five seconds after it was served. The scorer and the retention job are systemd timers. On a box with no systemd the installer falls back to cron for those two and tells you plainly that the daemon then needs a supervisor of your own.

Either installer reads your Apache or nginx configuration, finds your LogFormat and CustomLog directives, and shows you the mapping with five of your own log lines rendered as parsed records and a confidence percentage, then asks you to confirm. Nothing is ever ingested with a silently guessed format.

Then:

sudo systemctl enable --now loghound-tail.service
loghound-tail --status --human

And add one line to your site, in <head>:

<script src="https://loghound.example.com/b.js?v=1789104523" defer></script>

On a site this machine has no access log for — a search page, a marketing site, anything on another server — that same line is the whole installation, with one extra step: add that hostname to beacon.allowed_hosts in config/loghound.php. Until it is listed the beacon stages a row and nothing more, so no session is created and no search term is kept. It is a permission rather than a password: a browser cannot forge Origin, so an ordinary page cannot impersonate a host you listed, but anything that is not a browser can — which is why a session measured by the beacon alone is stored and shown as single-plane.

Do not copy that version. b.js is served with a long immutable cache, so the ?v= is what makes an upgrade reach a returning visitor at all — and a snippet pinned to a literal never will. Take the line from the panel's Settings → Beacon card, or from bin/loghound-setup --beacon-doc: both build it with your own URL and the ?v= taken from the file's own modification time, which is the only value that is ever correct.

The site does not have to be on this machine. That same line is the whole install on a host with no Loghound and no shared access log — a hosted search page, a marketing site, anything on another server. The page reports its own hostname, the collector cross-checks it against the browser-set Origin and against beacon.allowed_hosts, and the session is created from the beacon alone and marked as single-plane, because one plane is the plane a determined client controls.

It can carry more than a URL. Six attributes, two globals and one function: data-ident and data-signed-in attach an identity your site already has — the templated email from WordPress's or Drupal's own user object, not a literal — data-params names the query parameters to keep as search terms, and window.loghound.identify() covers a single-page app that signs somebody in after load. Whether Loghound stores what they send is a separate, server-side decision (beacon.store_identity is off by default). The panel's Settings → Beacon card renders the full option table with a column saying what your installation will actually keep; the same table, with the platform snippets, is docs/BEACON.md §1.1.

No Composer dependencies. No vendor/. No npm. No build step. git clone and install.sh on a bare box — or composer create-project opensolr/loghound, which downloads the same tree and installs nothing else.

Full details, including the LogFormat block that makes detection substantially stronger: docs/INSTALL.md.

What this does NOT catch

An honest limitations section, because a bot detector that oversells itself is worse than none — you would stop looking.

A well-funded, careful scraper. If an operator gives every session a genuinely unique header fingerprint, runs a real (non-headless) browser, drives it with synthesised mouse movement and scrolling, paces requests with human-like variance, and never reuses an exit IP, Loghound will classify it as human. Everything here raises the cost of scraping. None of it makes scraping impossible, and anyone claiming otherwise is selling something.

Privacy-conscious humans get flagged. This is the single biggest source of false positives and it deserves to be first. A visitor running uBlock Origin, NoScript, Brave's shields or Safari's Lockdown Mode may never load b.js. From plane 1 they look exactly like a non-JS client: HTML 200 served, no beacon, real browser UA. The no_js_on_html rule is weighted at 70 rather than 100 precisely because of this, and it is not enough on its own to reach the bot threshold. Read the false-positives section of docs/DETECTION.md before you act on any single verdict.

Traffic that never reaches your origin. Loghound reads your access logs. A request served from a CDN edge cache, from Cloudflare, or from Varnish in front of you produces no origin log line and does not exist as far as Loghound is concerned. If you run a CDN, you are analysing your cache misses.

Requests behind a proxy that does not forward the client IP. If X-Forwarded-For is not logged, or the proxy is not in your trusted_proxies list, every visitor appears to come from the proxy. IP-based clustering and geo become meaningless. Loghound will not silently pretend otherwise, but it also cannot fix it.

Low-and-slow scrapers. One page, one IP, one fingerprint, once a day, from a residential ASN. It is below every threshold in the ruleset by construction. The fingerprint-cluster signal — the strongest thing here — needs a fleet to detect a fleet.

API and mobile-app traffic, unless you also point Loghound at that vhost's log. There is no beacon in a native app, so plane 3 is permanently blind there and every client looks non-JS.

Carrier-grade NAT. Thousands of real mobile users share a small pool of addresses. That is why fp_cluster_proxy_fleet explicitly excludes as_type_s = mobile, and it is why an ASN misclassification turns into a batch of false positives.

It is not a WAF and it blocks nothing. Loghound observes and reports. It does not issue challenges, it does not rate-limit, it does not write firewall rules. Deciding what to do about what it finds is your job, and the reason every verdict ships with its reasons attached.

It does not do product analytics. No funnels, no goals, no A/B tests, no revenue attribution, no cohort retention. If you need those, use Matomo or GA4 — and see below.

How it compares

Written to be fair. Each of these tools is good at what it set out to do, and for most people one of them is the right answer.

	Loghound	GoAccess	Matomo	Plausible	GA4
Data source	logs + JS beacon	logs only	JS (log importer available)	JS only	JS only
Bot detection	3-plane correlation, 17 weighted rules, every verdict carries its reasons	UA blocklist	UA/IP blocklist, some heuristics	UA blocklist	undisclosed, not inspectable
Detects headless Chrome on residential proxies	yes, that is the point	no	no	no	no
Cross-IP fingerprint clustering	yes — the primary signal	no	no	no	no
Time-on-site	4 distinct numbers incl. engaged time and the last page	request span only	tab-open time	tab-open time	tab-open time
Storage	Opensolr (managed Solr 9)	in-memory / on-disk report	MySQL/MariaDB	ClickHouse	Google
Install	`git clone` or `composer create-project` + `install.sh`, no dependencies	one binary, apt/dnf	PHP app + DB, or cloud	Docker/Postgres, or cloud	none
Real-time	yes (softCommit, ~5s)	yes, genuinely instant	near	near	delayed
Cost	free, MIT	free, MIT	free self-hosted, paid cloud	paid cloud, free self-hosted	free

Loghound is narrow on purpose. If your question is "how many people read my blog post", use Plausible. If your question is "which campaign drove revenue", use GA4 or Matomo. If your question is "how much of this traffic is actually real, and can you prove it", that is the one Loghound was built to answer.

Loghound also works alongside them. It reads logs the web server is already writing; the beacon is independent of whatever other analytics you run.

Documentation


docs/INSTALL-WEB.md	Setting up from a browser: the system check, the setup token, the three screens, how the long operations run as jobs, and what each screen writes into the configuration
docs/INSTALL.md	Installation and the shell wizard: what `install.sh` does, the permission model, the recommended `LogFormat`, exactly which signals each extra header buys you, Solr and Opensolr setup, upgrading, troubleshooting
docs/DETECTION.md	The three planes, every rule with its weight and rationale, a worked example on real captured traffic, and a frank section on evasion and false positives
docs/ATTACKS.md	Hostile requests: every pattern with what it matches, what it misses and what it over-reports; why the status code is the whole point and why a 2xx still proves nothing; crawler impersonation
docs/BEACON.md	The beacon: the three clocks, every field it sends, every signal code, the wire protocol, and what it cannot detect
docs/PANEL.md	The web panel: the async contract every card follows, stepped jobs, request flow, the notes a security reviewer wants, the design system, demo mode
docs/SCHEMA.md	Both Solr cores field by field, why each is indexed/docValued/stored, index-size arithmetic, and what to turn off first
docs/SECURITY.md	Threat model, controls by surface, hardening checklist, known limitations, how to report a vulnerability
docs/PRIVACY.md	Exactly what is collected, the three IP modes, retention, GDPR posture, and what to tell your users
SPEC.md	The full technical specification, including the Solr schema
CONTRIBUTING.md	What will and will not be merged, the test runner, fixture rules

Requirements

PHP 8.1 or newer with curl, json, pcre, sqlite3, mbstring. No Composer packages — the manifest declares the PHP version and those extensions and nothing else, so there is no vendor/ directory and nothing to build.
An Opensolr account, and it is a hard requirement. Loghound provisions and manages its own two indexes there — it creates them, uploads their configsets, reloads the cores and verifies them — and it cannot do that on a Solr it does not administer, so there is no option to point it at one you run yourself.

Opensolr is free forever to start — no credit card, no expiry date. Create an account · Sign in · What the plans hold

Retention scales with the plan rather than being cut off by it: Loghound trims its oldest data before the account reaches its disk limit, so it keeps running on the free tier and simply holds less history. More disk buys more history. Setup asks for the account email and the API key from Account in the control panel, and provisions both indexes itself.

Everything settled during installation can be changed afterwards in Settings, without starting over: the Opensolr account and API key, which pair of indexes this installation uses, which log files are read, the panel username and password, and how much data is kept. There is also a Start over button, and it means it: it deletes both indexes from your Opensolr account, proves they are gone, removes the configuration with your account details in it, empties var/, and hands you back to the installer with nothing carried across. The only way to keep the data is an Opensolr backup taken beforehand, which is a separately billed feature.

Two indexes, however many sites. A plan also limits how many indexes an account may hold, so setup counts what the account has before it creates anything and, when there is no room, says so in plain numbers rather than failing halfway with one index created. It also offers you any pair of Loghound indexes the account already holds: joining one creates nothing, and several sites reporting into a single pair stay separable in the panel by the hostname on every record. Joining never clears, reshapes or overwrites what is already there.
systemd, for the daemon and the two timers.
Read access to your access logs. The installer puts the service user in adm.

Status

v1.0.0. See CHANGELOG.md.

Contributing

See CONTRIBUTING.md. Run php tests/run.php before opening a pull request; it needs no network access and no dependencies.

License

MIT. See LICENSE.

opensolr / loghound

Package info

Security

Fund package maintenance!

Statistics