hei / laravel-scarlett-player
The server side of Scarlett Player for Laravel: analytics beacon ingest, clip generation, player configuration and the embed route
Package info
github.com/Hackney-Enterprises-Inc/laravel-scarlett-player
pkg:composer/hei/laravel-scarlett-player
Requires
- php: ^8.3|^8.4|^8.5
- illuminate/auth: ^12.0|^13.0
- illuminate/bus: ^12.0|^13.0
- illuminate/cache: ^12.0|^13.0
- illuminate/console: ^12.0|^13.0
- illuminate/contracts: ^12.0|^13.0
- illuminate/database: ^12.0|^13.0
- illuminate/events: ^12.0|^13.0
- illuminate/filesystem: ^12.0|^13.0
- illuminate/http: ^12.0|^13.0
- illuminate/process: ^12.0|^13.0
- illuminate/queue: ^12.0|^13.0
- illuminate/routing: ^12.0|^13.0
- illuminate/support: ^12.0|^13.0
- illuminate/validation: ^12.0|^13.0
- illuminate/view: ^12.0|^13.0
Requires (Dev)
- larastan/larastan: ^3.11
- laravel/pint: ^1.0
- laravel/sanctum: ^4.0
- orchestra/testbench: ^10.0|^11.0
- pestphp/pest: ^4.0
- pestphp/pest-plugin-browser: ^4.0
Suggests
None
Provides
None
Conflicts
None
Replaces
None
This package is auto-updated.
Last update: 2026-10-13 19:57:23 UTC
README
The server side of Scarlett Player for Laravel. One package, modules toggled in config:
- Beacons: receives the analytics plugin's beacons, on both transports, and aggregates them into views.
- Clips: accepts clip requests from the clips plugin and renders them on a queue.
- Player and embed: builds the player configuration for a host model, ships a Blade component, and serves the embed page and oEmbed endpoint.
All three share one media contract: the host tells the package, once, what a player mediaId means.
Status: in development. The scaffold, config, facade, fake and
scarlett:doctorare in place; the module sections below fill in as each module lands.
Requirements
- PHP 8.3, 8.4 or 8.5
- Laravel 12 or 13
Install
composer require hei/laravel-scarlett-player php artisan vendor:publish --tag=scarlett-config
The service provider and the ScarlettPlayer facade alias are auto-discovered.
Set the beacon key the analytics plugin sends:
SCARLETT_BEACON_KEY=
Then check the installation:
php artisan scarlett:doctor
Upgrading from 0.1
- Server-side context columns. A
scarlett_viewstable migrated by 0.1.0 needs two nullable json columns,serverandserver_stamps, before you setbeacons.context(see Server-side context; thebeacon contextdoctor check fails until they exist). Without a resolver nothing reads or writes them, so a host that does not use the hook needs no migration. scarlett:clips:reconcilecan exit non-zero. A clip visibility write that fails (afalsefrom a disk configuredthrow => false, or a disk that throws) is now reported and fails the run instead of counting as re-synced. A scheduler or monitor that alerts on failed commands will see it; the clip stays in the pass's window and is retried on the next run.- One new event key for redacted beacons. The raw log's
event_keynow hashes the beacon as the browser sent it. A beacon that abeacons.pipelinestep redacts therefore gets a new key once across the upgrade: a v0.1.0 delivery and its v0.2.0 retry are stored as two raw rows. Beacons no step changes keep their v0.1.0 key.
Upgrading from 0.2
Add the raw-log order column. Fresh installs get nullable seq and its composite
index from the package's create migration. If you already published and ran the 0.2
migration, add these two schema lines in a new, host-owned migration:
use Illuminate\Database\Schema\Blueprint; use Illuminate\Support\Facades\Schema; Schema::table('scarlett_beacon_events', function (Blueprint $table): void { $table->unsignedInteger('seq')->nullable(); $table->index(['view_id', 'occurred_at', 'seq']); });
Keep the existing view_id index. The 0.3 raw-order change has no package upgrade
migration. Do not republish the full migration folder on an existing installation:
Laravel normally assigns new timestamps and can create duplicate create migrations. The signals
release adds a separate migration described in Signals and upgrading from 0.3.
The store tolerates a missing seq column and continues ingesting, but cannot write the
dedicated order key until you add it. scarlett:doctor warns under beacon seq column
when raw-event storage is on and the column is missing; it passes when the column exists
or beacons.store_raw_events is off. Restart long-running queue workers after migrating,
since the column check is cached per store instance.
Two keys stop being custom dimensions. Numeric beaconSeq and string seekSource
are now known fields. The player sends beaconSeq on every beacon and seekSource
(player or element) on seeking, from 1.19.3. Neither adds a view-row metric;
beaconSeq supplies raw seq, and both remain in the raw payload. Wrong types still
fall back to custom, and null is absent. If you deployed player 1.19.3 first, ignore
any old beaconSeq / seekSource keys already in scarlett_views.custom: this upgrade
does not remove them or backfill or reorder old raw rows. The view-row merge is unchanged.
Deduplication stays compatible with 0.2.1. Promoting these two keys does not change
the normalized browser-payload basis used for bodyHash and event_key. For hashing,
they retain their original interleaving with custom keys under the 0.2.1 classification,
independently of their new known-field classification for storage. Existing queued
payloads retain their stored hashes. Replaying the same request accepted by 0.2.1 must
not create a second raw row or error row, or repeat PlaybackErrorReported, solely
because these keys became known. This is not a new sorted-key or raw-JSON-byte hash.
The portable SQL seq range is 0 through 2147483647, even on engines that could
store larger integers. Overflow omits the SQL key rather than rejecting the beacon or
clamping to the maximum; the finite original value stays in the raw payload and view
aggregation continues. See Reading the raw log for rounding
and NULL behavior. This bound requires no migration change beyond the column/index
recipe above.
Deploy the ingest first. For tsp-web (and other existing hosts): run
composer update hei/laravel-scarlett-player to 0.3.0, add the column and index, deploy
the ingest, then move the host's @scarlett-player/* packages to 1.19.3. In 0.3.0 the
package's player.player_version remained 1.19.1 (0.5.0 moves it to 1.22.0; see
Player 1.22). Signals support uses the additional
migration below. See
Compatibility for the count changes to expect from player 1.19.3.
Publish tags
| Tag | What |
|---|---|
scarlett-config |
config/scarlett-player.php |
scarlett-migrations |
All package migrations, for fresh installations only |
scarlett-migrations-signals |
Only the additive signals migration, for upgrades from 0.3 |
scarlett-migrations-reconnects |
Only the additive player 1.22 migration (reconnects, element seeks, DVR time, error context), for upgrades from 0.4 |
scarlett-migrations-gauges |
Only the additive canonical gauges migration (completion_ratio, rebuffer_fraction with stamps), for upgrades from 0.5 |
scarlett-migrations-diagnostics |
Only the additive diagnostics migration (decoded_frames_total, dropped_frames_total, indexed viewer_ref), for upgrades from 0.7 |
scarlett-views |
The embed page and Blade component views, to resources/views/vendor/scarlett |
scarlett-js |
The JS initialiser, to resources/js/vendor/scarlett-player/init.js |
Upgrade migrations
A fresh installation publishes only scarlett-migrations: from 0.4.0 that folder
contains the signals upgrade, from 0.5.0 the player 1.22 upgrade, from 0.6.0 the
canonical gauges upgrade and from 0.8.0 the diagnostics upgrade, in order. The upgrade tags are for hosts that installed
before that upgrade existed. Each publishes
one migration, dated at publish time, so it sorts after create migrations published
earlier and a fresh migrate (a new environment, a test database) creates the tables
before altering them. Dating does not guarantee that order on its own: Laravel restarts
its clock for every vendor:publish, so an upgrade tag published in the same second as
the full folder gets the same timestamp and sorts first. An upgrade that runs before its
tables exist stops with UpgradeMigrationOrderException, naming the file, before it
changes anything: delete that copy if the full folder already contains the upgrade, or
rename it to sort after the create_scarlett_* migrations. Dating needs
database.migrations.update_date_on_publish on, which is the Laravel 11+ default. An
application whose config/database.php still says 'migrations' => 'migrations' (the
older string form) has it off and gets the file under its 0001_01_01_... name: set
'migrations' => ['table' => 'migrations', 'update_date_on_publish' => true] before
publishing, or write your own dated migration (below).
All four upgrade migrations are idempotent (the first three from 0.6.0, the diagnostics one
from its first release). A column or index that already exists is left alone, so publishing a tag twice (each publish adds another dated copy), a database that
already ran an earlier copy, or a host that added the columns itself all migrate as a
no-op. The signals migration adds qoe_version first, on its own, and marks existing
scores as QoE v1 straight after, so the backfill runs in the run that adds the column.
MySQL schema changes are not transactional; if a run stops partway, migrate again: a
rerun that finds qoe_version as the only signals column cannot tell whether the
backfill finished, so it marks scored rows still without a version as v1 again. Rows that
already carry a version are never changed. Rolling a copy back drops its columns only when no other copy of the
same upgrade (a migration whose name ends _add_signals_to_scarlett_tables,
_add_reconnects_to_scarlett_tables, _add_gauges_to_scarlett_tables or
_add_diagnostics_to_scarlett_tables) is still recorded as run.
Published with 0.4.x or 0.5.0? Those tags wrote the fixed names
0001_01_01_000005_add_signals_to_scarlett_tables.php and
0001_01_01_000006_add_reconnects_to_scarlett_tables.php. Next to dated create
migrations they sort first: the database that ran them is correct, but a fresh
migrate fails with "no such table". The beacon upgrade migration order doctor
check warns about it, and about any copy (fixed or dated, in database/migrations or
any registered migration path) that sorts before the create migrations it needs. To repair, delete the fixed-name file and publish its tag again
(the publish skips while that file exists), then migrate; the dated copy runs as a
no-op on the existing database and in order on a new one:
rm database/migrations/0001_01_01_000005_add_signals_to_scarlett_tables.php rm database/migrations/0001_01_01_000006_add_reconnects_to_scarlett_tables.php php artisan vendor:publish --tag=scarlett-migrations-signals php artisan vendor:publish --tag=scarlett-migrations-reconnects php artisan migrate
Remove only the files you have. The old rows stay in the migrations table; they are
harmless, and they keep a rollback of the dated copies from dropping the columns. If
your create migrations kept their 0001_01_01_... names too, nothing sorts out of
order and there is nothing to repair.
Owning the schema instead. A host that writes its own dated migrations can copy the
column lists from the upgrade sections below (every column nullable; *_at columns are
dateTime with precision 3) and skip the tags. Name the file with the package's suffix
(..._add_signals_to_scarlett_tables.php, ..._add_reconnects_to_scarlett_tables.php,
..._add_gauges_to_scarlett_tables.php, ..._add_diagnostics_to_scarlett_tables.php) if a published copy may also run, so a rollback of that copy keeps your columns.
Routes
Each module's routes load only when its switch is on, and each has its own middleware list, so a host overrides one without touching the others:
'routes' => [ 'prefix' => env('SCARLETT_ROUTE_PREFIX', 'api/scarlett'), 'middleware' => [ 'beacons' => ['api', 'throttle:scarlett-beacons'], 'clips' => ['web', 'auth', 'throttle:scarlett-clips'], 'embed' => ['web'], 'oembed' => ['api'], ], 'beacons' => true, 'clips' => true, 'embed' => true, // the embed page and the oEmbed endpoint ],
Route names are scarlett.<module>.<action>. The embed page lives at embed.route
(/v/{uuid} by default), outside the prefix. The scarlett-beacons and scarlett-clips
rate limiters read beacons.throttle and clips.throttle ("attempts,minutes").
Media contract
Every module asks the same question: what is this mediaId? The answer is a
Hei\ScarlettPlayer\Data\MediaSource, and the host supplies it one of three ways.
-
The model implements
ScarlettMedia(preferred). Add theHasScarlettClipstrait for the clip relation and ascarlettMediaId()default, and write one method:use Hei\ScarlettPlayer\Concerns\HasScarlettClips; use Hei\ScarlettPlayer\Contracts\ScarlettMedia; use Hei\ScarlettPlayer\Data\MediaSource; class Video extends Model implements ScarlettMedia { use HasScarlettClips; public function toScarlettMediaSource(): MediaSource { return new MediaSource( id: $this->uuid, playbackUrl: $this->hls_url, isLive: $this->is_live, isProtected: $this->is_ppv, duration: $this->duration_seconds, title: $this->title, poster: $this->poster_url, model: $this, ); } }
Point
media.modelat the class andmedia.keyat the column the player'smediaIdholds. -
A complete attribute map in config, for hosts that cannot touch the model:
'media' => [ 'model' => App\Models\Video::class, 'key' => 'uuid', 'attributes' => [ 'playback_url' => 'hls_url', // required 'is_live' => 'is_live', // required 'is_protected' => 'is_ppv', // required 'duration' => 'duration_seconds', 'source_disk' => 'disk:mezzanine', // a column, or a literal disk name 'source_path' => 'mezzanine_path', 'title' => 'title', 'poster' => 'poster_url', ], ],
-
Your own resolver for anything non-trivial (signed URLs, entitlement, multi-tenant): a class implementing
Hei\ScarlettPlayer\Contracts\ResolvesMedia, named inmedia.resolver. Returnnullfor an unknown id.
isLive and isProtected have no defaults, and protection fails closed: a mapping
without is_protected throws IncompleteMediaMappingException rather than assuming the
media is free. scarlett:doctor reports an incomplete mapping at install time.
How the default resolver (Hei\ScarlettPlayer\Media\ConfigModelResolver) behaves:
- It loads
media.modelwheremedia.keyequals themediaId. A nullmedia.keyuses the model's route key. No matching record returnsnull, which the package answers with a 404. - A model implementing
ScarlettMediaalways resolves throughtoScarlettMediaSource(); any attribute map is then ignored. - The mapping is checked before the lookup, so an incomplete map throws even for an
unknown id. The exception's
missingproperty lists the config keys to fill in. - Values fail closed as well as mappings: if a record's
is_protectedoris_livecolumn is null or not a readable boolean (1,0,true,false,yes,no,on,off), or itsplayback_urlcolumn is empty, it throws rather than guessing. source_diskprefixed withdisk:is a literal disk name; anything else is a column.HasScarlettClips::scarlettMediaId()returns the model's route key. Override it (orgetRouteKeyName()) when the player is given a different id.
Resolve a media id yourself through the facade:
use Hei\ScarlettPlayer\Facades\ScarlettPlayer; $source = ScarlettPlayer::resolve($mediaId); // throws MediaNotFoundException for an unknown id $builder = ScarlettPlayer::for($video); // a model or a mediaId
Beacons
The analytics ingest for @scarlett-player/analytics: one route, POST {prefix}/beacons
(scarlett.beacons.store), that answers 204 before any aggregation and queues a
ProcessBeacon job on beacons.connection / beacons.queue. The player ignores the
response, so a slow or failing store costs the host, never the viewer;
scarlett:beacon:test and scarlett:doctor are how you find out.
Wiring the player
createAnalyticsPlugin({ beaconUrl: 'https://app.example.com/api/scarlett/beacons', // must be https apiKey: '...', // SCARLETT_BEACON_KEY videoId: 'the mediaId', heartbeatInterval: 10000, // optional, ms; player.heartbeat_interval (seconds) sets it customDimensions: { plan: 'ppv' }, // optional, stored in custom });
The plugin sends its key only when beaconUrl is https:; there is no localhost
exception. On fetch (every in-session beacon) it sends X-API-Key. On page unload it
uses navigator.sendBeacon, which cannot carry a header, so it appends ?api_key=
instead. The route accepts either, compared with hash_equals, and refuses everything
while beacons.key is unset.
The key lands in access logs. ?api_key= is part of the URL, so nginx, a load
balancer or any proxy in front of the app records it. It is a low-privilege ingest key
that can only post beacons; never reuse a real API token for it. To keep it out of nginx
logs, log $uri rather than $request for this location:
log_format scarlett '$remote_addr [$time_local] "$request_method $uri" $status'; location /api/scarlett/beacons { access_log /var/log/nginx/access.log scarlett; }
Do the same wherever else a full URL is recorded: turn off query-string logging for the
beacon path at a load balancer or CDN, and add api_key to the scrubbed parameters of any
error tracker or request inspector (Sentry, Telescope, Flare) that captures request URLs.
If the key does leak, rotate it: set a new SCARLETT_BEACON_KEY and ship the same value in
the player's apiKey.
The route is stateless by necessity: the unload beacon cannot carry a CSRF token either,
so it runs in the api group with the scarlett-beacons rate limiter
(beacons.throttle, default 600,1 per IP; heartbeats are frequent).
CORS: the complete recipe
An embed iframe on embed.example.com beaconing to app.example.com is cross-origin. A
Blob of type application/json is not a CORS-safelisted content type, so both transports
preflight. The two transports then diverge:
| Transport | Credentials mode | Response the browser requires |
|---|---|---|
fetch (in-session, and the unload fallback) |
same-origin (default) |
Access-Control-Allow-Origin: * is enough |
navigator.sendBeacon (unload) |
include |
Access-Control-Allow-Origin: <exact origin> and Access-Control-Allow-Credentials: true; a * origin fails the request with no error visible anywhere |
So an ingest that "works" in the network tab can still be losing every unload beacon, the
one carrying the session totals. The package does not write config/cors.php for you; it
is host-owned config. Use this block:
'paths' => ['api/scarlett/beacons'], // the configured prefix + /beacons 'allowed_methods' => ['POST', 'OPTIONS'], 'allowed_origins' => ['https://embed.example.com'], // explicit; never '*' when supports_credentials is true 'allowed_origins_patterns' => [], 'allowed_headers' => ['Content-Type', 'X-API-Key'], 'exposed_headers' => [], 'max_age' => 600, 'supports_credentials' => true,
- Preflight is unauthenticated. Laravel's
HandleCorsanswersOPTIONSas global middleware, before the route.ScarlettApiKeyalso letsOPTIONSthrough, so a host that reorders middleware still cannot 401 a preflight. scarlett:doctorwarns when beacons are enabled and the beacon path is absent fromcors.paths,supports_credentialsis false,allowed_originscontains*alongsidesupports_credentials, orX-API-Keyis missing fromallowed_headers.scarlett:beacon:testcannot prove any of this; only a browser enforces CORS. The package's browser test does: the real plugin, cross-origin over HTTPS, receives the unloadviewEndwith the recipe and loses it withsupports_credentialsfalse.- Even with the recipe, Chromium can lose an unload beacon. Because the unload
beacon preflights, a same-site navigation that tears the page down while the
preflight is in flight can make Chromium drop the POST after a successful preflight
(about 1 in 3 in the package's headless Chromium 153 runs; much rarer cross-site).
The ingest treats a missing unload
viewEndas normal (the heartbeats carry the running totals), but session totals from same-site navigations are best effort.
What is accepted
One JSON object per request (a batched array is refused by name). The only things that
can refuse a beacon are the event, the four ids (viewId, sessionId, viewerId,
videoId) and the timestamp being missing or malformed (422), or the body exceeding
beacons.max_body_bytes (413, default 64 KiB). Everything else is accepted:
- Unknown keys are stored, never rejected, in the view's
customcolumn. A newer player's keys and yourcustomDimensionsboth land there. - A known key carrying another type is treated as a custom dimension of that name (the
player spreads
customDimensionsamong its own keys, so a dimension calleddurationarrives under a known name). - A key sent as
nullis treated as absent, exceptqoeScore: explicit null clears the score so access-denied views are excluded from averages. So player 1.22's nullavgBitrate,maxBitrateand (live)completionRatenever overwrite a known value. Long strings are truncated to their column.
What is stored
EloquentBeaconStore (the default, beacons.store = eloquent) keeps three tables.
Publish and run the migrations:
php artisan vendor:publish --tag=scarlett-migrations php artisan migrate
| Table | Rows | Pruned by |
|---|---|---|
scarlett_views |
one per viewId, merged from every beacon of that view |
retention.views days since its last update |
scarlett_beacon_events |
every beacon as received, while store_raw_events is on (about 90% heartbeats), except the event names in raw_events_except |
retention.events days since received |
scarlett_view_errors |
one per error beacon, with its video_id |
with its view |
Beacons arrive duplicated and out of order by design (keepalive fetch and sendBeacon
guarantee no order, a bfcache restore re-sends, a queue redelivers), so the view row is
merged per field, never per event:
| Class | Columns | Rule |
|---|---|---|
| Set-once | identity and environment except is_live, started_at, first_frame_at, ended_at, exit_type; viewer_ref (0.8.0, from the server context) |
the first beacon to arrive with a value keeps it; a second viewEnd cannot move ended_at |
| True wins | is_live |
any beacon with isLive: true sets it and nothing clears it; false only fills an empty column. A live viewStart fires before the player has read the playlist and says false (from player 1.18 it sends null, treated as absent), and a live stream that becomes a replay in the same session stays live |
| Monotonic | watch_ms, play_ms, rebuffer_ms, rebuffer_count, seek_count, pause_count, quality_changes, error_count, max_bitrate, startup_ms, warning_count, element_seek_count, reconnect_count, reconnect_ms, dvr_ms, pause_ms, decoded_frames_total, dropped_frames_total |
the larger value wins, whatever the order |
| Latest by timestamp | qoe_score, avg_bitrate, rebuffer_ratio, completion_rate, current_position, media_duration, the live latency summary |
written when the beacon is at least as new as the one that wrote that column (a *_at stamp per column); metrics_at is the newest |
| Canonical gauges | completion_ratio, rebuffer_fraction (0.6.0) |
the normalized 0..1 measurement of completionRate / rebufferRatio, each with its own stamp: written when the stamp is absent or the beacon is at least as new as the stamp, even when the stored value is null (a processed unavailable measurement is never refilled by an older numeric). Once is_live is true, completion_ratio stays null whatever arrives |
| Fill if absent | every column | a key absent from the beacon, or null, never touches its column: a null avgBitrate leaves the last known one |
Two viewEnd beacons for one view (the player can send both) merge. Before player 1.22
the unload variant was a subset and the other filled what it lacked; from 1.22 both
carry the same fields, and the first to arrive keeps ended_at and exit_type. Counters are never incremented from the event
stream; they are the player's own running totals. custom merges key by key, with a
stamp per key (custom_stamps): each key keeps the value from the newest beacon that
sent that key, so an older beacon delivered late still wins a key no newer beacon
wrote. A nested value is replaced whole, the same on every engine (the player itself
sends only flat scalars); custom_at is the newest stamp.
The same beacon delivered twice has the effect of once: raw events and errors are unique
on an event_key, the view row's rules are idempotent, and the transition events fire on
observed transitions only. The merge is three statements in one transaction (an
insert-or-ignore that decides ViewStarted, a guarded ended_at update that decides
ViewEnded, and one conditional update for the merge classes), each atomic on its row and
identical on SQLite, MySQL and Postgres. The view is linked to your model
(viewable_type, viewable_id) through ResolvesMedia after the transaction commits,
best effort; a beacons-only install with no media.model is never asked.
viewable_id is a string column so integer and uuid keys both fit; the package defines no
relation on it. A morphMany from your model to scarlett_views must use a string local
key on Postgres, which refuses to compare varchar with integer.
Signals and upgrading from 0.3
In 0.4.0 the default player pin remained 1.19.1; signals support targeted 1.20.0. Package 0.5.0 pins 1.22.0, which carries the signals contract.
For an existing 0.3 installation that has not run the signals migration, publish
only the additive migration with the dedicated tag below. It adds columns to
views and errors and marks existing non-null scores as QoE v1. The copy is dated at
publish time and is a no-op where the columns exist (see
Upgrade migrations, which also covers a fixed-name
0001_01_01_000005_... copy published by 0.4.x or 0.5.0). Fresh installations use
scarlett-migrations once for all migrations. Do not republish that full folder on
an existing host: Laravel's default timestamp rewriting can duplicate previously run
create migrations, even without --force. The separate host-owned raw seq upgrade
below is still needed before 0.3.
php artisan vendor:publish --tag=scarlett-migrations-signals php artisan migrate php artisan queue:restart php artisan scarlett:doctor
The migration adds, all nullable: on scarlett_views, qoe_version and
warning_count (unsigned integer), fatal_error_category (string), anonymous
(boolean), page_url and referrer_origin (text), page_load_to_init_ms and
player_init_ms (double), and for each of segment_count, segment_bytes,
segment_load_avg_ms, segment_load_max_ms, segment_errors,
segment_throughput_bps, decoded_frames and dropped_frames a double plus a
<name>_at dateTime with precision 3; on scarlett_view_errors, category and
severity (string), http_status, media_error_code and attempts (unsigned
integer), retries_exhausted, reconnect_exhausted and timed_out (boolean). A
host-owned migration also runs the backfill once, when it adds qoe_version:
update scarlett_views set qoe_version = 1 where qoe_score is not null.
The beacon signals columns doctor check warns about missing columns. Ingest
continues against older or partially upgraded schemas, skipping unavailable signal
columns. Legacy v1 scores still aggregate. Until qoe_version exists, v2 and
unknown-version scores aggregate as null to avoid contaminating the v1 series or
migration backfill; their original values remain in the optional raw log. Column
availability is cached per store instance: restart workers after migrating. Skipped
measurements are not automatically replayed into the new columns.
Deploy ingest and migrate first, then move the player (1.20.0 or later; 0.5.0 pins
1.22.0). Update player.player_version and module npm dependencies together, and
republish scarlett-js if the host uses the initialiser. Package defaults do not
replace a host's published config or JavaScript.
The recognized signal fields remain at the top level in the raw payload. Unknown
keys and known names carrying wrong types still go to custom. Existing custom
keys are not removed or backfilled. Field promotion preserves legacy event hashes
and queued hashes, so an upgrade alone does not duplicate raw events or errors.
| Signal | Persistence and merge |
|---|---|
qoeScore, qoeVersion |
qoe_score and qoe_version are one latest-by-timestamp measurement using qoe_score_at. Scores with no version key are v1; a present unusable version (including null or a string) has null qoe_version and is excluded from versioned averages. Explicit null clears the score and wins same-millisecond score ties; omission leaves it unchanged, including unload beacons |
warningCount |
warning_count, monotonic running total; never counted from sampled error beacons |
fatalErrorCategory |
fatal_error_category, first non-null value |
errorCategory, errorSeverity |
scarlett_view_errors.category and .severity, retained even when raw storage is off |
httpStatus, mediaErrorCode, attempts, retriesExhausted, reconnectExhausted, timedOut |
Corresponding snake_case columns on scarlett_view_errors; absent details stay null |
segmentCount, segmentBytes, segmentLoadAvgMs, segmentLoadMaxMs, segmentErrors, segmentThroughputBps |
One latest interval, guarded by segment_count_at. A valid segmentCount replaces the group; missing members become null, including omitted throughput in audio-only intervals. Without a valid count the group stays unchanged |
decodedFrames, droppedFrames |
Latest reported frame values, with individual timestamps; received intervals also remain in the optional raw log |
pageUrl, referrerOrigin, pageLoadToInitMs, playerInitMs |
Corresponding snake_case view columns, set once when present |
anonymous |
Boolean view column, true wins; absence stays null |
Segment and frame values are intervals, not cumulative view counters. Never merge them with MAX or sum the latest view rows as if they were lifetime totals. A missing native-HLS/MP4 segment measurement is unknown, not zero. Raw exclusions, privacy suppression and transport loss can leave gaps in interval history.
Access-denied fatal views report qoeScore: null; retain SQL NULL rather than
zero or a previous heartbeat's score. Average each scoring version separately:
$averages = DB::table('scarlett_views') ->selectRaw('qoe_version, AVG(qoe_score) AS average_qoe, COUNT(qoe_score) AS scored_views') ->whereNotNull('qoe_version') ->groupBy('qoe_version') ->get();
SQL AVG excludes null scores and includes a real zero. Do not coalesce null scores to zero or use a truthiness filter that removes zeroes. A group with only access denials has a null average. Compare QoE v1 and v2 as separate series.
Player 1.22 and upgrading from 0.4
Package 0.5.0 ingests everything @scarlett-player/analytics 1.22.0 sends and moves
the default player.player_version to 1.22.0 (from 1.19.1). Verified against the
published npm packages and the CDN's v1.22.0/ directory (see
Compatibility).
Upgrade order: update the package, publish and run the migration below, restart workers, deploy the ingest, then move the player:
- Embed mode follows
player.player_version: with the default (noSCARLETT_PLAYER_VERSION, no published config overriding it) the embed page and component loadv1.22.0/embed.jsand its addons as soon as the package deploys. A host that published the config or sets the env var keeps its own version until it changes it. - Module mode uses the host's own npm dependencies: move every
@scarlett-player/*package to 1.22.0 together, and setSCARLETT_PLAYER_VERSION=1.22.0if you pin it. Thescarlett-jsinitialiser did not change in 0.5.0. - From the 1.20.0 gate the builder writes the privacy flags explicitly: module mode's
analytics config gains
anonymous: falseandrespectDoNotTrack: false, and embed mode emitsdata-analytics-anonymous="false"anddata-analytics-respect-dnt="false"(the 1.22.0 embed treats any value other than"false"as on). Nothing turns privacy on unless you configure it.
For an existing installation (signals migration already run), publish only the
additive 1.22 migration. Like the signals tag, the copy is dated at publish time and
is a no-op where the columns exist (see Upgrade migrations,
which also covers a fixed-name 0001_01_01_000006_... copy published by 0.5.0).
Fresh installations use scarlett-migrations once.
php artisan vendor:publish --tag=scarlett-migrations-reconnects php artisan migrate php artisan queue:restart php artisan scarlett:doctor
Until the migration runs, ingest continues and skips the new columns (the optional
raw log keeps the values); the beacon reconnect columns doctor check warns. The
migration adds, all nullable: on scarlett_views, element_seek_count and
reconnect_count (unsigned integer), reconnect_ms, dvr_ms and pause_ms
(unsigned big integer), media_duration (double) and media_duration_at (dateTime,
precision 3); on scarlett_view_errors, network_state and ready_state (unsigned
integer), online (boolean), source_host (string) and reconnecting (boolean). It
rewrites no existing rows. Column availability is cached per store instance, so
restart workers after migrating.
| Field | Persistence and merge |
|---|---|
elementSeekCount |
element_seek_count, monotonic. Every element seek that was not a player seek's echo, uncoalesced |
reconnectCount |
reconnect_count, monotonic. Outages, not attempts; also carried by reconnecting and recovered |
reconnectDuration |
reconnect_ms, monotonic, an open outage included. After the first frame an outage is also a rebuffer, so it overlaps rebuffer_ms: do not add the two |
dvrTime |
dvr_ms, monotonic, live views only (NULL on VOD): play time behind the live edge after a seek |
pauseDuration |
pause_ms, monotonic: time paused, a pause still open included. Sent on heartbeats and both viewEnds from 1.22 (only the ended viewEnd before), so it was a known field already; it simply had no column |
duration (heartbeat) |
media_duration, seconds, latest by timestamp under media_duration_at. Taken only from heartbeat: recovered and rebufferEnd reuse duration for milliseconds and never touch it. Only a finite value above 0 on a beacon not saying isLive: true is stored, so a load() (which zeroes it) or a live source (0, Infinity sent as null, or a sliding window) never overwrites a known length. Meaningful only where is_live is not true: the guard is per beacon, and a live view's heartbeats before classification carry the initial NaN or a reset 0 (both unusable), because providers classify live before they write the window-length duration. A host that forces isLive: false on a live source, or a provider that reports a finite duration before classifying, can still leave a window length here |
networkState, readyState, online, sourceHost, reconnecting |
Corresponding snake_case columns on scarlett_view_errors; absent values stay null. sourceHost is a host name only |
attempt, delayMs, elapsedMs, longOutage |
Raw log only (reconnecting and recovered events) |
What changes for queries and listeners:
- Errors and reconnects. A fatal error the provider reconnects from arrives with
fatal: true,errorSeverity: 'warning'andreconnecting: true, and the view stays open.scarlett_view_errors.fatalnow records whether the error ended its view:errorSeveritydecides when present (players before 1.20 fall back tofatal), and a reconnecting error is never fatal. The wire value stays in the raw log. Count failures withfatal = trueorseverity = 'fatal', and in listeners usePlaybackErrorReported::isFatal(), not$payload->get('fatal').errorCategorycan now benetwork,mediaorsourcefrom an element MediaError code alone. - New events.
reconnecting(once per outage, plus once at the first long-outage repeat) andrecovered(durationis the outage in ms, unlike the heartbeat'sdurationin seconds) are stored in the raw log and merge theirreconnectCount. They are not errors, write noscarlett_view_errorsrow, and never end a view. seek_countnow counts player seeks plus coalesced element seek bursts (element seeks within 2 s of each other count once);element_seek_counthas every one.- Exit types.
exit_typegainsliveEnded: the element ended on a live view. It is stored as a plain string like the others (completed,abandoned,error,background); queries that readcompletedas "watched to the end" should treat live views andliveEndedseparately. Live views now reportcompletion_rateNULL; it is not set from any other beacon. - Bitrates.
avg_bitrateandmax_bitratestay NULL until the player knows a bitrate (native HLS, MP4 and WHEP often never do). Rows written by earlier players hold 0 for an unknown bitrate; the migration does not rewrite them, because a stored 0 cannot be told apart from a measured one. Exclude 0 as well as NULL when averaging bitrate across player versions. - Unload
viewEnd. It now carries the same fields as the endedviewEnd(counters,rebufferRatio,qoeScore,qoeVersion,completionRate). - Percent watched.
completion_rateis the player's own figure on the finalviewEnd: on 1.22.0, the audited version, a percentage, null on live and otherwise unbounded (an unknown duration falls back to the last known duration and position, or sends 0, and a position past the duration exceeds 100); other versions are unaudited. For unit-safe queries use the canonicalcompletion_ratio(0..1) instead (see Canonical gauges); the legacy column keeps the received wire value. For a figure from the view row at any point, divide position or play time bymedia_duration:current_position / media_duration * 100(where the viewer is) orplay_ms / 1000 / media_duration * 100(how much was played, which exceeds 100 on rewatching). Both are NULL when no heartbeat reported a usable duration (live, or a view shorter than one heartbeat). Keep only views whoseis_liveis not true, NULL included (where(fn ($q) => $q->whereNull('is_live') ->orWhere('is_live', false)); a plainwhereNotdrops NULL), never divide by zero, and do not coalesce it to 0. Rows written before the migration have nomedia_duration. - Live latency summaries exclude readings before the first frame and while behind the edge, and can be absent on a live view.
- Idle views. A view idle for 30 minutes ends with
abandoned; later playback arrives under a newviewId, a new row. Nothing is sent for a view after itsviewEnd.browsercan beInstagram,Facebook,Google App,LinkedInorTikTok. - Custom stores. The
BeaconStorecontract andBeaconPayloadkeep their shape. The fields above moved from$payload->customto the known fields ($payload->get('reconnectCount')); a driver that read them fromcustommust switch. Deduplication hashes are unchanged. A queued job serialized before the upgrade keeps its old classification; read promoted fields with$payload->knownValue('reconnectCount')to get the same value from such a job as from a fresh one, and decide whether an error ended its view with$payload->isFatalError().
Player 1.23 and 1.24
Package 0.7.0 moves the default player.player_version to 1.24.0 (from 1.22.0).
There is no migration and no schema change, and 0.7.0 adds no known field: a host on
0.6.0 keeps its tables and ingest code. Verified against the published npm 1.24.0
packages (@scarlett-player/embed, chapters, share, captions and core) and the
CDN's v1.24.0/embed.umd.cjs, which is byte-identical to the npm build (see
Compatibility).
Upgrade order is the same as for 1.22: update the package, then move the player. Embed
mode follows player.player_version, so the embed page and component load
v1.24.0/embed.js and its addons as soon as the package deploys, unless the host's
published config or SCARLETT_PLAYER_VERSION pins another version. Module mode uses the
host's own npm dependencies: move every @scarlett-player/* package to 1.24.0 together,
and set SCARLETT_PLAYER_VERSION=1.24.0 if you pin it. The embed data-* names the
builder writes are unchanged.
What changes on the wire, and what it means for readers:
- Gauges can be null (1.23.0 and later).
completionRateis sent as null when the duration is unknown (earlier players sent 0), andrebufferRatiois null when its watch and rebuffer inputs are unusable (a watch time of 0 still sends 0). Both stay bounded to 0..100. In the canonical columns a null is a stamped unavailable measurement:completion_ratiois null andcompletion_ratio_atis set, so averages leave it out. The legacycompletion_rateandrebuffer_ratiocolumns skip a null and keep the last value they received, which is why these gauges should be read fromcompletion_ratioandrebuffer_fraction(see Canonical gauges). - Seeks coalesce (1.23.0 and later). Player-requested seeks send at most one
seekingbeacon per second per view; element seeks are unchanged.seek_countstill counts every seek request, so it stays exact, but there are fewer rawseekingrows thanseek_count. A seek still held when the view ends can go out through the unload transport. - More views that never played (1.24.0). Loading a different source after a view has
ended starts a new view at once: a new
viewId,isLivesent as null, and the samevideo_idunless the host calledsetVideo(). A source that fails before it plays reportsviewStart, a fatalerrorandviewEnd(exit_typeerror) with noplayRequestorvideoStart. A view that never played has nostartup_msand aplay_msof 0, and there are more such rows for the same playback. Count plays with a filter such asplay_ms > 0rather than counting rows. - Retried media errors (1.24.0). hls.js
muxerrors are retried, and on a live source the player reconnects from them. Expect more warning-severity errors and morereconnectingandrecoveredbeacons. The wireerrorCodestaysMEDIA_DECODE_ERROR, anddetail.typeis not sent. Decide whether an error ended its view withPlaybackErrorReported::isFatal(), as before.
Diagnostics snapshots (1.23) and the core teardown bound (1.24) are not on the wire, so they change nothing in the stored data.
Player 1.25
Package 0.8.0 moves the default player.player_version to 1.25.0 (from 1.24.0) and
adds one additive migration, the diagnostics upgrade (frame totals and viewer_ref, see
Finding a viewer's views). Verified against the published npm 1.25.0
packages (@scarlett-player/embed, chapters, share, captions and core) and the
CDN's v1.25.0/ files (embed.js, embed.umd.cjs, both addons and hls.1.25.0.js),
which are byte-identical to the npm build. The embed data-* names, the
Chapter/CaptionSource shapes and the share snippet are unchanged from 1.24.
What changes on the wire, and what it means for readers:
-
sentAton every beacon. The built-in transport appends the time it sent the body (epoch ms), stamped per attempt. 0.8.0 knows the field: it is kept in the raw event log but is not a custom dimension, so it no longer lands in the view'scustommap, and it is left out of the deduplication hash, so two deliveries of one beacon with different dispatch times still store once. Comparing it withreceived_atin the raw log tells you whether a viewer's clock was wrong. AsentAtthat is not a number (a host's own dimension) stays a custom dimension and hashes as before. A custom transport sends none. -
Frame totals. Heartbeats and both viewEnds carry
decodedFramesTotalanddroppedFramesTotal, the view's running totals beside the per-intervaldecodedFrames/droppedFrames. 0.8.0 stores them in the monotonic view columnsdecoded_frames_totalanddropped_frames_total(the larger value wins), not incustom; their deduplication hash is the one 0.7.0 computed when it kept them as custom dimensions. A high dropped share with no rebuffering points to the device's decoder rather than the network:SELECT view_id, viewer_ref, dropped_frames_total, decoded_frames_total FROM scarlett_views WHERE decoded_frames_total > 0 AND dropped_frames_total > decoded_frames_total * 0.05 AND COALESCE(rebuffer_count, 0) = 0;
-
Live seek window. A
seekingbeacon on a live view carriesseekableStartandseekableEnd(seconds), kept as custom dimensions and in the raw log. -
Startup and rebuffers read higher, mostly on iOS Safari.
videoStartnow waits for a frame to be shown, and a picture that freezes without the browser reporting buffering counts as a rebuffer, sostartup_ms,rebuffer_countandrebuffer_msrise against 1.24 for the same playback. Compare like with like across the upgrade. -
Load timeouts reconnect. A load that hits
loadTimeoutMshands off to auto-reconnect, even before the stream is known to be live, so expect morereconnectingandrecoveredbeacons and errors withtimedOutset.
Batching stays unsupported: the batch envelope's sentAt changes nothing here.
Upgrade order for an existing host: update the package, publish only the diagnostics
upgrade, migrate, restart queue workers, then move the player. Until the migration runs,
ingest skips the new columns (the raw log keeps the values) and the
beacon diagnostic columns doctor check warns.
php artisan vendor:publish --tag=scarlett-migrations-diagnostics php artisan migrate php artisan queue:restart
Columns, all nullable on scarlett_views: decoded_frames_total and
dropped_frames_total (unsigned big integer), viewer_ref (string, 191) with the index
scarlett_views_viewer_ref_index. Existing rows are not rewritten.
Canonical gauges and upgrading from 0.5
The viewEnd gauges completionRate and rebufferRatio reached the legacy columns
completion_rate and rebuffer_ratio exactly as sent. Scarlett 1.22.0, the audited
version, sends percentages: completion is null on live, otherwise unbounded (an
unknown duration falls back to the last known duration and position, or sends 0; a
position past the duration exceeds 100), and rebuffer exceeds 100 when rebuffering outlasts watch time (0
with no watch time). Older versions are not audited and must not be assumed to match.
Stored alongside values of unknown provenance, an average over either column is
meaningless: 0.5 can be 0.5% or 50%, and a value's magnitude never proves its units.
0.6.0 therefore adds canonical ratio columns and never changes the legacy ones:
| Column | Meaning |
|---|---|
completion_ratio, completion_ratio_at |
The completion as a 0..1 ratio, and the timestamp of the measurement that wrote it |
rebuffer_fraction, rebuffer_fraction_at |
The rebuffer share of watch time as a 0..1 fraction, stamped the same way |
Each gauge is normalized after the host pipeline, from the beacon's own units:
- A beacon may declare its units with
gaugeScale(percentorratio), kept as a custom dimension. A recognized marker wins; a present but unrecognized marker, includinggaugeScale: null, is never second-guessed from the value or the registry. - Without a marker, only the source-audited producer registry decides. Initially
scarlett-player1.22.0 is proven percent from an audit of its analytics source (it sends no marker); other producers and versions stay unknown until audited. Scarlett Player 1.23.0 and later sendgaugeScale: 'percent'on viewEnd (finite percentages bounded to 0..100, null for unavailable measurements), so it is decided by its marker and needs no registry entry; 1.22.0 does neither, so its out-of-range values clamp here and its unknown-duration 0 is stored as 0. The view row's set-onceplayer_versionis not evidence for a later beacon. - Unknown or ambiguous units write null with the measurement's stamp: a processed
unavailable measurement, never a fabricated zero and never a stale average. An older
beacon cannot refill it. Live rows keep
completion_rationull regardless of arrival order;rebuffer_fractionis unaffected by the live rule. - Finite inputs clamp to 0..1 with the out-of-range condition recorded; percent divides by 100 even below one. Legacy columns keep the received values unchanged, and omitting a gauge still leaves both columns alone.
Until the migration runs, ingest continues and skips the canonical columns (the raw
log keeps everything); the beacon gauge columns doctor check warns. Upgrade order:
publish, migrate, restart workers, review the dry-run backfill, then point readers at
the new columns:
php artisan vendor:publish --tag=scarlett-migrations-gauges php artisan migrate php artisan queue:restart php artisan scarlett:doctor php artisan scarlett:views:backfill-gauges # dry run, writes nothing php artisan scarlett:views:backfill-gauges --apply # after reviewing the report
The backfill command fills the canonical columns for historical rows, preserving the
legacy ones, in bounded keyset batches. It is a dry run unless --apply is
passed. For each measurement it seeks that gauge's own retained raw evidence (the
post-pipeline payload matching the legacy stamp and value) and decides the scale only
from the evidence's marker and producer identity, the inputs ingest reads: proven
percent and proven ratio are normalized, tied conflicting evidence stays ambiguous,
and missing raw retention leaves the value unavailable. The raw log keeps an explicit
gaugeScale: null (from 0.6.0 on), so it stays an invalid marker as at ingest; raw
logs written by earlier versions dropped it, and Scarlett 1.22.0 never sends one. The
raw log also lays the server context over the browser's keys, while ingest reads the
marker and producer only from the browser's. The view's server map is merged across
beacons, so ownership of gaugeScale, playerName and playerVersion is decided per
event from server_stamps: evidence from the event that last wrote one of them is
ambiguous, evidence newer than every such writer is the browser's, and evidence older
than a writer is left unprocessed (no stamp, reported as server ownership unknown) so a
later rerun can still recover it. Ambiguous or missing-evidence measurements with a
trustworthy legacy stamp get a canonical null plus that stamp once, so a rerun resumes
without dividing twice; rows whose legacy stamp is missing are reported and left
untouched. Writes are compare-and-swap against the observed legacy value/stamp and the
canonical stamp, so live ingest wins a race and a canonical stamp (from ingest or an
earlier run) is never overwritten. The report summarizes proven percent, proven ratio,
clamped, invalid marker, ambiguous, missing raw evidence, server ownership unknown,
live, already processed, changed concurrently, insufficient provenance and unmeasured
rows; it prints no payload data. Historical rows without raw retention, and producers outside the
registry, remain unavailable by design.
Reader transition. Dashboards and queries must move their averages and filters to
the canonical columns; adding the columns does not repair queries that still read the
legacy ones. Average completion_ratio / rebuffer_fraction where the stamp is not
null (a null value with a stamp is a processed unavailable measurement: count it as
excluded, not zero), keep live views out of completion averages as before, and keep
reading the legacy columns only for the raw wire values. A canonical null without a
stamp means the gauge was never measured (or its scale could not be proven and the
backfill has not run yet).
Player privacy options
player.analytics_anonymous (SCARLETT_ANALYTICS_ANONYMOUS) and
player.analytics_respect_do_not_track (SCARLETT_ANALYTICS_RESPECT_DO_NOT_TRACK)
default to false. Boolean values and environment strings 1/on/yes or
0/off/no are accepted; invalid values fail configuration validation. Enabling
either flag requires a player pin of at least 1.20.0 in both modes; older pins
raise an exception. They apply in module mode and the 1.20.0 Full embed build,
including the package's embed page. They do not turn analytics on by themselves.
Override them per player:
ScarlettPlayer::for($video)->withAnalytics()->analyticsPrivacy( anonymous: true, respectDoNotTrack: true, );
<x-scarlett-player :media="$video" analytics anonymous respect-do-not-track /> <x-scarlett-player :media="$video" mode="embed" analytics anonymous respect-do-not-track />
Use bound booleans (:anonymous="false") for false overrides. Embed mode emits
data-analytics-anonymous and data-analytics-respect-dnt. Requesting privacy on
an older embed or an Audio/Video-only build fails rather than silently ignoring it.
Anonymous mode makes ephemeral per-view browser IDs without browser storage;
it does not disable server IP recording or a host's server context resolver.
DNT/GPC suppression happens in the player before transmission.
Function options belong in JavaScript, not serialized Blade JSON. Module mode
continues forwarding page options including beforeSend and playerInitTime:
window.scarlettPlayerOptions = { analytics: { anonymous: true, respectDoNotTrack: true, beforeSend: (payload) => payload, // return null to discard playerInitTime: Date.now(), }, };
Set these synchronously in the importing entry module, before any await.
The HTML embed has no beforeSend attribute. For direct JavaScript embed
creation use the player's ScarlettPlayer.create({ analytics: { beforeSend } })
API. Keep batch off with this package's endpoint.
Reading the raw log
Order a view's events by occurred_at, seq, id, never by id alone. The id records
insertion by the queue worker, not playback order; keepalive requests can arrive in the
opposite order, even when the player's timestamps share a millisecond.
SELECT * FROM scarlett_beacon_events WHERE view_id = ? ORDER BY occurred_at, seq, id;
seq is the player's per-view beaconSeq (1.19.3+), starting at 1 on each viewStart.
Sampling does not consume numbers. A gap means a beacon never reached the stored log
(for example, lost in transport or dropped by a pipeline), or an event was excluded by
beacons.raw_events_except. With ['heartbeat'] excluded, gaps at heartbeat positions
are expected and do not mean the view totals are missing those heartbeats.
Rows from players before 1.19.3 have seq NULL and order by time only; id is just a
deterministic tie-breaker and cannot recover their true order within one millisecond.
For a finite numeric beaconSeq, storage rounds to the nearest integer, with halves
away from zero, then clamps negative results to zero. The supported SQL range is
0 through 2147483647, portable across SQLite, MySQL and PostgreSQL. A rounded
value above 2147483647 omits seq from the SQL insert instead of binding an oversized
integer or clamping to the maximum. For example, 2147483647.4 stores 2147483647,
while 2147483647.5 leaves SQL seq NULL. The original finite beaconSeq stays in
the raw payload and aggregation continues unchanged.
An absent, null, wrong-typed or out-of-range value leaves seq NULL on the package
schema. Thus NULL does not necessarily mean an old player: it can also mean an unusable
order key, which cannot break timestamp ties. Non-finite numbers likewise must not be
bound to SQL seq; this does not add support for non-finite JSON values or change the
existing JSON encoding limitation. seekSource stays in the raw payload on seeking,
not in a new view column. Neither field changes the view-row aggregation rules above.
Playlists
From player 1.19 every playlist track is its own view: a track change (and setVideo()
with another id, or a replay after ended) closes the current view with exitType
abandoned and starts a new viewId, so each track gets its own scarlett_views row.
From 1.19.1 a track's beacons carry that track's own videoId (or the plugin's configured
one), never the playlist's internal item id, so for a view to link to your model each
track's videoId must be a media id your ResolvesMedia maps. The package does not emit
the embed's data-playlist attribute.
Server-side context
Everything in a beacon comes from the browser, and anything the package does not know
lands in custom, so a browser can send a custom dimension called user_id with any
value. custom is what the browser sent; never trust it for who the viewer is. For the
fields only your server knows (the signed-in user, the tenant), name a resolver in
beacons.context:
use Hei\ScarlettPlayer\Contracts\ResolvesBeaconContext; use Hei\ScarlettPlayer\Data\BeaconPayload; use Illuminate\Http\Request; final class BeaconContext implements ResolvesBeaconContext { public function resolve(Request $request, BeaconPayload $payload): array { return ['user_id' => $request->user('web')?->getAuthIdentifier()]; } }
// config/scarlett-player.php 'beacons' => [ 'context' => App\Analytics\BeaconContext::class, ],
Three rules:
- It runs at request time, in the beacon route, after validation and before the
beacon is queued, because the queued job has no request. It runs for every beacon,
heartbeats included, so keep it cheap. The route has no
authmiddleware (guests beacon too), so it must not throw for a guest. It is not wrapped: an exception is reported by your handler, the beacon is answered500and lost, and so is every beacon while it keeps throwing.scarlett:beacon:testposts through the route and runs it. - Every key it returns is server-owned on that beacon. The key is removed from the
browser's custom dimensions, whatever the browser sent, and the value is stored under
scarlett_views.server. A name the player also uses (duration) is allowed, but it lands only inserverand the raw log, never in that name's own column. The six identity keys (event,timestamp,viewId,sessionId,viewerId,videoId) are refused withInvalidBeaconContextException. - Null strips. A key returned as
nullstores nothing but still removes the browser's copy, so['user_id' => $request->user()?->id]means a guest's view has nouser_idfrom anywhere. A value stored by an earlier beacon of the view is never deleted by a later null.
scarlett_views.server is a JSON map beside custom, merged the same way: key by key,
each key keeping the value from the newest beacon that sent it (server_stamps). Query it
with Laravel's JSON paths; bind the value with the type your resolver returned:
DB::table('scarlett_views')->where('server->tenant_id', 7)->get();
The raw log, BeaconReceived, ViewStarted and the fake's ledger see the server value
under its name, after the browser's keys. The raw log's event_key hashes only what the
browser sent, so the server context never makes two deliveries of one beacon look like
two beacons. For an indexed column (a tenant_id you filter on), add it in your own
migration and fill it from ViewStarted.
A ProcessesBeacon step can add server context too, with $payload->withServer(), for
anything that does not need the request.
Migrated with 0.1? Add two nullable json columns, server and server_stamps, to
scarlett_views before setting beacons.context (the beacon context doctor check fails
until you do; without a resolver nothing touches them).
Recipe: tenant from the request
No middleware needed: the host, or a header your edge sets, is on every request.
public function resolve(Request $request, BeaconPayload $payload): array { return ['tenant_id' => Tenant::idForHost($request->getHost())]; }
Recipe: the signed-in user, from the session
The beacon route runs in the api group, which has no session. Append the cookie and
session middleware to the beacon group, then read the web guard:
'routes' => [ 'middleware' => [ 'beacons' => [ 'api', 'throttle:scarlett-beacons', \Illuminate\Cookie\Middleware\EncryptCookies::class, \Illuminate\Session\Middleware\StartSession::class, ], ], ],
return ['user_id' => $request->user('web')?->getAuthIdentifier()];
Never add auth (guests beacon too, and a 401 loses the view) or the CSRF middleware
(the unload sendBeacon cannot send a token).
Same origin only. The plugin sends every beacon except the unload viewEnd with
fetch, and sets no credentials option, so the browser attaches cookies only when the
ingest is on the page's own origin. With the page and the ingest on one origin, every
beacon carries the session. With the ingest on another origin (another subdomain
included, whatever the cookie's domain), in-session beacons arrive as guests; only the
unload sendBeacon, which always sends credentials, carries the session cookie, so a
view gets its user_id from that one beacon, or never when it is lost. For an ingest on
another origin, use the token recipe below.
Recipe: Sanctum SPA
'beacons' => [ 'api', \Laravel\Sanctum\Http\Middleware\EnsureFrontendRequestsAreStateful::class, 'throttle:scarlett-beacons', ],
return ['user_id' => $request->user('sanctum')?->getAuthIdentifier()];
The session rides the cookie, so this too is same origin only, for the reason above.
EnsureFrontendRequestsAreStateful adds Laravel's CSRF check to every request from a
stateful domain, and neither beacon transport sends a token, so beacons from your SPA are
answered 419 (Laravel 13 lets a same-origin request through on Sec-Fetch-Site; a
cross-origin ingest, and Laravel 12, do not). Exempt the beacon path, in
bootstrap/app.php:
->withMiddleware(function (Middleware $middleware): void { $middleware->validateCsrfTokens(except: ['api/scarlett/beacons']); })
Recipe: a token, for an ingest on another origin
Keep the default beacon middleware and read the sanctum guard, which accepts a Bearer
token with no session or stateful middleware:
return ['user_id' => $request->user('sanctum')?->getAuthIdentifier()];
Give the page a token and pass it through the plugin's headers option (merged over
Content-Type and X-API-Key on every fetch beacon), and add Authorization to
allowed_headers in config/cors.php:
window.scarlettPlayerOptions = { analytics: { headers: { Authorization: `Bearer ${token}` } }, };
The token is readable by any script on the page, so issue one that is short-lived and
limited to this purpose ($user->createToken('beacons', ['beacons'], now()->addHours(2))).
The unload viewEnd goes out through sendBeacon, which carries no headers, so it is
always a guest; a guest beacon's null never deletes the user_id the view already has.
Finding a viewer's views
Support hears "my stream broke last night", not a view id. Tag every view with your own
id for the viewer in the server context (the recipes above), and 0.8.0 copies it into
the indexed scarlett_views.viewer_ref column:
// config/scarlett-player.php 'beacons' => [ 'context' => App\Analytics\ScarlettBeaconContext::class, // The context key copied into viewer_ref; null turns it off. 'viewer_ref_key' => env('SCARLETT_BEACON_VIEWER_REF_KEY', 'user_id'), ],
A published config from before 0.8.0 has no viewer_ref_key (Laravel merges the
beacons array one level deep), and a missing key means the default user_id; set it
to null explicitly to turn it off.
viewer_ref is set once per view, from the first beacon whose server context carries a
non-empty string or an integer under that key (trimmed, cut to 191 characters). A guest
beacon leaves it alone, so a view that starts signed out picks it up from the first
signed-in beacon. A browser custom dimension of the same name is never used. Return your
own id, never a token or an email you would not want in an analytics table. It needs the
diagnostics upgrade (--tag=scarlett-migrations-diagnostics); without it the value stays
in the server column only.
Then look a customer up:
php artisan scarlett:views:inspect --ref=4182 --since=24h php artisan scarlett:views:inspect 1791915129315-900kizszm # one view by id php artisan scarlett:views:inspect --viewer=<viewerId> --video=<videoId> --json
Each matching view (newest first, --limit, default 5) prints its fields (player, device,
startup, rebuffers, reconnects, dropped frames, gauges, the server and custom maps), its
errors (severity, category, code, HTTP status, source host, message) and its raw beacon
timeline (--events, the newest 100 by default). For player 1.25 and later the timeline
shows each beacon's stored time minus its sentAt, with the median per view: network time,
the wait on the beacons queue (retries included) and how far the viewer's clock is off.
received_at is stamped when the queued job runs, not at the HTTP request, so a backed-up
queue shows up here as well. The timeline needs beacons.store_raw_events; the
command names any events beacons.raw_events_except leaves out (leaving out heartbeat
removes most of a view's timeline). --since takes 30m, 24h, 7d or a date and
filters on when the view was last updated. Read only, and for the Eloquent store only.
Your own store
beacons.store takes eloquent, null (accept and discard), or the class name of your
own Hei\ScarlettPlayer\Contracts\BeaconStore:
interface BeaconStore { public function record(BeaconPayload $payload): void; }
A driver inherits the route, the key, the validation and BeaconPayload, and must honour
the same contract: idempotent, order-independent, merged per field with these classes,
and transitions (not receipts) for the events. Contracts\BeaconStore spells it out.
Privacy and retention
The player's viewerId is a persistent anonymous id when you do not supply one:
persistent, therefore not automatically anonymous.
beacons.store_ip(defaultfalse): no address is stored, and the migration creates noip_addresscolumn. Turn it on before migrating; turned on later, addresses are skipped andscarlett:doctorfails thebeacon ip columncheck.beacons.anonymize_ip(defaulttrue): the address is truncated (IPv4 last octet, IPv6 last 64 bits) before the job is queued, so a full address never reaches the queue.beacons.pipeline: classes implementingContracts\ProcessesBeacon, run in order before storage. A step can redact ($payload->withCustom(),->withIp()), replace the server context (->withServer(), see Server-side context) or returnnullto drop the beacon. What a step returns is what the raw log, the store andBeaconReceivedsee.
final class DropEmail implements ProcessesBeacon { public function handle(BeaconPayload $payload, Closure $next): ?BeaconPayload { return $next($payload->withCustom(Arr::except($payload->custom, ['email']))); } }
beacons.raw_events_except(default[]): event names never written toscarlett_beacon_events.['heartbeat']cuts the raw log to about a tenth and loses only the per-heartbeat trail: every beacon still merges into itsscarlett_viewsrow, and an excludederrorstill reachesscarlett_view_errors.beacons.retention: days per table,['events' => 90, 'views' => 365];nullkeeps a table forever.scarlett:views:pruneapplies it and runs daily from your scheduler whilebeacons.schedule_pruneis on.scarlett_view_errorsfollowsretention.views. The daily run needs Laravel's scheduler running (php artisan schedule:runfrom cron every minute, orschedule:work); withschedule_pruneoff, run the command yourself.
Events
| Event | Fires |
|---|---|
BeaconReceived |
once per delivery, after the pipeline, with the payload the steps let through; a dropped beacon fires nothing. Not deduplicated: a redelivered job fires it again |
ViewStarted |
once, when a view is first stored (by whichever beacon arrives first) |
ViewEnded |
once, when a view first gains an end |
PlaybackErrorReported |
once per error beacon actually stored, warnings and reconnecting errors included. Use $event->isFatal() (severity fatal, not reconnecting) to decide playback failed, never the payload's fatal; $event->isReconnecting() marks a player 1.22 reconnect |
The last three come from EloquentBeaconStore; a host store fires its own.
Commands
php artisan scarlett:beacon:test [--url=https://app.example.com/api/scarlett/beacons] php artisan scarlett:views:prune [--days=30] # --days overrides retention.events php artisan scarlett:views:inspect [view] [--ref=] [--viewer=] [--video=] [--since=] [--limit=5] [--events=100] [--json]
scarlett:views:inspect prints a viewer's views with their errors and beacon timeline,
for support (see Finding a viewer's views).
scarlett:beacon:test posts a synthetic view (viewStart, heartbeat, reconnecting,
recovered, viewEnd, in the player 1.25 shape with null bitrates, sentAt on every beacon, frame totals and gaugeScale: 'percent' on both viewEnds) through the header
path (a VOD view ending abandoned) and again through the query-string path (a live
view ending liveEnded), and fails unless every one is answered 204. The beacons are real (queued and stored under a view id starting
scarlett-beacon-test-). It proves the route, the key and the body; it does not prove
browser CORS.
scarlett:doctor adds, for beacons: the key, the store binding, the route, the CORS
recipe, the beacon queue (warns when it shares a queue with clips), the IP column,
server-side context, the raw-log seq column, the signals, player 1.22
(beacon reconnect columns), canonical gauges (beacon gauge columns) and diagnostics
(beacon diagnostic columns) upgrade columns, and any upgrade migration copy, in any
registered migration path, that sorts before the create migrations it needs
(beacon upgrade migration order, see Upgrade migrations).
Testing against beacons
ScarlettPlayer::fake() swaps in a Testing\FakeBeaconStore that also fills the fake's
beacon ledger, so beacons posted through the route (with a sync queue) can be asserted:
$fake = ScarlettPlayer::fake(); $this->postJson('/api/scarlett/beacons?api_key='.config('scarlett-player.beacons.key'), $body) ->assertNoContent(); $fake->assertBeaconRecorded(fn (array $beacon) => $beacon['event'] === 'viewEnd');
Bind FakeBeaconStore yourself for assertRecorded(), assertRecordedCount() and
assertNothingRecorded() on BeaconPayload objects. The player's wire fixtures for
1.19.1 are under tests/Fixtures/wire/1.19.1/, captured from the real transports by the
player repo's harness: every beacon event per transport, the three viewEnd variants, a
live session and a clip create and retry. They stay as the captured replay for players
still on 1.19.x. Additive 1.19.3 request envelopes under tests/Fixtures/wire/1.19.3/
cover viewStart with beaconSeq and both seeking sources, and
tests/Fixtures/wire/1.22.0-derived/ covers the 1.22.0 additions (reconnect outage,
reconnecting-marked error, liveEnded, full unload viewEnd). Both are derived
(recapture owed) from the player source, not captured transports.
Clips
A clip is a free, publicly watchable video, even when its source is paid. So a clip is
always a separately rendered, separately stored asset with its own URL, and
clips.max_duration is a rights control: it decides how much of a paid event a viewer
may republish, and it is checked on the rendered file as well as the request.
Add the relation to your media model with the HasScarlettClips trait (see Media
contract). Migrations are publish-only:
php artisan vendor:publish --tag=scarlett-migrations php artisan migrate
The scarlett_clips migration stores clippable_id as a string, so integer, uuid and
ulid media keys all fit. user_id is an integer: if your users use uuid or ulid keys,
edit that column after publishing.
Endpoints
| Method | Route name | Path | What it does |
|---|---|---|---|
POST |
scarlett.clips.store |
{prefix}/clips |
Validates, stores the clip as pending, queues the render, answers 202 |
GET |
scarlett.clips.show |
{prefix}/clips/{uuid} |
Status, with playbackUrl or previewUrl when the viewer may have one |
GET |
scarlett.clips.play |
{prefix}/clips/{uuid}/play |
Redirects to the asset once the clip is ready and public, else 404 |
GET |
scarlett.clips.preview |
{prefix}/clips/{uuid}/preview |
Temporary signed; re-checks the preview ability, then redirects |
Point the clips plugin's endpoint.url at route('scarlett.clips.store'). The plugin
posts its ClipRange verbatim (camelCase) and gives up after 15 seconds, so the create
endpoint never renders inline. A retry with the same clientRequestId returns the first
clip, and queues its render again only if that dispatch never happened. Both the first
request and a retry answer:
{
"uuid": "9b2c...", "status": "pending", "visibility": "pending_review",
"mediaId": "abc123", "title": "The knockout",
"startTime": 120.5, "endTime": 150.5, "duration": 30,
"failureReason": null,
"statusUrl": "https://app.test/api/scarlett/clips/9b2c...",
"playbackUrl": null, "previewUrl": null
}
Validation messages are written for viewers, because the plugin shows the message of a
422 in its overlay: "Clips can be at most 60 seconds.", "Clips from live streams aren't
available yet.". The bounds come from clips.min_duration and clips.max_duration, and
the duration is re-derived from endTime - startTime; the client's duration is
ignored. Live clips are refused in v1: isLive must be false, every live field
(seekableStart, seekableEnd, startDate, endDate) must be null, and the resolved
media must not be live. An unknown mediaId is a 404.
Every clip route answers JSON, including auth and validation failures, even though the
plugin sends no Accept header.
Authentication recipes
routes.middleware.clips guards creating a clip. The status, play and preview routes
run the same list without its auth / auth:<guard> entries and the
throttle:scarlett-clips entry, and authorize themselves: status through
ClipPolicy::view (guests only when clips.allow_guests is on, and then only for clips
that are ready and public; the viewer is resolved through the guards your auth: entry
names), play publicly once the clip is
ready and public, and preview through its signature plus ClipPolicy::preview. Keep
this in mind when you read the configured list: it is not what the read routes enforce.
Default: same-origin Blade or Inertia app. Nothing to change:
'clips' => ['web', 'auth', 'throttle:scarlett-clips'],
The web group starts the session and checks CSRF. The plugin's default
credentials: 'same-origin' sends the session cookie, and the initialiser's headers()
reads X-CSRF-TOKEN from the page's csrf-token meta tag on every request. No session
is a JSON 401; a missing or wrong token is a 419.
Sanctum SPA (another origin, cookie auth):
'clips' => [ 'api', \Laravel\Sanctum\Http\Middleware\EnsureFrontendRequestsAreStateful::class, 'auth:sanctum', 'throttle:scarlett-clips', ],
Add the SPA's host to sanctum.stateful and set supports_credentials in
config/cors.php. EnsureFrontendRequestsAreStateful turns on CSRF protection, and a page
on another origin has no csrf-token meta tag, so the SPA does the Sanctum handshake
instead: request GET /sanctum/csrf-cookie once (with credentials), then send the
XSRF-TOKEN cookie's value back as the X-XSRF-TOKEN header on every clip request.
Without that header the create request is a 419. Sending cookies and a CSRF header from
the browser does nothing unless the server side is configured this way.
Pass the plugin options through the initialiser by setting window.scarlettPlayerOptions
in your entry point; the Blade component's players pick them up. Set it synchronously,
before any await: a top-level await pauses your entry module, and the players can
start during the pause without the options. Do the handshake inside the headers function
instead, which the plugin resolves on every request:
function xsrfToken() { const match = document.cookie.match(/(?:^|;\s*)XSRF-TOKEN=([^;]*)/); return match ? decodeURIComponent(match[1]) : ''; } // Started now, awaited per request: never awaited at the top level. const csrfReady = fetch('https://app.example.com/sanctum/csrf-cookie', { credentials: 'include' }); window.scarlettPlayerOptions = { clips: { endpoint: { credentials: 'include', headers: async () => { await csrfReady; return { 'X-XSRF-TOKEN': xsrfToken() }; }, }, }, };
If the SPA page also renders a csrf-token meta tag, Laravel reads that stale
X-CSRF-TOKEN before your X-XSRF-TOKEN and answers 419; return
'X-CSRF-TOKEN': null from the headers function to drop it (see
JS initialiser (module mode)).
Sanctum token:
'clips' => ['api', 'auth:sanctum'],
and send the token through the same options; no CSRF is involved:
window.scarlettPlayerOptions = { clips: { endpoint: { headers: () => ({ Authorization: `Bearer ${token}` }) } }, };
The status route authenticates the same token through the sanctum guard.
All three are tested in tests/Feature/Clips/MiddlewareRecipesTest.php.
Policy: protected media fails closed
The package registers Hei\ScarlettPlayer\Policies\ClipPolicy for the Clip model
unless you register your own. Its create lets signed-in viewers clip unprotected media
only, and denies every protected source (MediaSource::$isProtected). Nobody moderates
by default.
Turning paid media into public files must be your explicit decision, and writing a
create() is that decision. Extend the default and override what you need:
use Hei\ScarlettPlayer\Data\MediaSource; use Hei\ScarlettPlayer\Models\Clip; use Hei\ScarlettPlayer\Policies\ClipPolicy; use Illuminate\Contracts\Auth\Authenticatable; class AppClipPolicy extends ClipPolicy { public function create(?Authenticatable $user, MediaSource $media): bool { return $user !== null && (! $media->isProtected || $user->can('clip-paid-events')); } public function moderate(?Authenticatable $user, ?Clip $clip = null): bool { return $user?->is_moderator === true; } } // In a service provider's boot(): Gate::policy(Clip::class, AppClipPolicy::class);
A registered policy that inherits the default create() still denies protected media.
ClipPolicy::hostPolicyPublished($gate) answers whether you have written one, and the
player config builder's withClips() asks the same question.
The abilities are create (with the MediaSource), view, preview and moderate.
Visibility, moderation and delivery
New clips start at clips.visibility (pending_review). Moderate on the model:
$clip->approve($moderator); // visibility public, fires ClipApproved $clip->reject($moderator); // visibility hidden, fires ClipRejected
There is no admin UI. Both are conditional on the clip's current state in the database,
so a moderator acting on a stale model never undoes a render or a rejection: approving a
clip that was rejected or failed meanwhile throws ClipStateException. Rejecting a clip
that has not rendered yet stops its render; one rejected mid-render has its file deleted
by the job. scarlett:clips:reconcile deletes a rejected clip's asset after
clips.delete_rejected_after days (null keeps it), after which the clip can no longer be
approved.
Assets are written private, always, at {clips.path}/{uuid}.mp4 on clips.disk.
Clips\ClipUrlIssuer is the only class that produces a clip URL:
ready+public: a playback URL. Withclips.public_delivery = signed-redirect(default) it is the/playroute, which answers a 302 to a temporary object URL valid forclips.play_ttlseconds, so the object stays private and you can front/playwith a CDN. Withdisk-public,approve()makes the object public (or, for a clip approved before it rendered, the render does) and the URL is the object URL;reject()makes it private again. A CDN in front of adisk-publicbucket may keep serving its cached copy after a reject until that copy expires; usesigned-redirectif a reject must take effect at once. Moderation and the object's visibility change together under a per-clip cache lock (clips.lock_waitseconds of waiting), so use a cache store with locks (redis, database, file); a moderation that cannot get the lock throwsLockTimeoutExceptionand can be retried. A visibility write the disk refuses throwsClipStorageException, including afalsefrom a disk configuredthrow => false:reject()then leaves the row untouched, andscarlett:clips:reconcilereports the clip and exits with a failure code until a retry lands (a disk that throws counts the same; a clip locked by a moderation in flight does not). A clip approved before it rendered still announcesClipReadywhen its public write is refused (the object stays private), but not when a reject landed mid-render and the write hiding the object is refused.readyand not public: no playback URL. The status response carries apreviewUrl(a temporary signed route, valid forclips.preview_ttlseconds) for viewers who passpreview: the submitter while the clip waits for review, or a moderator.- anything else: nothing.
Signed-redirect delivery and previews need a disk that issues temporary URLs (S3 and
compatible disks do); scarlett:doctor checks it.
Rendering
Renders run on the queue in Jobs\RenderClip, which carries only the clip id. The
local-ffmpeg generator ships, so one box with ffmpeg and ffprobe works. Add your
own with app(ClipGeneratorManager::class)->extend('fleet', fn ($app, array $config) => ...)
and a clips.generators.<name> entry with 'driver' => 'fleet'.
The local-ffmpeg generator reads three keys under clips.generators.local-ffmpeg:
| Key | Default | What it is |
|---|---|---|
binary |
FFMPEG_BINARY, ffmpeg |
The ffmpeg executable, a name on PATH or an absolute path |
ffprobe |
FFPROBE_BINARY, ffprobe |
The ffprobe executable; ClipVerifier uses it for every driver |
timeout |
300 |
Render budget in seconds; RenderClip::$timeout is this plus 30 |
clips.accuracy chooses how an unprotected source is cut:
keyframe(default): seek on the input and stream copy. Fast, but the copy can start up to one GOP before the in point.exact: decode, seek on the output, re-encode. Frame accurate, slower.
Protected sources always render exact, whatever the config says.
Every render, from any driver, passes Clips\ClipVerifier before it can become ready.
It reads the video packet timestamps with ffprobe and refuses a file whose packets start
more than clips.duration_tolerance before the container start, or whose packet span
(not the container's nominal duration, which hides pre-roll) exceeds the requested
interval plus the tolerance, or, for a protected source, clips.max_duration plus the
tolerance. A refused file is deleted, the clip is marked failed with
render_exceeds_bounds, and ClipFailed fires; nothing reaches the clip disk.
A keyframe miss falls back to exact. When a keyframe render of an unprotected source
fails verification (common on sources with a GOP longer than the tolerance), the job
re-renders it exact once, in the same attempt, and logs the miss at info level
("Scarlett clip keyframe render exceeded bounds; re-rendering exact."). That costs a
second render; if your sources have long GOPs, set accuracy to exact.
Workers and recovery
Run clip workers on their own queue, with a timeout that covers the render:
php artisan queue:work --queue=scarlett-clips --timeout=330
RenderClip::$timeout is clips.generators.<driver>.timeout + 30 (330 by default).
The worker's --timeout must be at least that, and the connection's retry_after (or
Horizon's timeout) must be larger, or Laravel re-delivers a job that is still
rendering. The job is ShouldBeUniqueUntilProcessing per clip, so however long the
queue backlog, a retry or the reconciler never queues a second render; once a worker
starts it, a second delivery exits unless the first render is older than the job timeout
plus 60 seconds (its worker died).
A render is idempotent: a job that finds its clip ready, failed or rejected exits,
every transition is a conditional update, and a retry that finds the asset already
written re-verifies it and finishes without rendering again. Attempts are counted on the
clip; after clips.max_attempts it is failed.
scarlett:clips:reconcile runs every minute through the scheduler
(clips.schedule_reconcile, on by default). It queues pending clips whose dispatch
failed or went stale (clips.redispatch_after), retries renders stuck in processing
past the job timeout, and deletes rejected assets past their retention. Under
disk-public it also sets each clip's object visibility to what its row wants, which
heals a storage write that failed between a moderation and its object. That pass looks
only at clips changed in the last day, because every object write is a billable storage
call; if you suspect older drift, run php artisan scarlett:clips:reconcile --resync-all
once. Keep the scheduler running.
scarlett:doctor checks that ffmpeg and ffprobe run and that the clip disk keeps a
private write private and issues temporary URLs.
Events
ClipRequested (a new clip, not a retry), ClipProcessing, ClipReady,
ClipFailed (with a reason: render_exceeds_bounds, render_error,
attempts_exhausted, media_not_found, live_source), ClipApproved and
ClipRejected (with the moderator as by). Each carries the clip, never a URL; ask
ClipUrlIssuer for one.
Testing
ScarlettPlayer::fake() records every clip the endpoint accepts:
$fake = ScarlettPlayer::fake()->withMedia($source); $this->actingAs($user)->postJson(route('scarlett.clips.store'), $range)->assertStatus(202); $fake->assertClipRequested(fn (array $clip) => $clip['mediaId'] === 'abc123');
Each record has uuid, mediaId, clientRequestId, startTime, endTime, duration,
title and userId.
Player and embed
Two modes, one config
createPlayer() takes plugin objects with init() and destroy(), which PHP cannot produce,
so the package emits a host config (a versioned JSON schema, scarlettConfigVersion: 1)
and pairs it with one of two front ends:
module(default): the host bundles@scarlett-player/*and the package's JS initialiser, which turns the host config into plugins.embed:data-*attributes read by the pinned@scarlett-player/embedbundle fromplayer.cdn_url. Nothing to bundle, fewer features.
Pick one globally with player.mode, or per call with ->mode() or the component's mode
attribute. What each mode can carry:
| Feature | module (host bundles @scarlett-player/* + the initialiser) |
embed (data-* attributes + the @scarlett-player/embed bundle) |
|---|---|---|
| src, poster, autoplay, muted, loop, start time | yes | yes |
| brand colour / brand text colour | yes | yes; data-brand-color, data-brand-text-color |
| analytics (beaconUrl, videoId, apiKey) | yes; plus headers() in the initialiser |
yes; data-analytics-*, no extra headers |
| analytics anonymous IDs and DNT/GPC | yes, from player 1.20.0; anonymous, respectDoNotTrack; beforeSend through JavaScript options |
yes, from player 1.20.0; data-analytics-anonymous, data-analytics-respect-dnt; Full build only |
| analytics live flag (isLive from the MediaSource) | yes; the initialiser passes it to the analytics plugin, so viewStart is right before the playlist loads | no; the embed has no attribute for it; from player 1.18 viewStart carries null before the playlist loads, which the ingest treats as absent, so the view is marked live by a later beacon: late, never wrong |
analytics heartbeat interval (player.heartbeat_interval) |
yes | no; no data-analytics-heartbeat-interval attribute; the player default applies |
| share URL + embed base URL | yes | yes; data-share-url, data-embed-base-url |
| clips (endpoint, CSRF header) | yes | yes, from player 1.17.0; data-clips-endpoint, data-clips-csrf="meta", data-clips-media-id, data-clips-min-duration, data-clips-max-duration, with the embed.addon.clips addon; Full and Video builds only; the host page needs its csrf-token meta tag |
| chapters | yes | yes, from player 1.17.0; data-chapters (JSON or a WebVTT URL), with the embed.addon.chapters addon; Full and Video builds only |
| captions | yes | yes, from player 1.17.0; data-captions, no addon; Full and Video builds only |
This table is generated from Hei\ScarlettPlayer\Player\FeatureMatrix, the same table the
builder enforces (FeatureMatrix::toMarkdown(); a test fails if the two differ). A cell
reading "from player X" is checked against player.player_version: below it, asking for that
feature in embed mode throws UnsupportedInEmbedMode, which names the module-mode
alternative. On the default 1.25.0 (from 1.17.0), embed mode carries clips, chapters and captions; chapters
and clips need the embed's addon files, which the component loads for you.
Config builder
use Hei\ScarlettPlayer\Facades\ScarlettPlayer; $config = ScarlettPlayer::for($video) // ScarlettMedia model, media id, or MediaSource ->mode('module') ->autoplay()->muted() ->brandColor($tenant->brand_color) ->withAnalytics() // beaconUrl, videoId and apiKey from the package route and config ->heartbeatInterval(5) // seconds; overrides player.heartbeat_interval (module mode only) ->withClips() // endpoint from the package route; csrf: true ->withChapters($video->chapters) // [['time' => 0, 'label' => 'Intro', 'endTime' => 95], ...] or a WebVTT URL ->withCaptions($video->captionTracks) // [['language' => 'en', 'label' => 'English', 'src' => '...'], ...] ->withShare(route('videos.show', $video)) // share button; the embed snippet uses embedUrl() ->toArray(); // the host config, version 1
Chapters use the player's own field names: time and label (required), endTime,
subtitle and thumbnail (optional; end is accepted as an alias for endTime). A chapter
with an endTime stops there, so sparse chapters leave gaps unpainted; a chapter without one
runs until the next chapter starts. Caption tracks take language, label and src, plus
optional kind (subtitles or captions) and default.
beaconUrl, the clips endpoint and mediaId always come from the package's own routes and
your MediaSource. withAnalytics() also passes MediaSource::$isLive to the analytics plugin,
because the player's own viewStart reports isLive: false until the playlist has loaded;
the embed bundle has no attribute for it (see the matrix). withAnalytics() and withClips() throw if their routes are switched off.
The heartbeat interval is player.heartbeat_interval (SCARLETT_HEARTBEAT_INTERVAL), in
seconds; heartbeatInterval() overrides it for one player, and null goes back to the config
value. The analytics block then carries heartbeatInterval in milliseconds, which is what the
plugin takes (2.5 becomes 2500). Unset, nothing is emitted and the player's own 10 s
applies. The value must be from 0.001 to 2147483.647 seconds (the longest delay browsers honour
in setInterval(); a longer one fires almost at once); anything else throws
InvalidPlayerConfigException. The
embed bundle has no attribute for it: in embed mode a configured interval is left out and the
page still renders, while calling heartbeatInterval() throws UnsupportedInEmbedMode.
withClips() also refuses protected media until you register your own policy for
Hei\ScarlettPlayer\Models\Clip, and that policy must declare its own create(). The
package's default policy does not count, and neither does a subclass that inherits its
create().
JS initialiser (module mode)
php artisan vendor:publish --tag=scarlett-js
npm install @scarlett-player/core @scarlett-player/hls @scarlett-player/native @scarlett-player/ui \
@scarlett-player/analytics @scarlett-player/clips @scarlett-player/chapters @scarlett-player/captions \
@scarlett-player/share
Import it once from your Vite entry point:
import './vendor/scarlett-player/init.js';
It exposes initScarlettPlayer(container, config, options), initAll(root, options) and
buildPlugins(config, options), and sets window.ScarlettPlayerHost. Every import in it is
static, so every package listed above must be installed.
Importing it starts nothing. The Blade component's inline script starts its player,
through initAll(document, window.scarlettPlayerOptions) if the initialiser has loaded, or
by leaving a flag the initialiser picks up once your bundle has finished running. So set
page-wide options in your entry point, after the import, and every component player gets
them:
import './vendor/scarlett-player/init.js'; window.scarlettPlayerOptions = { // Sanctum SPA: send the session cookie cross-origin. clips: { endpoint: { credentials: 'include' } }, // Token recipe instead: clips: { endpoint: { headers: () => ({ Authorization: `Bearer ${token()}` }) } }, // Extra beacon headers, resolved per beacon (in-session beacons only; see Beacons). analytics: { headers: () => ({ 'X-Tenant': tenantId }) }, };
Set it synchronously, before any await in your entry module: a top-level await yields,
and the players can then start while the options are still undefined.
options.clips.endpoint and options.analytics are merged into the plugin configs. With
csrf: true (what withClips() emits), X-CSRF-TOKEN from <meta name="csrf-token"> is
sent on every clip request, and your endpoint headers are added to it, never swapped for
it. A header you set to null is removed, X-CSRF-TOKEN included. On a Sanctum SPA page
that matters: Laravel checks X-CSRF-TOKEN before X-XSRF-TOKEN, so a stale
csrf-token meta tag answers 419 however fresh the XSRF cookie is. Drop the meta tag, or
send { 'X-CSRF-TOKEN': null }. The clips endpoint is emitted as a path, so the session cookie goes with it from any host
that serves the page. The full server-side recipes are under Clips.
For one player that needs its own options, render it with manual
(<x-scarlett-player :media="$video" manual player-id="hero" />): the container carries
data-scarlett-manual, no start call is emitted, and you call
initScarlettPlayer('#hero', config, options) yourself, with the config from
document.getElementById('hero-config').
Blade component
<x-scarlett-player :media="$video" autoplay muted class="aspect-video" /> <x-scarlett-player :media="$video" mode="embed" brand-color="#e50914" /> <x-scarlett-player :media="$video" analytics clips :chapters="$video->chapters" share-url="{{ url()->current() }}" />
Attributes: media (required), mode, autoplay, muted, loop, controls, start-time,
poster, title, brand-color, brand-text-color, analytics, heartbeat-interval
(seconds, module mode), clips, chapters, captions, share-url, share-embed, player-id, manual and nonce (for a CSP). Any other
attribute, such as class, goes on the container.
- Module mode renders the container, the host config in a
<script type="application/json">and a one-line call into the initialiser. - Embed mode renders the container with the embed bundle's documented
data-*attributes (no invented names) and a<script>for the bundle. Its URL isplayer.embed_bundle, a template defaulting to{cdn_url}/v{player_version}/embed.js, the player CDN's layout: setSCARLETT_CDN_URLto the CDN origin plus its package prefix, with no version (https://assets.thestreamplatform.com/scarlett-player), and the pinned version fills in the rest.embed.jsis an ES module, so it is loaded withtype="module"; a template ending in.cjs(the UMD build) is loaded as a classic script. For a floating version setSCARLETT_EMBED_BUNDLE='{cdn_url}/latest/embed.js', once the CDN serves/latest/(it serves versioned directories only today). - Embed builds. Clips, chapters and captions need the Full (
embed.js) or Video (embed.video.*) build. Withplayer.embed_bundleon the Audio build (embed.audio.*), asking for them throwsUnsupportedInEmbedMode, naming the build. - Embed addons. With
:chaptersorclips, the component also loadsembed.addon.chapters.jsandembed.addon.clips.jsafter the bundle. They come from the bundle's own directory, in its flavour (.umd.cjsbeside a.cjsbundle), and each is loaded once per page.embedAddonUrls()on the builder lists them if you write the tags yourself. Keep the addons in the same version directory as the bundle: an addon refuses an embed of another version. Captions need no addon. - Embed clips post to the clips route with the page's cookies and
X-CSRF-TOKENfrom the page's<meta name="csrf-token">(the component emitsdata-clips-csrf="meta", the opt-in the embed requires). So they work on your own pages, under the same recipes as module mode (see Clips), with the meta tag present. They cannot work on the package's embed page, which runs cross-origin in someone else's iframe with no session, so that page never enables clips.
<x-scarlett::player> is the same component.
Embed page
GET /v/{uuid} (embed.route, route scarlett.embed.show) serves the media full-window in
the embed bundle, with noindex (meta and X-Robots-Tag), Open Graph tags, the brand colour
and a Content-Security-Policy: frame-ancestors built from embed.allowed_domains (* when
empty; each domain also allows its subdomains). The brand colour comes from
MediaSource::$meta['brand_color'] and ['brand_text_color']. When beacons are on, their
route is registered and beacons.key is set, the page also beacons through the analytics
plugin (data-analytics-*): an embed iframe beaconing cross-origin is exactly the case the
CORS recipe under Beacons exists for. It never enables clips: in someone else's iframe there
is no session and no CSRF meta tag of yours to send. It carries no chapters or captions either,
since the MediaSource has no field for them. Publish the view with --tag=scarlett-views.
embed.allowed_domains takes bare host names (example.com, which also allows its
subdomains). A scheme, port, path or leading *. is stripped, and anything that is still not
a host name throws InvalidEmbedConfigException.
The page honours autoplay, muted, startTime and shareUrl from the query string.
embed.route must contain exactly one parameter, the media id.
ScarlettPlayer::embedUrl($video); // a URL: give it to the share plugin as embedBaseUrl ScarlettPlayer::embedUrl($video, now()->addDay()); // signed, expiring ScarlettPlayer::embedCode($video); // the <iframe> snippet: give it to a CMS paste field
Signing. embedUrl() signs when the media is protected, when embed.always_sign is on,
or when an expiry is given. A signature minted without an explicit expiry lasts
embed.signed_ttl seconds (one day by default), because the URL is printed into every page
that shows the media (the share button's embed base) and would otherwise be a permanent
link to paid content; set it to null to sign without expiry. The page then requires a valid signature, and any signature that
is present is always checked. The share plugin appends startTime and shareUrl to
whatever base URL it is given, so ValidateEmbedSignature leaves embed.unsigned_params
(startTime, shareUrl, autoplay, muted by default) out of the check. The signature
covers the media id and the expiry: expires and signature are always signed, even if
listed in unsigned_params. The presentation parameters are validated separately:
startTime must be plain seconds (42 or 42.5), and shareUrl must be an absolute
http(s) URL with no userinfo, backslash, whitespace or control character, whose host is
plain ASCII (no % escapes, no non-ASCII characters), on an allowed domain when
embed.allowed_domains is set. A parameter that fails is dropped and
never echoed. A tampered id answers 403.
oEmbed
GET {prefix}/oembed?url=<embed page URL> (route scarlett.oembed.show) answers with an
oEmbed video response whose html is embedCode(). It honours maxwidth and maxheight,
answers 404 for a URL that is not this app's embed page, and 501 for a format other than
json. For media that must be signed, the URL has to carry a valid signature (401
otherwise), and the snippet keeps that URL's expiry, so a lookup never mints a longer-lived
link.
Queues and workers
Beacons and clips use separate queues and, optionally, separate connections, so a five-minute render never sits in front of ten thousand heartbeats:
| Key | Default |
|---|---|
beacons.connection |
SCARLETT_BEACON_QUEUE_CONNECTION, null for the app default |
beacons.queue |
SCARLETT_BEACON_QUEUE, scarlett-beacons |
clips.connection |
SCARLETT_CLIP_QUEUE_CONNECTION, null for the app default |
clips.queue |
SCARLETT_CLIP_QUEUE, scarlett-clips |
A queue name selects nothing about asynchrony. scarlett:doctor warns when either
resolved connection is sync outside the local environment, and notes that database
writes each job row synchronously (one insert per heartbeat: acceptable, but a database
write all the same).
The two jobs
| Job | Queue | Carries | Timeout | Tries |
|---|---|---|---|---|
ProcessBeacon |
beacons.queue |
the beacon payload | 30 s | 5, backoff 5 s, 30 s, 120 s |
RenderClip |
clips.queue |
the clip id only | generator timeout + 30 s | clips.max_attempts (3) |
Beacon jobs are small and plentiful: the analytics plugin sends a heartbeat every
player.heartbeat_interval seconds per viewer, 10 by default. Each player sends 60 / interval
heartbeats a minute against the scarlett-beacons limit (beacons.throttle, 600,1 per IP):
a 2 s interval is 30 a minute per player, so a page with several players, or many viewers
behind one NAT, needs a higher limit. Clip jobs are few, CPU-heavy and slow.
Timeouts: the 330-second arithmetic
RenderClip::$timeout is clips.generators.<driver>.timeout + 30. With the default
local-ffmpeg timeout of 300 s, that is 330 s. The unique lock
(ShouldBeUniqueUntilProcessing, an hour) covers the wait in the queue; once a worker starts,
a second delivery cannot claim the clip until the render is older than the timeout plus 60
seconds.
Two settings outside the package must agree with it, or Laravel hands a still-rendering job to a second worker:
- The clips worker runs with
--timeoutof at least 330. - The clips connection's
retry_after(or Horizon's supervisortimeout, plus the connection'sretry_after) is greater than 330. 360 leaves room.
Raise the generator timeout and all three move together: timeout T gives a job timeout of
T + 30, a worker --timeout of at least T + 30, and a retry_after above T + 30.
A dedicated clips connection is the clean way to give clips a long retry_after without
slowing redelivery for everything else:
// config/queue.php 'connections' => [ 'redis' => [/* your app default, retry_after 90 */], 'redis-clips' => [ 'driver' => 'redis', 'connection' => 'default', 'queue' => 'scarlett-clips', 'retry_after' => 360, 'block_for' => null, ], ],
SCARLETT_CLIP_QUEUE_CONNECTION=redis-clips
Worker allocation
# Beacons: several cheap workers. php artisan queue:work --queue=scarlett-beacons --timeout=60 # Clips: few workers, one per CPU-heavy slot, on the clips connection. php artisan queue:work redis-clips --queue=scarlett-clips --timeout=330
Do not put both queues on one worker: a render then blocks every heartbeat behind it.
scarlett:doctor warns when beacons and clips share one queue on one connection.
Clip workers need the pcntl extension: without it a queue worker cannot enforce
RenderClip's timeout, a hung render is never killed, and the clip is only recovered when
scarlett:clips:reconcile finds it stuck. scarlett:doctor warns when clips are on and
pcntl is missing.
Horizon
// config/horizon.php 'environments' => [ 'production' => [ 'scarlett-beacons' => [ 'connection' => 'redis', 'queue' => ['scarlett-beacons'], 'balance' => 'auto', 'minProcesses' => 2, 'maxProcesses' => 10, 'timeout' => 60, 'tries' => 5, ], 'scarlett-clips' => [ 'connection' => 'redis-clips', 'queue' => ['scarlett-clips'], 'balance' => 'simple', 'processes' => 2, // one per CPU-heavy slot 'timeout' => 330, // at least RenderClip::$timeout 'tries' => 3, // clips.max_attempts ], ], ],
Horizon's supervisor timeout must stay below the connection's retry_after (here 330
under 360), as Horizon itself requires.
Diagnostics
php artisan scarlett:doctor
Prints one row per check and exits non-zero when any check fails (warnings alone pass):
| Check | Fails or warns when |
|---|---|
| media mapping | media.model is unset, not a model, or neither implements ScarlettMedia nor maps playback_url, is_live and is_protected. A custom resolver passes with a note |
| queue connections | a module's connection is not defined in queue.connections (fail), or is sync outside local (warn) |
| beacon key | beacons are on and beacons.key is empty or has surrounding whitespace |
| beacon store | the BeaconStore binding does not resolve (fail), or the null store is bound while beacons are on (warn) |
| embed bundle | player.mode is embed and the bundle URL cannot be built (fail), or only the embed page would need it (warn) |
| embed domains | an embed.allowed_domains entry is not a host name (fail; otherwise every embed page request fails) |
| beacon ip column | beacons.store_ip is on but scarlett_views.ip_address does not exist, because it is created only when store_ip was on at migrate time (fail); the table is missing or cannot be inspected (warn) |
| beacon signals columns | missing signal columns on views/errors (warn); passes with the complete schema or a custom store; use the signals-only upgrade tag and restart workers |
| beacon diagnostic columns | missing decoded_frames_total, dropped_frames_total or viewer_ref on views (warn); passes with the complete schema or a custom store; publish --tag=scarlett-migrations-diagnostics, migrate and restart workers |
| beacon seq column | raw-event storage is on but scarlett_beacon_events.seq is missing, or the table cannot be inspected (warn); passes when the column exists or beacons.store_raw_events is off |
| beacon context | beacons.context is set but is not a class implementing ResolvesBeaconContext, or scarlett_views.server does not exist, on a table migrated by 0.1 (fail); the table is missing or cannot be inspected (warn) |
| beacon route | routes.beacons is on but scarlett.beacons.store is not registered, e.g. a stale route cache (fail); the route is registered while beacons.enabled is off (warn) |
| beacon cors | beacons are on and the CORS config would lose the unload beacon: the path is missing, supports_credentials is off, * origins with credentials, or X-API-Key not allowed (warn) |
| beacon queue | beacons and clips share one queue on one connection (warn) |
| Clip toolchain (ffmpeg, ffprobe) | the configured binaries are not runnable (fail) |
| Clip disk | clips.disk is not a configured disk, cannot be written, does not keep a private write private, or cannot issue the temporary URLs previews need (fail) |
| Clip worker timeout (pcntl) | clips are on and the pcntl extension is missing, so a worker cannot enforce RenderClip's timeout (warn) |
Testing with the fake
ScarlettPlayer::fake() swaps the facade root for Hei\ScarlettPlayer\Testing\FakeScarlett,
which records what your app resolved and the beacons and clip requests it saw:
use Hei\ScarlettPlayer\Data\MediaSource; use Hei\ScarlettPlayer\Facades\ScarlettPlayer; $fake = ScarlettPlayer::fake()->withMedia(new MediaSource( id: 'video-1', playbackUrl: 'https://cdn.example.test/video-1.m3u8', isLive: false, isProtected: false, duration: 120.0, )); // ... exercise your app ... $fake->assertResolved('video-1'); // or assertNothingResolved() $fake->assertBeaconRecorded(fn (array $beacon): bool => $beacon['event'] === 'viewStart'); $fake->assertClipRequested();
Media registered with withMedia() answers without touching the database; any other id
falls through to the bound resolver.
Compatibility
| Package | Player wire contracts (@scarlett-player/*) |
Wire fixture set | Notes |
|---|---|---|---|
v0.1.0 |
1.17.x, pinned at 1.17.0 (player.player_version) |
tests/Fixtures/wire/1.17.0/, captured |
Captured by the player repo's harness against the v1.17.0 checkout (21 fixtures, a 34-beacon session sequence, 12 harness assertions passing); the browser test runs the npm 1.17.0 embed bundle |
v0.2.0 |
1.17.x, pinned at 1.17.0 (player.player_version) |
the 1.17.0 set, captured (removed in v0.2.1) |
The same captured fixture set as v0.1.0, unchanged. Embed mode now carries chapters, captions and clips on 1.17.0, with the embed.addon.chapters and embed.addon.clips addon files beside the bundle |
v0.2.1 |
1.19.x, pinned at 1.19.1 (player.player_version) |
tests/Fixtures/wire/1.19.1/, captured |
Recaptured by the player repo's harness against the v1.19.1 checkout (21 fixtures, a 26-beacon session sequence, 14 harness assertions passing); the same keys, events and exit types as 1.17.0, and the harness has no video-change scenario, so the 1.19 new-view boundary is not in the set. Embed mode carries chapters, captions and clips (from player 1.17.0), with the embed.addon.chapters and embed.addon.clips addon files beside the bundle; the browser test runs the npm 1.19.1 embed bundle |
v0.3.0 |
Still pinned at 1.19.1; additive ingest support for the 1.19.3 source contract | 1.19.1 captured set retained; tests/Fixtures/wire/1.19.3/ derived (recapture owed) |
Knows beaconSeq and seekSource; nullable raw seq orders timestamp ties. On upgrading the player to 1.19.3, rebuffer_count drops sharply, mostly on Safari: waiting under the default 250 ms rebufferGraceMs no longer counts; rebufferStart is sent about 250 ms after the stall began, while rebuffer_ms still counts from the first waiting. seek_count and seeking rows rise because progress-bar, keyboard, replay and native-control seeks were not sent before 1.19.3. Both are breaks in continuity across the player upgrade, not ingest bugs. No player repin in this package release |
v0.4.0 |
Default pin 1.19.1; prepared for 1.20.0 signals and QoE v1/v2 ingest | tests/Fixtures/wire/signals-candidate/, real browser captures from the local signals source, still labeled 1.19.3 by its package metadata |
Structured errors, nullable scores, interval metrics, page context and privacy options. Published 1.20.0 artifact verification and recapture remain pending; npm returned 404 during preparation. Batching remains off and unsupported |
v0.5.0 |
1.22.x, pinned at 1.22.0 (player.player_version, was 1.19.1) |
tests/Fixtures/wire/1.22.0-derived/, derived (recapture owed); the captured 1.19.1 set retained |
Verified against the published npm 1.22.0 analytics, embed, chapters, share and captions packages and the CDN v1.22.0/ files (byte-identical to npm); the browser test runs the npm 1.22.0 embed bundle. The player harness could not capture 1.22.0: three of its own assertions still expect the pre-1.22 unload subset. Heartbeat and unload viewEnd carry every counter; elementSeekCount, reconnectCount, reconnectDuration, dvrTime; reconnecting and recovered events; error networkState, readyState, online, sourceHost, reconnecting; pause_ms and heartbeat-only media_duration; liveEnded; null bitrates and live completionRate. Needs the additive scarlett-migrations-reconnects migration. scarlett_view_errors.fatal now follows severity. seek_count coalesces element seek bursts from 1.22. Embed attributes, Chapter/CaptionSource shapes, addon files and the share snippet are unchanged from 1.19.1 |
v0.6.0 |
1.22.x and 1.23.x, pinned at 1.22.0; ingests the gaugeScale: 'percent' marker 1.23.0 adds to viewEnd (kept as a custom dimension) |
Unchanged from v0.5.0 |
Additive canonical gauges: completion_ratio / rebuffer_fraction with independent stamps, normalized to 0..1 from an explicit marker or the source-audited producer registry (scarlett-player 1.22.0 proven percent; magnitude never decides units). Unknown scale, invalid marker and live completion write a stamped null; legacy completion_rate / rebuffer_ratio keep the received wire values. Needs the additive scarlett-migrations-gauges migration on existing hosts; beacon gauge columns doctor check; dry-run-first scarlett:views:backfill-gauges. Readers must move averages and filters to the canonical columns (see Canonical gauges) Also carries the never-released 0.5.1 migration fix: the scarlett-migrations-signals and scarlett-migrations-reconnects copies are dated at publish time, both upgrade migrations are idempotent and resumable, and a copy that sorts before its tables stops with a named error (see Upgrade migrations) |
v0.7.0 |
1.22.x, 1.23.x and 1.24.x, pinned at 1.24.0 (player.player_version, was 1.22.0) |
tests/Fixtures/wire/1.24.0/ (captured by the player harness at 1.24.0) and tests/Fixtures/wire/1.24.0-derived/ (derived, recapture owed) |
Verified against the published npm 1.24.0 embed, chapters, share, captions and core packages and the CDN v1.24.0/ embed bundle (byte-identical to npm); the browser test runs the npm 1.24.0 embed bundle. Nulls for unknown completionRate and unusable rebufferRatio (1.23), coalesced seeking beacons (1.23), a new view on a new source after a view ended, including views that never played (1.24), and retried hls.js mux errors with live reconnects (1.24). No migration and no schema change. The embed attribute names, the Chapter/CaptionSource shapes, the addon file names and the share snippet are unchanged from 1.22.0 |
v0.8.0 |
1.22.x through 1.25.x, pinned at 1.25.0 (player.player_version, was 1.24.0) |
tests/Fixtures/wire/1.25.0/ (captured by the player harness at 1.25.0) |
Verified against the published npm 1.25.0 embed, chapters, share, captions and core packages and the CDN v1.25.0/ files (byte-identical to npm); the browser test runs the npm 1.25.0 embed bundle. sentAt is a known field, kept in the raw log and left out of custom and the deduplication hash; the live seek window is a custom dimension. Startup and rebuffer readings rise against 1.24 (frame-based videoStart, frozen-picture rebuffers), and load timeouts reconnect. Additive diagnostics migration (scarlett-migrations-diagnostics): monotonic decoded_frames_total / dropped_frames_total and the indexed viewer_ref, copied from the server context (beacons.viewer_ref_key); scarlett:views:inspect for support lookups. The embed attribute names, the Chapter/CaptionSource shapes, the addon file names and the share snippet are unchanged from 1.24.0 |
Player 1.19.3's rebufferGraceMs: 0 restores immediate rebuffer counting. Its
rebufferStart.timestamp is the send time, not backdated to the first waiting;
confirmed stall durations include the grace. This is player behaviour, not a new PHP
config key or embed data-* attribute.
Two contract rules keep the package and the player from drifting:
- Additive tolerance. Unknown top-level beacon keys are stored in
custom, never rejected, so upgrading the player ahead of the package never breaks ingest. - Batching remains unsupported. Keep the player's
batchoption (from 1.20.0) disabled. Both an array body and{ batch: 1, sentAt, events }receive 422. This release does not emitdata-analytics-batchor enable batching. A batch-capable ingest must be implemented and tested before opting in; accepting both shapes is a separately versioned contract.
The 1.19.1 wire fixture set is fully captured and kept as the replay for 1.19.x players.
The 1.19.3 and 1.22.0-derived beacon fixtures are derived from the analytics source
and explicitly marked derived (recapture owed); they do not claim a captured transport
(the 1.22.0 set was re-verified against the published source). The share-built embed
iframe fixture is also derived: it is generated by the share plugin's own snippet
builder, taken byte for byte from the published 1.25.0 dist (its buildEmbedSnippet() is
identical to 1.22.0), because the capture harness has no share scenario.
The embed bundle location follows player.embed_bundle, a template defaulting to
{cdn_url}/v{player_version}/embed.js: the player CDN serves one directory per release
(for example, v1.25.0/) and no /latest/ alias yet, so the embed bundle is always the pinned version.
Development
composer install vendor/bin/pest # all groups; browser and integration skip without their tools vendor/bin/pest --parallel # the same, one process per core SCARLETT_BROWSER=1 vendor/bin/pest --group=browser vendor/bin/pest --group=integration # needs ffmpeg and ffprobe on PATH composer lint # pint --test composer analyse # phpstan level 8
The suite runs on in-memory SQLite. The database group (the beacon store and the clips
relation) also runs on MySQL and Postgres in CI, because the merge SQL is dialect-specific.
To run it locally, point the DB_* variables at a server and pass --parallel: each worker
then creates and uses its own database, DB_DATABASE suffixed with its worker number.
The worker databases are kept between runs and reused, so two runs at the same time against
one server need different DB_DATABASE values.
DB_CONNECTION=mysql DB_HOST=127.0.0.1 DB_PORT=3306 DB_DATABASE=scarlett DB_USERNAME=root DB_PASSWORD=secret \
vendor/bin/pest --parallel --group=database
Browser tests
The browser group needs Node 20 or later, openssl and Playwright's Chromium (npm install, then
npm run browser:install). It serves the page on 127.0.0.1:8001, the ingest on 127.0.0.1:8002
and a TLS proxy for the beacon origin on 127.0.0.1:8443. See tests/Browser/README.md
for what each origin is for and how to run the proxy by hand.
License
MIT. See LICENSE.