Search by

evolvephp / insight

josiahking

Local diagnostic capture, persistence, query and access-policy foundation for EvolvePHP 2.

dev-main 2026-10-21 11:01 UTC

This package is auto-updated.

Last update: 2026-11-16 13:41:01 UTC


README

evolvephp/insight

Local diagnostic capture, persistence, query and access-policy foundation for EvolvePHP 2.

For practical composition and operational boundaries, see Insight operations.

Responsibility

Evolve Insight consumes safe Core execution observations and explicitly submitted diagnostic candidates, then collects them into immutable, bounded diagnostic batches. This package provides the storage-neutral DiagnosticBatchSink contract, immutable DiagnosticBatch values, DiagnosticBatchCollector, which implements Core's ObservationSink boundary, an explicit DiagnosticPipeline composition helper for applications that opt in to collection plus persistence, and a storage-neutral read/query foundation guarded by an application-supplied access policy.

Applications may also explicitly wrap accepted database, cache, queue, storage and outbound HTTP interfaces with Insight-owned diagnostic decorators. The decorators submit candidates through DiagnosticEntrySink, which DiagnosticPipeline and DiagnosticBatchCollector implement, so the existing capture policy remains the sole admission path. Application code supplies one ExecutionCorrelation to Core's ExecutionOrchestrator as an ExecutionContextAttacher and to a DiagnosticRecorder alongside the chosen sink. Correlation retains only the active execution identifier value and bounded statement-repeat counts; detaching clears them. An operation outside an attached execution creates no infrastructure entry. Insight does not discover infrastructure services or register decorators globally.

Incoming HTTP server diagnostics

HTTP server diagnostics are explicit opt-in PSR-15 composition. Use one ExecutionCorrelation for both the ExecutionOrchestrator and Insight's HTTP middleware. Install HttpServerDiagnosticMiddleware outside RoutingRequestHandler so route-not-found, method-not-allowed and routing exceptions are included. Install MatchedRouteDiagnosticMiddleware early in the routing middleware list, after routing has attached RouteMatch. An outer middleware installed further inward can miss failures raised before it.

$correlation = new ExecutionCorrelation();
$pipeline = DiagnosticPipeline::collecting(
    $batchSink,
    maximumRetainedObservationCount: 100,
    maximumRetainedDiagnosticEntryCount: 100,
);
$routing = new RoutingRequestHandler(
    $matcher,
    [new MatchedRouteDiagnosticMiddleware(), ...$applicationRoutingMiddleware],
);
$handler = new MiddlewarePipeline(
    [new HttpServerDiagnosticMiddleware($pipeline, $correlation), ...$applicationOuterMiddleware],
    $routing,
);
$executions = new ExecutionOrchestrator($services, $pipeline, [$correlation]);
$kernel = new HttpKernel($handler, $executions);

The outer middleware records one evolve.http.server/request entry per active execution. The inner middleware copies only the declared route template from RouteMatch into request-local state. The entry may contain a bounded method, declared route template up to the diagnostic string limit, returned status code, outcome, monotonic duration and throwable class. Missing or oversized route templates are omitted. A returned 4xx or 5xx status is a successful returned operation. Insight does not read URI paths, queries, route parameter values, headers, cookies, bodies, identities or trace headers for this entry. The same capture policy, redaction, sampling and retention bounds apply through DiagnosticPipeline. Diagnostic failures do not change the response or throwable and do not retry the handler.

Infrastructure diagnostics

  • DatabaseDiagnosticDecorator records operation, outcome, monotonic duration when available, bounded statement fingerprint, parameter count and types, optional operation name, affected-row count for execute(), bounded repeat occurrence and normalized failure details. A query result iterable is returned untouched; rows are not consumed to count them. Repeat counts group the database operation kind, bounded SQL fingerprint and developer operation name when present; the grouping does not prove semantic query equivalence or diagnose N+1 behavior. The repeat group map is limited to 64 groups per execution and resets on detach. Transaction callbacks receive a decorator around the active connection supplied by the adapter, so operations invoked through that connection enter the same diagnostic batch.
  • DatabaseDiagnosticPolicy accepts an explicit positive slow threshold in nanoseconds. An operation is marked slow at or above that threshold; no default threshold is inferred. Optional raw SQL inspection is disabled by default and, when enabled, truncates SQL at a configured maximum. Its sql attribute is classified as BusinessSensitivePayload, so persistence also requires the application's DiagnosticCapturePolicy to accept that classification. Bound parameter values and names are never captured.
  • CacheDiagnosticDecorator wraps all PSR-16 operations without recording keys, values, defaults, TTL details or iterable contents. It does not pre-consume iterable inputs or returned results.
  • QueuePublisherDiagnosticDecorator and QueueReceiverDiagnosticDecorator record publication and receive attempts. A received delivery is wrapped to record acknowledge and reject only while its originating execution remains attached; its message and settlement behavior remain delegated to the original delivery. Queue names, payloads and message metadata are excluded.
  • StorageDiagnosticDecorator wraps put, open and delete; a returned reader is wrapped for read and close only while its originating execution remains attached. Storage keys and object bytes are excluded. Write chunks are not pre-consumed, reads are not expanded, and repeated close calls remain delegated to the reader.
  • HttpClientDiagnosticDecorator wraps PSR-18 requests and records a conservative HTTP method, response status, outcome and duration when available. A 4xx or 5xx response remains a returned response. Request and response bodies, headers, cookies, full URIs, paths, queries, user-info and credentials are excluded; streams are never read or rewound and no tracing headers are injected.

All infrastructure recording is best effort: diagnostic clock, metadata, sink or capture failures do not change the underlying operation's result, replace its throwable or retry it. Infrastructure diagnostics remain local Insight evidence, not OpenTelemetry export.

Insight also provides a storage-neutral projection boundary for finalized diagnostic batches. DiagnosticBatchProjector detaches a DiagnosticBatch into a primitive-only DiagnosticBatchSnapshot made of string-backed execution identity, execution kind, ordered DiagnosticObservationSnapshot values, ordered accepted diagnostic-entry snapshots and separate dropped observation and diagnostic-entry counts. Snapshots do not retain Core Observation, execution identifier, diagnostic-entry objects, diagnostic-attribute objects, request, response, container, throwable or execution-scope objects.

Insight includes a detached diagnostic capture-policy foundation for diagnostic sources. DiagnosticEntry and DiagnosticAttribute represent bounded primitive-only diagnostic data: execution identifier value, diagnostic category, diagnostic name and ordered attributes whose values are limited to string, int, float, bool or null. Attribute names, entry identifiers, categories, names and string values are bounded, and non-finite floats, arrays, objects, resources and callables are not accepted.

Current bounded behavior:

  • collection starts only after Core reports an execution start
  • active collection state is isolated by Core execution identifier value
  • observations for unknown or already completed executions are ignored
  • candidate diagnostic entries are retained only when their execution identifier exactly matches an active execution
  • diagnostic entries for unknown or already completed executions are ignored and never start collection
  • candidate diagnostic entries pass through DiagnosticCapturePolicy before becoming retainable batch data
  • retained diagnostic entries are bounded per execution and keep accepted arrival order
  • retained observations are bounded per execution
  • dropped observation counts are deterministic and non-negative
  • dropped diagnostic-entry counts are deterministic, non-negative and count only policy-accepted entries that exceeded the per-execution entry bound
  • completed executions are forgotten before the finalized batch is handed to the configured sink
  • sink failures propagate to the existing Core instrumentation boundary

Current watcher behavior:

  • ObservationDiagnosticWatcher is an explicit extension point for deriving candidate diagnostic entries from one existing Core Observation
  • DiagnosticPipeline accepts zero or more configured observation watchers and validates them during composition
  • watcher registration is caller-owned; Insight does not discover watchers, mutate Core state, read environment configuration or install global/static runtime registration
  • watcher order is preserved, and each watcher's produced-entry order is preserved
  • ExecutionStarted observations open the execution batch before watcher candidates are captured
  • ExecutionCompleted watcher candidates are captured before the completion observation finalizes and forgets the execution
  • watcher-produced entries must carry the exact triggering execution identifier value and are rejected before capture when they do not
  • watcher candidates flow through the same DiagnosticBatchCollector::capture() path as explicitly submitted diagnostic entries, including filtering, redaction, sampling, retained-entry bounds and dropped-entry accounting
  • watcher failures during completion still allow the collector to observe completion and finalize the execution before the failure is rethrown through the normal observation sink call path

Current first-party abnormal diagnostics:

  • failed HandlerCompleted observations produce evolve.execution / handler-failed
  • failed ScopeCloseCompleted observations produce evolve.runtime / scope-close-failed
  • QuarantineRequired observations produce evolve.runtime / quarantine-required

These diagnostics use bounded primitive operational attributes already present on the triggering observation, including execution kind, error type when available and normalized process-reuse decision when available. They do not include exception messages, stack traces, request or response values, route parameters, headers, cookies, payloads, containers, scopes, service instances, tenant identifiers, timestamps, durations or arbitrary application values.

Current storage behavior:

  • DiagnosticBatchStore defines minimal save, exact execution-identifier lookup and newest-first bounded reads
  • DiagnosticBatchReader defines the separate read/query boundary for exact detail lookup and bounded cursor queries without adding query methods to the persistence contract
  • InMemoryDiagnosticBatchStore keeps snapshots in insertion order, requires an explicit positive stored-batch count and prunes oldest retained snapshots first
  • SqliteDiagnosticBatchStore provides an optional persistent local-development adapter for caller-supplied SQLite PDO connections, requires an explicit positive stored-batch count and prunes by SQLite sequence order
  • DiagnosticBatchSnapshotCodec stores snapshots as a versioned primitive JSON payload, writes the current expanded schema with diagnostic entries, continues to read legacy observation-only version 1 payloads, and rejects malformed, unsupported or unexpected persisted data during reads
  • duplicate execution identifiers are rejected and never replace the original snapshot
  • StoringDiagnosticBatchSink projects accepted batches and saves the detached snapshot through a configured store
  • storage remains optional and unwired; installing Insight does not create storage automatically

Both first-party stores use explicit count-bounded retention. Callers configure a positive maximum stored-batch count, and successful unique saves leave no more than that many retained diagnostic batch snapshots. Pruning is deterministic oldest-first insertion order; SQLite uses its monotonic sequence column as the insertion-order authority. Duplicate execution identifiers are rejected before pruning, so a duplicate save at capacity does not evict or mutate retained snapshots. Time-based retention is not provided.

The SQLite store creates its diagnostic table only when explicitly constructed with a SQLite PDO; it does not discover a default path, read application database configuration or automatically use application storage. SQLite reads use deterministic insertion-order sequence values for newest-first results, not diagnostic timestamps. Corrupt stored payloads are not decoded during construction, but the affected find() or latest() read fails explicitly.

pdo_sqlite is a runtime requirement only for applications that explicitly use SqliteDiagnosticBatchStore; it is not required for installing or using the non-SQLite Insight functionality.

Current query and access behavior:

  • DiagnosticBatchQuery supports a page size from 1 to 100, an optional exclusive cursor and optional exact filters for persisted execution kind, diagnostic category and diagnostic name
  • cursors are execution identifiers from the final item of the previous page, and the next page starts strictly older than that retained batch
  • unknown or pruned cursors fail closed instead of restarting from newest or returning a misleading empty page
  • query results are newest-first according to the store's insertion authority: in-memory insertion order or SQLite sequence
  • DiagnosticBatchPage contains detached DiagnosticBatchSummary items and a next cursor only when another older matching result exists
  • summaries expose only execution identifier, execution kind, retained observation count, retained diagnostic-entry count and dropped counts
  • category and name filters are exact, and when both are supplied they must match the same diagnostic entry snapshot
  • exact detail lookup returns the detached persisted DiagnosticBatchSnapshot
  • DiagnosticQueryService requires an application-supplied DiagnosticAccessPolicy before list or detail reads and performs authorization before delegating to storage
  • Insight does not provide a default access policy, discover users, define roles, authenticate requests, inspect sessions, infer localhost or automatically allow development access

Current capture-policy behavior:

  • DiagnosticDataClassification provides explicit machine-readable classifications for public operational metadata, internal operational metadata, personal data, authentication data, secret data, business-sensitive payloads and regulated data
  • DiagnosticCapturePolicy accepts public and internal operational metadata by default; personal, business-sensitive and regulated data are excluded unless explicitly enabled by application code
  • secret and authentication data are never deliberately accepted raw
  • DefaultDiagnosticRedactor deterministically suppresses secret and authentication attributes, and replaces common sensitive operational machine names such as authorization, password, cookies, tokens, access tokens, refresh tokens, API keys, secrets, session identifiers and set-cookie names with [REDACTED]
  • compact and separator-based machine names such as token, accessToken, access_token, refreshToken, refresh.token, apiKey, api-key, sessionId, session.id, sessionIdentifier, setCookie and set-cookie are treated as sensitive when otherwise classified as operational metadata
  • unrelated lookalikes such as token_count are not redacted merely because they contain a sensitive word
  • DiagnosticCaptureFilter supports exact category and diagnostic-name disabling for volume control only
  • DeterministicDiagnosticSampler supports integer percentage sampling from 0 to 100 using a stable hash of the execution identifier
  • sampling controls diagnostic volume only; it is not authorization, authentication, security enforcement, redaction, legal retention or an error-retention guarantee
  • accepted attributes keep their original order, use first-accepted-wins duplicate-name handling and are capped by a deterministic retained-attribute limit

Redaction and classification filtering happen before accepted rich diagnostic values reach persistence. Candidate capture entries can be submitted explicitly to DiagnosticBatchCollector or DiagnosticPipeline, and configured observation watchers can derive candidates from existing Core execution observations. Policy-accepted entries become part of the same finalized execution batch as Core observations, are projected into detached diagnostic-entry snapshots and can persist through the configured projector, snapshot codec and store. This integration remains opt-in and does not add automatic runtime wiring.

Operational characteristics:

  • composition is explicit and opt-in
  • Insight does not discover watchers, create hidden global registration, inspect the environment or automatically expose local diagnostics
  • retention is count-bounded rather than time-based
  • in-memory and SQLite query behavior is bounded by the requested page size, but exact filters may require scanning retained snapshots
  • SQLite is local-development persistence through a caller-owned PDO; it is not application database integration
  • local Insight diagnostics are framework-owned diagnostic evidence, not production telemetry export
  • Core remains independent of Insight, and Insight remains independent of OpenTelemetry and Evolve Observe

Native Dashboard

The first-party native PHP dashboard is optional and caller composed. Build a DiagnosticQueryService from a caller-owned DiagnosticBatchReader and an explicit DiagnosticAccessPolicy, then call DashboardRoutes::native($queries, $responseFactory, $localizationContext, DashboardExposure::localDevelopment()). The returned RouteCollection contains only GET /__evolve/insight and GET /__evolve/insight/{execution} by default; the caller adds those routes to its Evolve HTTP routing composition. A validated route prefix can be passed as the fifth argument. Both native() and compose() require a caller-selected DashboardExposure. Installing Insight alone does not register routes, construct storage, render views or inspect the environment.

DashboardRoutes::native composes the framework-owned PHP templates through ViewSource, FilesystemViewPathResolver and NativePhpViewRenderer. DashboardRoutes::compose accepts a caller-supplied ViewRenderer and Translator instead. Framework copy uses the Insight en PHP message catalog through standard i18n LocalizationContext fallback. Neither Twig nor Blade is required. Persisted and request-derived values pass through native View escaping; no diagnostic value is trusted HTML. Pages use only bundled inline styling and request no external assets.

The list defaults to page_size=25. Accepted parameters are page_size (1 to 100), cursor, execution_kind, category and name; filters are exact and the newest-first next link retains active filters with an exclusive cursor. Malformed accepted values, unknown query keys or unknown/pruned cursors yield HTTP 400. An authorized missing execution yields 404. Denied list and detail reads yield 403 through DiagnosticQueryService before storage access, so denied detail does not reveal existence. HTML responses include Content-Type: text/html; charset=UTF-8 and Cache-Control: no-store, including errors.

Dashboard exposure is a separate caller choice from diagnostic authorization. Call DashboardExposure::localDevelopment() for local or development composition. There is no implicit exposure default. Production exposure requires the deliberate DashboardExposure::productionAuthorized() choice; no host, IP, environment variable or global is consulted. Neither exposure choice supplies or bypasses DiagnosticAccessPolicy. The dashboard reads only detached persisted snapshots and cannot restore values removed by capture policy. Authentication, user and role management, automatic registration, dashboard access audit emission, aggregation, offset pagination and production telemetry export are outside this dashboard.

Requirements

PHP ^8.4

Dependencies

  • evolvephp/core
  • evolvephp/database-contracts
  • evolvephp/http
  • evolvephp/i18n
  • evolvephp/view
  • evolvephp/queue-contracts
  • evolvephp/storage-contracts
  • psr/simple-cache
  • psr/http-client
  • psr/http-factory
  • psr/http-message
  • psr/http-server-handler
  • psr/http-server-middleware

Publication Status

EvolvePHP 2 is pre-release. This package is not yet independently published, and the current canonical source is the EvolvePHP monorepo:

https://github.com/josiahking/evolvephp

Current Limitations

This package does not provide time-based retention, timestamp or range queries, log/event/plugin/module/service-resolution/worker-memory/tenant diagnostic watchers, authentication, user or role management, default access policies, OpenTelemetry, Evolve Observe, trace propagation, automatic registration, application database integration, production telemetry export or production-ready diagnostics. The current scope-close and quarantine diagnostics are generic runtime signals only; they do not prove a specific request-scope leak, tenant leak or memory leak.

Licence

BSD-3-Clause. See LICENSE.md.