README

[[TOC]]

Overview

The Audit package is an open source Composer package for use in Laravel applications that is used by other Provisionesta packages to provide a consistent log syntax and add support for database transactions (upcoming release) for time sensitive events. Although this is purpose built for our packages, you are welcome to adopt this for your own standardized logging.

This is maintained by the open source community and is not maintained by any company. Please use at your own risk and create merge requests for any bugs that you encounter.

Problem Statement

When using Laravel Logging with the Log::info('Message', ['key1' => 'value', 'key2' => 'value']) syntax, it is easy to have inconsistency with log formatting that results in a variety of log messages and varying context keys.

The Provisionesta\Audit\Log::create() method provides a pre-defined set of context keys that allow us to improve indexing and searchability in external logging platforms, and ensures that all events provide as much context data as possible in a consistent format.

Sometimes you need to get a formatted array that can be added to a changelog or actioned upon programmatically instead of trying to tail a log file. An array is returned for each log entry that is created.

(Upcoming release) Since some log events need to be actioned, this package adds support for an audit_transactions database table that allows you to centrally manage it like a background job queue and trigger different actions and workflows based on your own application and business requirements.

Issue Tracking and Bug Reports

We do not maintain a roadmap of feature requests, however we invite you to contribute and we will gladly review your merge requests.

Please create an issue for bug reports.

Contributing

Please see CONTRIBUTING.md to learn more about how to contribute.

Maintainers

Name	GitLab Handle	Email
Jeff Martin	@jeffersonmartin	`provisionesta [at] jeffersonmartin [dot] com`

Contributor Credit

Jeff Martin

Installation

Requirements

Requirement	Version
PHP	`^8.0`
Laravel	`^8.0`, `^9.0`, `^10.0`, `^11.0`

Upgrade Guide

See the changelog for release notes.

Add Composer Package

composer require provisionesta/audit:^1.1

If you are contributing to this package, see CONTRIBUTING.md for instructions on configuring a local composer package with symlinks.

Publish the configuration file

This is optional. The configuration file the custom schemas if you are returning the parsed log entry as a variable.

php artisan vendor:publish --tag=audit

Usage Examples

Basic Usage

use Provisionesta\Audit\Log;

Log::create(
    event_ms: $event_ms,
    event_type: 'okta.api.post.success.ok',
    level: 'info',
    message: 'Success',
    metadata: [
        'okta_request_id' => 'REDACTED',
        'rate_limit_remaining' => $request->headers['x-rate-limit-remaining'],
        'uri' => 'users',
        'url' => 'https://dev-12345678.okta.com/api/v1/users?activate=true'
    ],
    method: __METHOD__
);

Comprehensive Usage

You can copy and paste this example anywhere in your code that you would create a log entry. Any arguments that are not relevant can be removed and will be considered null.

use Provisionesta\Audit\Log;

Log::create(
    attribute_key: 'xxx',
    attribute_value_old: 'xxx',
    attribute_value_new: 'xxx',
    count_records: count($array),
    dump: false,
    dump_keys: [],
    duration_ms: $duration_ms,
    duration_ms_per_record: (int) ($duration_ms / count($records)),
    errors: [],
    event_ms: $event_ms,
    event_ms_per_record: (int) ($event_ms / count($records)),
    event_type: '{provider}.{entity}.{action}.xxx',
    level: 'info',
    log: true,
    message: '{What happened}',
    metadata: [],
    method: __METHOD__,
    occurred_at: $entity->created_at,
    parent_id: $parent->id,
    parent_type: 'App\\Models\\{Provider}\\Application',
    parent_reference_key: 'name',
    parent_reference_value: $entity->organization->name,
    record_id: $entity->id,
    record_type: 'App\\Models\\{Provider}\\{Entity}',
    record_provider_id: $entity->provider_id,
    record_reference_key: 'name',
    record_reference_value: $entity->name,
    tenant_id: $entity->provider_organization_id,
    tenant_type: 'App\\Models\\{Provider}\\Organization',
    transaction: false
);

Example Output

[YYYY-MM-DD HH:II:SS] local.DEBUG: ApiClient::post Success {"event_type":"okta.api.post.success.ok","method":"Provisionesta\\Okta\\ApiClient::post","event_ms":627,"metadata":{"okta_request_id":"REDACTED","rate_limit_remaining":"16","uri":"users","url":"https://dev-12345678.okta.com/api/v1/users?activate=true"}"}

{
    "event_type": "okta.api.post.success.ok",
    "method": "Provisionesta\\Okta\\ApiClient::post",
    "event_ms": 627,
    "metadata": {
        "okta_request_id": "REDACTED",
        "rate_limit_remaining": "16",
        "uri": "users",
        "url": "https://dev-12345678.okta.com/api/v1/users?activate=true"
    }
}

Log Parameter Definitions

Parameter Name	Example Usage	Description
event_type (Required) `string`	`provider.entity.action.result.reason`	The octet notation event type that follows our codestyle conventions.

level (Required)
string debug
info
notice
warning
error
critical
alert
emergency The log level for the log entry message (Required)
string Validation Failed A short message to include in the logs. This will be auto-prefixed

 with the fully-qualified method name (the "noun") so you can keep
 the message focused on the "verb" language.</td>

method (Required)
string METHOD The method where this audit log is created in or is on behalf of. attribute_key
string (State Changes) The database column name that has changed. attribute_value_old
string (State Changes) The value in the database before the update. attribute_value_new
string (State Changes) The API value that is now updated in the database. count_records
int count($array) (Multiple records) Count of records processed. dumpconfig
string default (Response Schema) The array key in config/audit.php that contains the date, keys, strings schema configuration. The other dump* parameters are ignored if dump_config is set. dump_date
string c
Y-m-d
Y-m-d H:i:s (Response Schema) The PHP datetime format string for timestamps returned in the response array. dump_keys
array See docs (Response Schema) An filtered list of array of keys from the Log::create() method that returned in the response array. dump_strings
array See docs (Response Schema) An array of key value pairs of static strings that should be included in the in the response array (instead of having to add them yourself with collection transformation later). duration_ms
Carbon $duration_ms Carbon instance (timestamp) used for long running batch jobs to provide a point-in-time duration since job started. duration_ms_per_record
int (int) ($duration_ms / count($records)) Number of milliseconds divided by count of records. This is not auto-calculated to allow flexibility for custom Carbon timestamps errors
array Flat array of error message(s) that will be encoded as JSON event_ms
Carbon $event_ms Carbon instance (timestamp) that was initialized at the start of the action and provides a point-in-time duration for this specific action within a longer running job. event_ms_per_record
int (int) ($event_ms / count($records)) Number of milliseconds divided by count of records. This is not auto-calculated to allow flexibility for custom Carbon timestamps job_batch
string (Background Job Logs) The human identifier string or system ID of the batch of jobs. Format is at your discretion. job_id
string (Background Job Logs) The human identifier string or system ID of the specific job that triggered this log entry. Format is at your discretion. job_platform
string github
gitlab
lambda
redis
{your string} (Background Job Logs) The human identifier string of the platform that the background jobs are running in. Format is at your discretion. job_pipeline_id
string (Background Job Logs) The system ID of the CI/CD pipeline (if applicable). job_timestamp
string now()->getTimestamp() (Background Job Logs) The timestamp that the job or pipeline was started. This is useful for identifying which scheduled job timestamp triggered this event. job_transaction_id
string now()->getTimestamp() (Background Job Logs) An alternative to job_id that can be used for additional indexable identifiers used by your application or business logic. log
bool true (default)
false Whether to create a system log entry for this event. See docs. metadata
array An array of custom metadata that should be included in the log occurred_at
string 2023-02-01T03:45:27.612584Z A datetime that will be formatted with Carbon for when the event occurred at based on a created_at or updated_at API timestamp parent_id
string {uuid} (Many-to-Many Relationship Events) The database ID of the database model with a many-to-many relationship. parent_type
string App\Models\Path\To\ModelName (Many-to-Many Relationship Events) The fully-qualified namespace of the database model with a many-to-many relationship. parent_provider_id
string a1b2c3d4e5f6 (Many-to-Many Relationship Events) The API ID of the database model with a many-to-many relationship that is usually stored in the database in the provider_id column. parent_reference_key
string name (Many-to-Many Relationship Events) The database column name for value that is human readable in logs parent_reference_value
string (Many-to-Many Relationship Events) The value of the human readable database column record_id
string {uuid} The database ID of the affected database model record_type
string App\Models\Path\To\ModelName The fully-qualified namespace of the database model record_provider_id
string a1b2c3d4e5f6 The API ID of the affected database model that is usually stored in the database in the provider_id column. record_reference_key
string name The database column name for value that is human readable in logs record_reference_value
string The value of the human readable database column tenant_id
string {uuid} The database ID of the top-level organization/tenant for the provider tenant_type
string App\Models\VendorName\Organization The fully-qualified namespace of the database model of the top-level entity (organization, tenant, etc) for the provider. transaction
bool true
false (default) Whether to create a Transaction database entry for this event

Advanced Usage

Background Job Log Entry

You can add the job_* parameters if you are running background jobs and want to add metadata to your logs and transactions. All of these values (except job_timestamp) are freeform strings that you can standardize however you'd like.

use Provisionesta\Audit\Log;

Log::create(
    // ...
    job_batch: '{string}',
    job_id: '{string}',
    job_platform: '{string}',
    job_pipeline_id: '{string}',
    job_timestamp: now()->getTimestamp(),
    job_transaction_id: '{string}',
    // ...
);

Skipping Log Creation

You can specify true/false booleans for log, and transaction parameters. By default, log is true and transaction is false.

The parsed and formatted schema is always returned as an array.

`log`	`transaction`	Behavior
`true`	`false`	(Default) Log entry is created.
`true`	`true`	Log entry is created. Database row is created for transaction.
`false`	`true`	No log is created. Database row is created for transaction.
`false`	`false`	No log or transaction database row is created. Used for schema parsing.

Response Schema

One of the benefits to this package is the formatted array with predictable keys.

As an alternative to creating a transaction, you can also return a formatted array with all of the keys with provided values (or their default values). This is useful if you simply need an array that you will use for your own changelog or some other purpose. You can also disable log creation and use this like a data transfer object (DTO).

Simply define a variable to get the returned array.

use Provisionesta\Audit\Log;

// Create a log entry with no returned array
Log::create(
    // ...
    log: true
    // ...
);

// Define a variable with the returned array
$schema = Log::create(
    // ...
    log: true
    // ...
);

// Append the return array to an existing array of records that are not created in system logs
foreach($records as $record) {
    // ...

    $changelog[] = Log::create(
        // ...
        log: false
        // ...
    );
}

Example Response Array with No Configuration

use Provisionesta\Audit\Log;

$event_ms = now();

$result = Log::create(
    event_ms: $event_ms,
    event_type: "okta.api.post.success.ok",
    level: "info",
    message: "Success",
    metadata: [
        "okta_request_id" => "REDACTED",
        "rate_limit_remaining" => "199",
        "uri" => "users",
        "url" => "https://dev-12345678.okta.com/api/v1/users?activate=true"
    ],
    method: "Provisionesta\Okta\ApiClient::get"
);

dd($result);

// [
//     "datetime" => "2024-03-02T18:51:10+00:00",
//     "event_type" => "okta.api.post.success.ok",
//     "level" => "info",
//     "message" => "Success",
//     "method" => "Provisionesta\Okta\ApiClient::get",
//     "attribute_key" => null,
//     "attribute_value_old" => null,
//     "attribute_value_new" => null,
//     "count_records" => null,
//     "duration_ms" => null,
//     "duration_ms_per_record" => null,
//     "errors" => [],
//     "event_ms" => 4,
//     "event_ms_per_record" => null,
//     "job_batch" => null,
//     "job_id" => null,
//     "job_platform" => null,
//     "job_pipeline_id" => null,
//     "job_timestamp" => null,
//     "job_transaction_id" => null,
//     "metadata" => [
//       "okta_request_id" => "REDACTED",
//       "rate_limit_remaining" => "199",
//       "uri" => "users",
//       "url" => "https://dev-12345678.okta.com/api/v1/users?activate=true",
//     ],
//     "occurred_at" => null,
//     "parent_id" => null,
//     "parent_type" => null,
//     "parent_provider_id" => null,
//     "parent_reference_key" => null,
//     "parent_reference_value" => null,
//     "record_id" => null,
//     "record_type" => null,
//     "record_provider_id" => null,
//     "record_reference_key" => null,
//     "record_reference_value" => null,
//     "tenant_id" => null,
//     "tenant_type" => null,
// ]

Specifying Keys in Response Array

There are a large number of parameter keys in the schema. To avoid having to use Laravel Collections transform methods with the result array, you can simply pass an array of keys to the dump_keys array that you want to be included.

use Provisionesta\Audit\Log;

$result = Log::create(
    // ...
    dump_keys: [
        'event_type',
        'message',
        'attribute_key',
        'attribute_value_old',
        'attribute_value_new',
        'record_id',
        'record_type',
        'record_provider_id',
        'record_reference_key',
        'record_reference_value'
    ],
);

dd($result);

// [
//     "datetime" => "2024-03-02T18:53:35+00:00",
//     "event_type" => "okta.api.post.success.ok",
//     "message" => "Success",
//     "attribute_key" => null,
//     "attribute_value_old" => null,
//     "attribute_value_new" => null,
//     "record_id" => null,
//     "record_type" => null,
//     "record_provider_id" => null,
//     "record_reference_key" => null,
//     "record_reference_value" => null,
// ]

Specifying Custom Static Strings in Response Array

If you need to add custom static strings to your array, they can be specified in the dump_strings array. Strings are returned at the end of the array.

use Provisionesta\Audit\Log;

$result = Log::create(
    // ...
    dump_keys: [
        'event_type',
        'message',
        'attribute_key',
        'attribute_value_old',
        'attribute_value_new',
        'record_id',
        'record_type',
        'record_provider_id',
        'record_reference_key',
        'record_reference_value'
    ],
    dump_strings: [
        'custom_key' => 'my_value',
        'another_key' => 'my_value',
    ],
    // ...
);

dd($result);

// [
//     "datetime" => "2024-03-02T18:54:55+00:00",
//     "event_type" => "okta.api.post.success.ok",
//     "message" => "Success",
//     "attribute_key" => null,
//     "attribute_value_old" => null,
//     "attribute_value_new" => null,
//     "record_id" => null,
//     "record_type" => null,
//     "record_provider_id" => null,
//     "record_reference_key" => null,
//     "record_reference_value" => null,
//     "custom_key" => "my_value",
//     "another_key" => "my_value",
// ]

Custom Datetime Format in Response Array

The datetime is always returned in the first key (table column) of the array. You can customize the format using a string of supported PHP datetime format characters. By default, we use c for the ISO 8601 format (YYYY-MM-DDTHH:II:SS+00:00). You may want to simplify this with Y-m-d or Y-m-d H:i.

use Provisionesta\Audit\Log;

$result = Log::create(
    // ...
    dump_date: 'c',
    dump_keys: [
        'event_type',
        'message',
        'attribute_key',
        'attribute_value_old',
        'attribute_value_new',
        'record_id',
        'record_type',
        'record_provider_id',
        'record_reference_key',
        'record_reference_value'
    ],
    dump_strings: [
        'custom_key' => 'my_value',
        'another_key' => 'my_value',
    ],
    // ...
);

dd($result);

// [
//     "datetime" => "2024-03-02",
//     "event_type" => "okta.api.post.success.ok",
//     // ....
// ]

Standardized Configurations for Response Array

Reminder: You need to publish the configuration file for it to appear in config/audit.php or it will use the default one in the vendor/provisionesta/audit directory that cannot be modified.

It can be difficult to manage your simplified schemas throughout your code base.

You can define standardized simplified schemas in the config/audit.php file for each of your use cases. The default key is a placeholder that can be customized and you can add additional arrays for each type of resource if needed (ex. okta_user).

After a schema is defined, simply set the dump_config key to the same key that was defined in config/audit.php.

use Provisionesta\Audit\Log;

$result = Log::create(
    // ...
    dump_config: 'okta_user'
    // ...
);

// config/audit.php

return [
    'dump' => [
        'default' => [
            'date' => 'c'
            'strings' => [
                'custom_key' => 'my_value'
            ],
            'keys' => [
                'event_type',
                'message',
                'record_id',
                'record_type',
                'record_provider_id',
                'record_reference_key',
                'record_reference_value'
            ],
        ],
        'okta_user' => [
            'date' => 'c',
            'strings' => [
                'custom_key' => 'my_value'
            ],
            'keys' => [
                'event_type',
                'message',
                'attribute_key',
                'attribute_value_old',
                'attribute_value_new',
                'record_id',
                'record_type',
                'record_provider_id',
                'record_reference_key',
                'record_reference_value'
            ]
        ]
    ]
];

Real World Example for Response Arrays

use Provisionesta\Audit\Log;

$result = Log::create(
    attribute_key: $attribute,
    attribute_value_old: $manifest_record[$attribute],
    attribute_value_new: $api_record[$attribute],
    duration_ms: $this->duration_ms,
    event_type: $this->event_type . '.datadumper.manifest.attribute.changed.' . $attribute,
    level: 'info',
    log: true
    message: Str::title($attribute) . ' Attribute Value Changed',
    method: __METHOD__,
    record_provider_id: $manifest_record['provider_id'],
    record_reference_key: $this->reference_key,
    record_reference_value: $manifest_record[$this->reference_key] ?? null,
    transaction: true
);

dd($result);

// [
//     'datetime' => 'YYYY-MM-DDTHH:II:SS+00:00',
//     'event_type' => 'okta.user.sync.datadumper.manifest.attribute.changed.manager',
//     'message' => 'Manager Attribute Value Changed',
//     'attribute_key' => 'manager',
//     'attribute_value_old' => 'klibby',
//     'attribute_value_new' => 'dmurphy',
//     'record_id' => null,
//     'record_type' => 'okta_user',
//     'record_provider_id' => '00u1b2c3d4e5f6g7h8i9',
//     'record_reference_key' => 'handle',
//     'record_reference_value' => 'jpardella'
//     'custom_key' => 'my_value',
// ]

provisionesta / audit

Maintainers

Details

README

Overview

Problem Statement

Issue Tracking and Bug Reports

Contributing

Maintainers

Contributor Credit

Installation

Requirements

Upgrade Guide

Add Composer Package

Publish the configuration file

Usage Examples

Basic Usage

Comprehensive Usage

Example Output

Log Parameter Definitions

Advanced Usage

Background Job Log Entry

Skipping Log Creation

Response Schema

Example Response Array with No Configuration

Specifying Keys in Response Array

Specifying Custom Static Strings in Response Array

Custom Datetime Format in Response Array

Standardized Configurations for Response Array

Real World Example for Response Arrays