README

Fetch and cache files from HTTP, cloud storage, or any Laravel disk — safely, even under heavy concurrency.

The Problem

You have queue workers processing jobs that need the same files — images, documents, exports. Each worker fetches its own copy. Downloads duplicate. Disk fills up. Workers corrupt each other's writes. Pruning deletes a file another worker is reading.

Worker A ──fetch──► image.jpg ──write──► /cache/abc123   ✗ corrupt
Worker B ──fetch──► image.jpg ──write──► /cache/abc123   ✗ overwrite
Worker C ──────────read───────────────► /cache/abc123   ✗ partial data

Pruner   ──────────delete─────────────► /cache/abc123   ✗ gone mid-read

The Solution

File Stash gives every worker a safe, shared file cache with proper locking:

Worker A ──fetch──► image.jpg ──LOCK_EX──► /cache/abc123 ──unlock──► done
Worker B ────────── (waits) ──────────────LOCK_SH──► read ──► done
Worker C ────────── (waits) ──────────────LOCK_SH──► read ──► done

Pruner   ────────── (skips locked files) ────────────────────► prune

One download. Shared reads. No corruption. No race conditions.

When Is This Useful?

File Stash helps when your application needs to download a file and process it locally — not just move it between storages, but actually do something with the bytes:

Image processing pipelines — resize, watermark, or convert images from S3/CDN across multiple queue workers
PDF processing — extract text, merge, or convert documents from cloud storage
ML/AI workers — load model weights or input data from cloud storage, process locally
Video/audio processing — extract thumbnails, transcode, analyze media files
Data import jobs — parse CSV/Excel files from external sources in parallel workers
Email attachments — fetch, attach, and discard temporary files

In all these cases, Storage::get() returns file contents as a string (loaded into memory). File Stash gives you a local file path you can pass to any tool — FFmpeg, Imagick, Python scripts, shell commands — without holding the entire file in PHP memory.

How It Works

                          ┌───────────────────────────────┐
                          │        File Stash Cache       │
                          │   /storage/cache/files/       │
                          │                               │
  ┌─────────────┐  get()  │  ┌─────────┐   File exists?   │
  │  Worker 1   │────────►│  │ SHA-256 │──── yes ──► LOCK_SH ──► read ──► callback
  └─────────────┘         │  │  hash   │                  │
  ┌─────────────┐  get()  │  │         │──── no ───► fetch ──► LOCK_EX ──► write
  │  Worker 2   │────────►│  └─────────┘             │    │
  └─────────────┘         │                          │    │
  ┌─────────────┐  get()  │     Sources:             │    │
  │  Worker 3   │────────►│     • https://...   ◄────┘    │
  └─────────────┘         │     • s3://...                │
                          │     • local://...             │
  ┌─────────────┐ prune() │                               │
  │  Scheduler  │────────►│  File-level locks prevent     │
  └─────────────┘         │  deletion of files in use     │
                          └───────────────────────────────┘

Each file is identified by a SHA-256 hash of its URL. The first worker to request a file fetches and caches it; subsequent workers read the cached copy through a shared lock. File-level locks prevent pruning of files that are currently in use, and lifecycle locks coordinate destructive operations like clear().

Installation

composer require jackardios/laravel-file-stash

The service provider and FileStash facade are auto-discovered.

Publish the config (optional):

php artisan vendor:publish --provider="Jackardios\FileStash\FileStashServiceProvider" --tag="config"

Requirements: PHP ^8.1, Laravel ^10 / ^11 / ^12

Quick Start

Process a remote file locally

use FileStash;
use Jackardios\FileStash\GenericFile;

$file = new GenericFile('https://cdn.example.com/uploads/photo.jpg');

FileStash::get($file, function ($file, $cachedPath) {
    // $cachedPath is a real local path — pass it to any tool
    $image = Image::read($cachedPath);
    $image->resize(300, 200)->save(storage_path('thumbs/photo.jpg'));
});

Process a file from a Laravel storage disk

Any configured disk works — S3, GCS, SFTP, local. The prefix before :// is the disk name from config/filesystems.php:

// "reports" is the disk name from config/filesystems.php
$file = new GenericFile('reports://exports/sales-2024.xlsx');

FileStash::get($file, function ($file, $path) {
    // $path is a local file — pass it to any library that needs a file path
    $spreadsheet = IOFactory::load($path);
    $data = $spreadsheet->getActiveSheet()->toArray();
    // Process $data...
});

Note: reports:// refers to a Laravel disk named reports, not a URL protocol. Configure your disks in config/filesystems.php.

Batch processing

Process multiple files — each is held with a shared lock so nothing gets pruned or corrupted during the callback:

$attachments = Attachment::where('report_id', $reportId)->get();

$files = $attachments->map(
    fn ($a) => new GenericFile($a->storage_url)
)->all();

FileStash::batch($files, function ($files, $paths) {
    // All files are locked for reading — safe from pruning and deletion
    $zip = new ZipArchive();
    $zip->open(storage_path('app/export.zip'), ZipArchive::CREATE);
    foreach ($paths as $i => $path) {
        $zip->addFile($path, basename($files[$i]->getUrl()));
    }
    $zip->close();
});

One-time files (auto-cleanup)

Use getOnce() when you only need the file temporarily — it's deleted from cache after the callback:

$invoice = new GenericFile('https://billing.example.com/invoices/INV-2024-001.pdf');

FileStash::getOnce($invoice, function ($file, $path) {
    // Send email with attachment, then discard the cached file
    Mail::to('user@example.com')->send(new InvoiceMail($path));
});
// Cached file is automatically removed from disk

Custom File Implementations

GenericFile works for simple cases. For domain models, implement the File interface directly on your Eloquent model:

use Jackardios\FileStash\Contracts\File;

class Document extends Model implements File
{
    public function getUrl(): string
    {
        // Remote URL
        return $this->cdn_url;

        // Or a Laravel disk path
        // return "documents://{$this->path}";
    }
}

Then pass models directly:

$document = Document::find(1);

FileStash::get($document, function ($document, $path) {
    // $document is your Eloquent model — access any attribute
    $text = (new PdfParser())->parseFile($path)->getText();
    $document->update(['extracted_text' => $text]);
});

File Sources

The URL prefix determines where the file is fetched from:

URL format	Source	Example
`https://...` or `http://...`	Remote HTTP via Guzzle	`https://cdn.example.com/photo.jpg`
`diskname://path`	Laravel filesystem disk	`photos://uploads/photo.jpg`

The diskname must match a key in your config/filesystems.php disks array.

Local file paths like /var/files/image.jpg are not supported directly. Configure a local disk and use mydisk://image.jpg.

Concurrency Model

File Stash is designed for environments with multiple parallel queue workers processing the same files. Here's what happens under load:

Concurrent reads — safe

Worker A ─── get("img.jpg") ──► LOCK_SH ──► read ──► unlock
Worker B ─── get("img.jpg") ──► LOCK_SH ──► read ──► unlock   (parallel, no waiting)
Worker C ─── get("img.jpg") ──► LOCK_SH ──► read ──► unlock

Multiple workers can read the same cached file simultaneously. Shared locks (LOCK_SH) do not block each other.

Write during reads — safe

Worker A ─── get("new.jpg") ──► cache miss ──► LOCK_EX ──► download + write ──► unlock
Worker B ─── get("new.jpg") ──► cache miss ──► waits... ──────────────────────► LOCK_SH ──► read

If two workers request the same uncached file, one acquires the exclusive lock and fetches; the other waits, then reads the cached result.

Batch + prune — safe

Worker  ─── batch([a, b, c]) ──► LOCK_SH on each file ──► callback ──► unlock
Pruner  ─── prune()          ──► tries LOCK_EX on file ──► skipped (file is busy)

While your callback runs, each cached file is held with a shared lock (LOCK_SH). The pruner tries to acquire an exclusive lock (LOCK_EX) before deleting — if it can't, it skips the file. Your files are safe for the entire duration of the callback.

clear() goes further — it acquires an exclusive lifecycle lock, so it waits until all batch()/get() operations finish before deleting anything.

Lock configuration

// config/file-stash.php
'lock_max_attempts'      => 3,    // retries before giving up
'lock_wait_timeout'      => -1,   // seconds to wait (-1 = forever)
'lifecycle_lock_timeout' => 30,   // seconds for batch/clear coordination

For strict latency requirements, throw instead of waiting:

use Jackardios\FileStash\Exceptions\FileLockedException;

try {
    FileStash::get($file, $callback, throwOnLock: true);
} catch (FileLockedException) {
    // File is busy, handle gracefully
}

API Reference

Core Methods

// Cache and use a file
FileStash::get(File $file, ?callable $callback, bool $throwOnLock = false): mixed

// Cache, use, then delete
FileStash::getOnce(File $file, ?callable $callback, bool $throwOnLock = false): mixed

// Cache and use multiple files (prune-safe)
FileStash::batch(array $files, ?callable $callback, bool $throwOnLock = false): mixed

// Cache, use, then delete multiple files
FileStash::batchOnce(array $files, ?callable $callback, bool $throwOnLock = false): mixed

Cache Management

FileStash::exists(File $file): bool          // Check if a file's source exists
FileStash::forget(File $file): bool          // Remove a specific cached file
FileStash::prune(): array                     // Remove expired/oversized files
FileStash::clear(): void                      // Delete all unused cached files
FileStash::metrics(): CacheMetrics            // Get hit/miss/eviction counters

Pruning

Pruning removes cached files that are older than max_age or exceed the max_size limit.

Automatic pruning

The service provider registers a scheduled command that runs automatically. By default, it prunes every 5 minutes:

// config/file-stash.php
'prune_interval' => '*/5 * * * *',

Make sure Laravel's scheduler is running:

* * * * * cd /path-to-your-project && php artisan schedule:run >> /dev/null 2>&1

Manual pruning

Run the artisan command:

php artisan prune-file-stash

Or call it programmatically:

$stats = FileStash::prune();
// ['deleted' => 12, 'remaining' => 48, 'total_size' => 524288000, 'completed' => true]

Clearing all cache

FileStash::clear();           // Delete all unused cached files
FileStash::forget($file);     // Remove a specific file

The cache is also cleared automatically when you run php artisan cache:clear.

Events

Enable event dispatching for observability:

// config/file-stash.php
'events_enabled' => true,

Event	When	Key properties
`CacheHit`	File served from cache	`$file`, `$cachedPath`
`CacheMiss`	File not cached, fetching	`$file`, `$url`
`CacheFileRetrieved`	File successfully cached	`$file`, `$cachedPath`, `$bytes`, `$source`
`CacheFileEvicted`	File deleted from cache	`$path`, `$reason`
`CachePruneCompleted`	Prune finished	`$deleted`, `$remaining`, `$totalSize`, `$completed`

All events are in the Jackardios\FileStash\Events namespace.

When events_enabled is false (default), zero overhead — no objects allocated, no dispatching.

use Jackardios\FileStash\Events\CacheHit;

Event::listen(CacheHit::class, function (CacheHit $event) {
    Log::info("Cache hit: {$event->file->getUrl()}");
});

Metrics

Track cache effectiveness per-process:

$metrics = FileStash::metrics();

$metrics->hits;        // cache hits
$metrics->misses;      // cache misses
$metrics->retrievals;  // files fetched from source
$metrics->evictions;   // files removed
$metrics->errors;      // failed retrievals

$metrics->hitRate();   // float|null — percentage
$metrics->toArray();   // all counters as array
$metrics->reset();     // zero out counters

Push to your monitoring stack:

app()->terminating(function () {
    $m = FileStash::metrics()->toArray();
    // Send to Prometheus, StatsD, Datadog, etc.
});

Configuration

All settings support environment variables. Publish the config to customize:

php artisan vendor:publish --provider="Jackardios\FileStash\FileStashServiceProvider" --tag="config"

Cache limits

Key	Env	Default	Description
`path`	—	`storage/framework/cache/files`	Cache directory
`max_file_size`	`FILE_STASH_MAX_FILE_SIZE`	`-1` (unlimited)	Max file size in bytes
`max_age`	`FILE_STASH_MAX_AGE`	`60`	TTL in minutes before pruning
`max_size`	`FILE_STASH_MAX_SIZE`	`1E+9` (1 GB)	Soft limit for total cache size

HTTP

Key	Env	Default	Description
`timeout`	`FILE_STASH_TIMEOUT`	`-1`	Total request timeout (seconds)
`connect_timeout`	`FILE_STASH_CONNECT_TIMEOUT`	`30`	Connection timeout (seconds)
`read_timeout`	`FILE_STASH_READ_TIMEOUT`	`30`	Stream read timeout (seconds)
`http_retries`	`FILE_STASH_HTTP_RETRIES`	`0`	Retry attempts (4xx except 429 not retried)
`http_retry_delay`	`FILE_STASH_HTTP_RETRY_DELAY`	`100`	Base delay in ms (exponential backoff)
`user_agent`	`FILE_STASH_USER_AGENT`	`Laravel-FileStash/4.x`	User-Agent header
`max_redirects`	`FILE_STASH_MAX_REDIRECTS`	`5`	Max redirects to follow

Security

Key	Env	Default	Description
`allowed_hosts`	`FILE_STASH_ALLOWED_HOSTS`	`null` (all)	Host whitelist for SSRF protection
`mime_types`	—	`[]` (all)	Allowed MIME types

// Wildcards supported
'allowed_hosts' => ['example.com', '*.cdn.example.com'],

# Comma-separated in .env
FILE_STASH_ALLOWED_HOSTS=example.com,*.cdn.example.com

Concurrency

Key	Env	Default	Description
`lock_max_attempts`	`FILE_STASH_LOCK_MAX_ATTEMPTS`	`3`	Lock acquisition retries
`lock_wait_timeout`	`FILE_STASH_LOCK_WAIT_TIMEOUT`	`-1` (forever)	Lock wait timeout (seconds)
`lifecycle_lock_timeout`	`FILE_STASH_LIFECYCLE_LOCK_TIMEOUT`	`30`	Batch/clear coordination timeout
`batch_chunk_size`	`FILE_STASH_BATCH_CHUNK_SIZE`	`100`	Files per chunk (prevents fd exhaustion)

Pruning

Key	Env	Default	Description
`prune_interval`	`FILE_STASH_PRUNE_INTERVAL`	`/5 * * *`	Cron schedule for auto-pruning
`prune_timeout`	`FILE_STASH_PRUNE_TIMEOUT`	`300`	Prune timeout (seconds)

Performance

Key	Env	Default	Description
`touch_interval`	`FILE_STASH_TOUCH_INTERVAL`	`60`	Min seconds between `touch()` on hot files
`events_enabled`	`FILE_STASH_EVENTS_ENABLED`	`false`	Enable event dispatching

Exceptions

All exceptions are in Jackardios\FileStash\Exceptions with public readonly properties for structured handling:

Exception	When	Properties
`FileIsTooLargeException`	File exceeds `max_file_size`	`int $maxBytes`
`FileLockedException`	File locked, `throwOnLock` is `true`	—
`HostNotAllowedException`	Host not in `allowed_hosts`	`string $host`
`MimeTypeIsNotAllowedException`	MIME type not allowed	`string $mimeType`
`InvalidConfigurationException`	Invalid config value	`string $key`, `string $reason`
`SourceResourceIsInvalidException`	Invalid stream resource	—
`SourceResourceTimedOutException`	Stream read timed out	—
`FailedToRetrieveFileException`	All retries exhausted	—

use Jackardios\FileStash\Exceptions\HostNotAllowedException;

try {
    FileStash::get($file, $callback);
} catch (HostNotAllowedException $e) {
    Log::warning("Blocked: {$e->host}");
}

Testing

The facade ships with a fake that skips real HTTP/disk operations:

use FileStash;
use Jackardios\FileStash\GenericFile;

public function test_it_processes_file(): void
{
    FileStash::fake();

    $file = new GenericFile('https://example.com/image.jpg');

    $result = FileStash::get($file, fn ($file, $path) => 'processed');

    $this->assertEquals('processed', $result);
}

The fake supports all contract methods: get, getOnce, batch, batchOnce, forget, exists, prune, clear, metrics.

Acknowledgements

This package is based on biigle/laravel-file-cache by Martin Zurowietz. It extends the original with lifecycle locks, batch chunking, events, metrics, SSRF protection, HTTP retries, and other improvements.

License

MIT

jackardios / laravel-file-stash

Maintainers

Package info

Statistics

Security

README

The Problem

The Solution

When Is This Useful?

How It Works

Installation

Quick Start

Process a remote file locally

Process a file from a Laravel storage disk

Batch processing

One-time files (auto-cleanup)

Custom File Implementations

File Sources

Concurrency Model

Concurrent reads — safe

Write during reads — safe

Batch + prune — safe

Lock configuration

API Reference

Core Methods

Cache Management

Pruning

Automatic pruning

Manual pruning

Clearing all cache

Events

Metrics

Configuration

Cache limits

HTTP

Security

Concurrency

Pruning

Performance

Exceptions

Testing

Acknowledgements

License