README

A thin Laravel wrapper around the opendataloader-pdf CLI: hand it a PDF, get back Markdown — one string per physical page, blank pages preserved — via Laravel's own Process facade. No queue, no bindings to a specific PDF library, no state.

This package doesn't parse PDFs itself. It shells out to the CLI, asks for its --markdown-page-separator output so physical pages survive the round trip, and turns its exit codes and stderr into a small, catchable exception that tells you whether you (a config problem) or the file (unreadable, scanned, no text layer) is at fault.

Requirements
Installing the CLI
Package installation
Configuration
Quick start
Building an upload endpoint
Rate limiting
Error handling
Verifying the setup
Troubleshooting
Testing
License

Requirements

PHP 8.2 or newer
Laravel 10, 11, 12, or 13
The opendataloader-pdf CLI on the machine running your app (see below) — it is not a Composer dependency
Java 11+ on that same machine, since the CLI shells out to a bundled Java engine

Installing the CLI

The official quick-start is at opendataloader.org/docs/quick-start-python. The steps below are the same thing, adapted for a Ubuntu server running PHP-FPM — the two places people actually get stuck.

1. Java 11+.

sudo apt install openjdk-17-jdk
java -version

Ubuntu registers this with update-alternatives, so java lands on /usr/bin/java and is already on PATH for every user and service on the box. (Unlike, say, a Homebrew install on macOS, which deliberately keeps Java off PATH by default — set path in the config, or OPENDATALOADER_PDF_PATH in .env, to point at it there.)

2. The CLI itself, installed somewhere every user can reach. The upstream docs say pip install -U opendataloader-pdf, which works as-is on Ubuntu 22.04 and earlier. On Ubuntu 23.04+ (including 24.04 LTS), the system Python refuses a plain pip install outside a virtualenv ("externally-managed-environment", PEP 668) — install it as an isolated CLI application with pipx instead:

sudo apt install pipx
PIPX_HOME=/opt/pipx PIPX_BIN_DIR=/usr/local/bin pipx install opendataloader-pdf

PIPX_HOME/PIPX_BIN_DIR matter here: a bare pipx install puts everything under the installing user's own home directory (~/.local/...). That's fine as a normal deploy user, but if you run this as root — common on a freshly provisioned VPS — it lands under /root, which is chmod 700 by default and unreadable to every other account on the box, including PHP-FPM. Pointing both variables at shared, world-traversable locations instead (/opt/pipx for the venv, /usr/local/bin for the executable, both already on everyone's PATH) sidesteps that regardless of which user runs the install command.

Verify it landed where you expect:

ls -la /usr/local/bin/opendataloader-pdf

3. Confirm it as the user PHP-FPM actually runs as — not your login shell. PATH and installed packages can differ between an interactive SSH session and the service account:

ps aux | grep php-fpm | grep -v grep      # usually www-data on Ubuntu
sudo -u www-data /usr/local/bin/opendataloader-pdf --help

This must print the CLI's usage text, including --markdown-page-separator — this package relies on that flag to keep physical pages apart. If it's missing, update: pipx upgrade opendataloader-pdf. Once installed, the package's own opendataloader-pdf:check command automates this whole check.

Package installation

composer require muchwat/opendataloader-pdf-laravel

For local package development, use a path repository instead:

{
    "repositories": [
        {
            "type": "path",
            "url": "../opendataloader-pdf",
            "options": { "symlink": true }
        }
    ]
}

Laravel package discovery registers the service provider and OpendataloaderPdf facade automatically.

Configuration

Publish the config file:

php artisan vendor:publish --tag=opendataloader-pdf-config

Then set these in .env — everything is off until you do:

OPENDATALOADER_PDF_COMMAND=/usr/local/bin/opendataloader-pdf
# Only needed if the CLI's own internal call to `java` can't find it:
OPENDATALOADER_PDF_PATH=
OPENDATALOADER_PDF_TIMEOUT=120

OPENDATALOADER_PDF_COMMAND doubles as the on/off switch: it's empty by default, enabled() is true once it isn't, so a fresh install of a host application is never left silently trying to run an unconfigured feature.

Quick start

use Muchwat\OpendataloaderPdf\Exceptions\PdfExtractionException;
use Muchwat\OpendataloaderPdf\PdfExtractor;

$extractor = app(PdfExtractor::class); // or resolve it via constructor injection

try {
    $pages = $extractor->extractPages($pdfPath); // list<string>, one entry per physical page
    $markdown = $extractor->extractMarkdown($pdfPath); // same content joined with blank lines
} catch (PdfExtractionException $e) {
    // see "Error handling" below
}

The facade offers the same three methods:

use Muchwat\OpendataloaderPdf\Facades\OpendataloaderPdf;

if (OpendataloaderPdf::enabled()) {
    $pages = OpendataloaderPdf::extractPages($pdfPath);
}

extractPages() always returns at least one element. A blank physical page comes back as an empty string in its correct position rather than being dropped, so page count and page order both survive the round trip. Output from a CLI version that predates page markers still comes back correctly — as a single-element array — rather than throwing.

Building an upload endpoint

The package deliberately ships no controller or route — how a PDF reaches extractPages() is entirely up to the host application (an upload form, a queued job, an Artisan command). A typical HTTP endpoint looks like this:

use Illuminate\Http\Request;
use Muchwat\OpendataloaderPdf\Exceptions\PdfExtractionException;
use Muchwat\OpendataloaderPdf\PdfExtractor;

class PdfImportController
{
    public function status(PdfExtractor $extractor)
    {
        // Lets a frontend decide whether to show an "Import PDF" button at
        // all, without spawning a process just to check.
        return response()->json(['enabled' => $extractor->enabled()]);
    }

    public function extract(Request $request, PdfExtractor $extractor)
    {
        if (! $extractor->enabled()) {
            return response()->json(['status' => 'disabled']);
        }

        $request->validate([
            'attachment' => 'required|file|mimes:pdf|max:25600',
        ]);

        try {
            $pages = $extractor->extractPages($request->file('attachment')->getRealPath());

            return response()->json(['status' => 'ok', 'pages' => $pages]);
        } catch (PdfExtractionException $e) {
            $message = $e->isConfigurationIssue && ! $request->user()?->isAdmin()
                ? 'Automatic PDF import is not available right now. Please paste the text in manually.'
                : $e->getMessage();

            return response()->json(['status' => 'error', 'message' => $message]);
        }
    }
}

Route::get('pdf-extraction', [PdfImportController::class, 'status'])->middleware('auth');
Route::post('pdf-extraction', [PdfImportController::class, 'extract'])->middleware(['auth', 'throttle:pdf-extraction']);

Returning HTTP 200 with a status: 'error' body (rather than a 4xx/5xx) for anything past file validation is a deliberate choice worth keeping: a misconfigured server shouldn't turn into a hard failure for the person uploading — they can still paste the text in by hand.

Rate limiting

Each call spawns a CLI process — a JVM under the hood — so it is meaningfully more expensive than an ordinary request. If you expose extraction over HTTP, throttle it:

use Illuminate\Cache\RateLimiting\Limit;
use Illuminate\Support\Facades\RateLimiter;

RateLimiter::for('pdf-extraction', function ($request) {
    return Limit::perMinute(5)->by($request->user()->id);
});

Extraction runs synchronously within the request; there's nothing here that needs a queue worker, but do raise your web server's own request timeout to comfortably cover OPENDATALOADER_PDF_TIMEOUT if you increase it.

Error handling

PdfExtractionException::$isConfigurationIssue (readonly bool) tells you whether the server is misconfigured — disabled, CLI missing, Java unreachable, timeout too low — or the file is the problem — unreadable, scanned/image-only, malformed. Use it to decide how much detail is safe to show:

try {
    $pages = $extractor->extractPages($pdfPath);
} catch (PdfExtractionException $e) {
    report($e);

    $message = $e->isConfigurationIssue
        ? 'PDF import is temporarily unavailable.' // show an admin the real $e->getMessage() instead
        : $e->getMessage(); // safe to show anyone - it only describes this one file
}

A configuration problem is also logged via Laravel's Log facade (warning level) with the resolved command and, where relevant, stderr — so it's visible in storage/logs/laravel.log even for a user who only sees the generic message.

Verifying the setup

php artisan opendataloader-pdf:check

Confirms the configured command resolves and runs, that it can see a Java runtime, and that it supports --markdown-page-separator. Run it as the same user your app actually runs as in production:

sudo -u www-data php artisan opendataloader-pdf:check

Troubleshooting

"Unable to locate a Java Runtime"

java resolves fine in your own shell, but the extraction process still can't see it — typically a service manager with a hardened/restricted PATH= override. Confirm the directory with which java (usually /usr/bin on Ubuntu), then set:

OPENDATALOADER_PDF_PATH=/usr/bin

Re-run opendataloader-pdf:check to confirm.

Command not found, but it works in your terminal

You're most likely testing as a different user than the one running your app. Confirm PHP-FPM's actual user (ps aux | grep php-fpm) and re-check as that user specifically — see step 3 above. Prefer the CLI's full resolved path in OPENDATALOADER_PDF_COMMAND over a bare name in production.

Every extraction fails with a generic "could not run the command" error

If your deployment restricts PHP's disable_functions, make sure proc_open is not in that list — Laravel's Process facade needs it to start the subprocess at all. With proc_open disabled, every attempt fails the same way regardless of whether the CLI itself is installed correctly.

`pip install` fails with "externally-managed-environment"

Ubuntu 23.04+ (including 24.04 LTS). Use pipx instead — see step 2 above.

"No text could be extracted from this PDF"

The PDF has no text layer — usually a scanned document. This package can't do OCR; the file needs it done upstream before extraction.

Testing

composer test

Runs the package's own Pest suite against orchestra/testbench, with Process::fake() standing in for the real CLI — no Java or CLI install is required to run the tests.

License

MIT. See LICENSE.

muchwat / opendataloader-pdf-laravel

Maintainers

Package info

Statistics

Security

README

Contents

Requirements

Installing the CLI

Package installation

Configuration

Quick start

Building an upload endpoint

Rate limiting

Error handling

Verifying the setup

Troubleshooting

"Unable to locate a Java Runtime"

Command not found, but it works in your terminal

Every extraction fails with a generic "could not run the command" error

`pip install` fails with "externally-managed-environment"

"No text could be extracted from this PDF"

Testing

License

muchwat / opendataloader-pdf-laravel

Maintainers

Package info

Statistics

Security

README

Contents

Requirements

Installing the CLI

Package installation

Configuration

Quick start

Building an upload endpoint

Rate limiting

Error handling

Verifying the setup

Troubleshooting

"Unable to locate a Java Runtime"

Command not found, but it works in your terminal

Every extraction fails with a generic "could not run the command" error

pip install fails with "externally-managed-environment"

"No text could be extracted from this PDF"

Testing

License

`pip install` fails with "externally-managed-environment"