oneofftech/parse-client

Parse PDF document keeping the structure.

dev-main 2024-10-14 08:47 UTC

Requires

• php: ^8.2
• saloonphp/saloon: ^3.10

Requires (Dev)

• jonpurvis/lawman: ^1.2
• laravel/pint: ^1.0
• pestphp/pest: ^2.20

MIT

• OneOffTech

• pdf
• parse
• parsing
• text-extract

README

Parse client is a library to interact with OneOffTech Parse service. OneOffTech Parse is designed to extract text from PDF files preserving the structure of the document to improve interaction with Large Language Models (LLMs).

OneOffTech Parse is based on PDF Text extractor. The client is suitable to connect to self-hosted versions of the PDF Text extractor.

Note

The Parse client package is under development and is not ready for production use.

Installation

You can install the package via Composer:

composer require oneofftech/parse-client

Usage

The Parse client is able to connect to self-hosted instances of the PDF Text extractor service or the cloud hosted OneOffTech Parse service.

Use with self-hosted instance

Before proceeding a running instance of the PDF Text extractor is required. Once you have a running instance, you can instantiate the connector by passing the url that the extractor service is listening on.

use OneOffTech\Parse\Client\Connectors\ParseConnector;

$client = new ParseConnector(baseUrl: "http://localhost:5000");

/** @var \OneOffTech\Parse\Client\Dto\DocumentDto */
$document = $client->parse("https://domain.internal/document.pdf");

Note

• The URL of the document must be accessible without authentication.
• Documents are downloaded for the time of processing and then the file is immediately deleted.

Use the cloud hosted service

Important

The cloud hosted service is currently in private beta. Drop us a message.

Go to parse.oneofftech.de and obtain an access token. Instantiate the client and provide a URL of a PDF document.

use OneOffTech\Parse\Client\Connectors\ParseConnector;

$client = new ParseConnector("token");

/** @var \OneOffTech\Parse\Client\Dto\DocumentDto */
$document = $client->parse("https://domain.internal/document.pdf");

Note

• The URL of the document must be accessible without authentication.
• Documents are downloaded for the time of processing and then the file is immediately deleted.

Specify the preferred extraction method

Parse service supports different processors, pymupdf or pdfact. You can specify the preferred processor for each request.

use OneOffTech\Parse\Client\ParseOption;
use OneOffTech\Parse\Client\DocumentProcessor;
use OneOffTech\Parse\Client\Connectors\ParseConnector;

$client = new ParseConnector("token");

/** @var \OneOffTech\Parse\Client\Dto\DocumentDto */
$document = $client->parse(
    url: "https://domain.internal/document.pdf", 
    options: new ParseOption(DocumentProcessor::PYMUPDF)
);

PDFAct vs PyMuPDF

PDFAct offers more flexibility than PyMuPDF. You should evaluate the extraction method best suitable for your application. Here is a small comparison of the two methods.

Document structure

Parse is designed to preserve the document's structure hence the content is returned in a hierarchical fashion.

Document
 ├─Page
 │  ├─Text (category: heading)
 │  └─Text (category: body)
 └─Page
    ├─Text (category: heading)
    └─Text (category: body)

For a more in-depth explanation of the structure see Parse Document Model.

Testing

Parse client is tested using PEST. Tests run for each commit and pull request.

To execute the test suite run:

composer test

Changelog

Please see CHANGELOG for more information on what has changed recently.

Contributing

Thank you for considering contributing to the Parse client! The contribution guide can be found in the CONTRIBUTING.md file.

Security Vulnerabilities

Please review our security policy on how to report security vulnerabilities.

Credits

• OneOffTech
• All Contributors

Supporters

The project is provided and supported by OneOff-Tech (UG).

License

The MIT License (MIT). Please see License File for more information.