README

Prove that your hand-written PHP rules still agree with the Schematron they were transcribed from.

composer require --dev stboris/schematron-diff
vendor/bin/schematron-diff run en16931-ubl

ruleset:         EN 16931 UBL 1.3.16 (bundled example)
implementation:  josemmo/einvoicing v0.3.1, 27 EN 16931 base rules
scope:           BR-*
corpus:          en16931-broken (10 documents)

DOCUMENT                                        RESULT
------------------------------------------------------------------------------
br-01-no-specification                          agree (1)
br-16-no-lines                                  agree (4), 3 not implemented
br-co-17-vat-scaled-by-100                      agree (3), 3 not implemented
...

10 agreed, 0 disagreed

The problem

National e-invoicing rules ship as ISO Schematron, and Schematron needs XSLT 2.0. PHP's XSLTProcessor is libxslt, which is 1.0 only. So the rules get reimplemented in PHP by hand, one class at a time.

Hand transcription drifts. A regex that is subtly wrong, a rule you read as applying to invoices when it applies to credit notes too, a let binding thirty lines above the assertion that inverts the test. You will not find these by rereading your own code, because you wrote the bug by reading the file and you will reread it the same way.

You cannot run Schematron in PHP. But nothing says the comparison has to run where the validator runs. This tool runs the official Schematron under Saxon, captures SVRL, and diffs the failed-assert ids against what your PHP said about the same documents.

Java is needed once, in CI. Never on a user's machine, and never by the code being tested. A local java is used when there is one, otherwise a throwaway container.

Setup

Declare your rulesets in schematron-diff.json in your project root:

{
    "rulesets": {
        "hr-cius-2025": {
            "label": "HR CIUS/EXT 2025",
            "schematron": "research/schematron/HR-CIUS-EXT-EN16931-UBL.sch",
            "corpus": "research/fixtures",
            "implementation": "App\\Testing\\MyValidatorAdapter",
            "scope": ["HR-BR-"],
            "sha256": "ec80647ffab65bb7d0c91f96aaa30fd1f9b50d8b59f1bee96d01cfed54d4bcc2"
        }
    }
}

sha256 is optional and worth setting. A pinned ruleset that quietly changed under the rules written against it is the failure this tool exists to make impossible, so it is checked before anything is compiled.

Two hashes are worth recording, and they cover different bytes:

Key	Covers
`sha256`	the local file at `schematron`, which is what gets verified and run
`source_sha256`	the upstream artefact at `source`, usually an archive
`source_member`	which file inside that archive was taken

Only sha256 is enforced. The other two exist so somebody checking your provenance can reproduce the extraction instead of measuring the archive, getting a different number, and having to guess whether that is drift or a naming problem.

Then write an adapter for whatever you want to test:

use Stboris\SchematronDiff\Implementation;

final class MyValidatorAdapter implements Implementation
{
    public function brokenCodes(string $path): array
    {
        return $this->validator->validateFile($path)->brokenCodes();
    }

    /** Which ids you implement at all. Null if you cannot say. */
    public function implementedCodes(): ?array
    {
        return $this->validator->implementedCodes();
    }

    public function describe(): string { return 'my-package 1.2.3'; }
    public static function isAvailable(): bool { return true; }
    public static function make(): self { return new self(); }
}

Commands

schematron-diff list                       rulesets this project declares
schematron-diff run <ruleset> [corpus]     verify then compare
schematron-diff verify <ruleset> [corpus]  run the Schematron, write SVRL
schematron-diff compare <ruleset>          diff the reports against the PHP

--json for machine-readable output, --quiet to suppress progress.

Exit code is 1 on any disagreement, so it gates a release. A rule your implementation does not implement is reported separately and does not fail the run.

Three things that make the output worth reading

Scope. The EN 16931 UBL Schematron carries 979 assertions, of which only 223 are business rules. The other 756 are UBL-CR-*, UBL-SR-* and UBL-DT-* syntax restrictions that no business-rule implementation claims to cover. Unfiltered, every document reports hundreds of misses and the real result is invisible. scope declares which id prefixes the comparison is entitled to an opinion about.

A gap is not a disagreement.

	Meaning	Fails the run
false positive	your PHP flagged an id the Schematron did not	yes
missed	the Schematron flagged an id you implement and did not flag	yes
not implemented	the Schematron flagged an id you never implemented	no

Report the third the same way as the first two and a partially complete implementation looks catastrophically broken while the actual defects drown in the noise.

A corpus where everything passes proves nothing. Reference example documents are supposed to pass, so a diff over them is two implementations agreeing about nothing. You need documents that break, one rule at a time. The bundled resources/corpus/en16931-broken is generated by bin/make-en16931-fixtures.php and shows the shape.

Run both. Broken documents test whether you catch what the Schematron catches; clean documents test whether you invent failures that are not there. Neither answers the other.

The corpus is usable without this tool, and without PHP

A conformance corpus is documents plus the rule ids each one should produce, and neither of those is PHP. Both bundled corpora ship an expected.json next to the documents:

{
    "documents": {
        "br-co-17-vat-scaled-by-100.xml": {
            "targets": "BR-CO-17",
            "why": "VAT category tax amount must equal taxable amount times rate.",
            "expected": ["BR-CO-14", "BR-CO-17", "BR-S-09"]
        }
    }
}

Take the XML files and that manifest into a .NET, Java, Python, Delphi or Node test suite and run them with whatever you already have. Nothing here needs to be installed for the data to be useful.

Two things to know if you do:

expected is every id the Schematron reported, not only the ones you implement. Filter to the prefixes your implementation claims, and treat an id you do not implement as a coverage gap rather than a failure.

The expectations are measured, never transcribed. They are written by running the official Schematron and recording what came back:

schematron-diff snapshot en16931-ubl     # write expected.json from a real run
schematron-diff check en16931-ubl        # fail if it has drifted

An expectation somebody typed in from reading the specification is exactly the artifact this project exists to distrust, so the corpus is not allowed to contain any. check runs in CI here, which means a published expectation that has quietly stopped matching the Schematron turns the build red rather than misleading you silently.

Rulesets move, so pin more than the bytes

A ruleset is a moving target. sha256 tells you the file changed; it does not tell you whether anything you care about changed with it, and a release note is not the same as knowing.

So a ruleset can pin several versions, and a corpus can record which documents get a different verdict from each:

schematron-diff list                                   # shows pinned versions
schematron-diff run en16931-ubl@1.3.15                 # any version, explicitly
schematron-diff flips en16931-ubl 1.3.15 1.3.16        # record verdict changes
schematron-diff check-flips en16931-ubl 1.3.15 1.3.16  # fail if they moved

1 document(s) change verdict between 1.3.15 and 1.3.16:
  br-co-25-payable-without-due-date.xml
    1.3.15     BR-CO-25
    1.3.16     (clean)

  stopped firing in 1.3.16: BR-CO-25

BR-CO-25 is the clean case, and the bundled corpus carries it. A positive amount due for payment with no due date and no payment terms is fatal under EN 16931 UBL 1.3.15 and valid under 1.3.16, because the rule was removed. Anyone who implemented it from the 2017 standard text is now rejecting compliant invoices, and no amount of hashing the .sch would have told them.

Versions share everything outside the versions block, so only the artefact and its hashes are repeated. The default version keeps the plain expected.json; other versions write expected-<version>.json, so pinning a second version never renames a file somebody else is already consuming.

Thanks to Tobias for the flip-fixture idea, and to PatrickPi1312 for an independent cross-check of the corpus and for catching that the manifest did not say which bytes each hash covered.

What agreement does not mean

Agreeing with the Schematron is not the same as being correct against the standard, but not for the reason it is tempting to reach for. BR-CO-25 shows why: it is not missing from 1.3.16 by oversight, it was deliberately removed, and an implementation that still enforces it is wrong about today's rules rather than ahead of them. Diffing against a pinned artefact is not a weaker substitute for checking against the standard's text. It is the thing you actually want, because the artefact moves and the text does not.

BR-CO-27 is a different case again. It is defined in the text of EN 16931-1:2017+A1:2019 and is absent from every UBL and CII binding from 1.3.13 through 1.3.16, so it was never removed - it was never bound. A rule missing from both the Schematron and your PHP cannot appear as a disagreement: this tool will report perfect agreement and both of you will be silent about it in exactly the same way.

The claim it supports is narrow and worth stating precisely: your transcription matches the artifact you transcribed it from. That is a real claim and worth being able to make. It is not your validator makes this document valid, and no amount of green CI turns one into the other.

The bundled example

en16931-ubl runs against EN 16931 UBL 1.3.16 with josemmo/einvoicing as the implementation, so there is something to run before you have configured anything. The adapter under src/Implementations/ is a worked example of adapting a third-party library, including the awkward parts - that one throws on the first failing rule, so the adapter runs the rule callables itself to collect them all.

Nothing in the package depends on it. Delete the file and the tool still works.

Licence

MIT. The bundled EN 16931 Schematron is EUPL 1.2, published by the European Commission (ConnectingEurope/eInvoicing-EN16931), and is redistributed under its own terms rather than this package's.

stboris / schematron-diff

Maintainers

Package info

Statistics

Security