README

Unofficial Symfony integration for TypeSafe AI's Jev model. Jev answers yes/no (noul), multiple-choice (choice) and rubric (score) questions about any text or JSON, and returns calibrated probabilities. This bundle is not affiliated with or endorsed by TypeSafe AI. "TypeSafe" and "Jev" are used only to name the API this package talks to.

What you get:

A typed client (JevClientInterface) built on Symfony HttpClient. It supports named clients, retries matching the official SDKs, strict response decoding and a full exception hierarchy.
#[JevNoul] and #[JevChoice] validator constraints, e.g. "reject comments that are probably spam".
Messenger evaluation in the background, with a DecisionMadeEvent for fan-out.
Workflow guards that allow or block a transition based on Jev's answer.
A web profiler panel listing every call with its questions, answers, probabilities, confidence, tokens, request ids, retries and timing.
Console commands jev:models and jev:ask.
PSR-3 logging with structured context and a metrics hook for Prometheus or OpenTelemetry. Neither is a hard dependency.
JevClientFake for your application tests. It makes no HTTP calls.

Requirements

Component	Requirement
PHP	8.4+
Symfony	7.4 or 8.x
Optional	`symfony/validator`, `symfony/messenger`, `symfony/workflow` (+ `symfony/property-access`), `symfony/web-profiler-bundle`

Install

composer require vbcherepanov/jev-symfony-bundle

With Symfony Flex, the recipe registers the bundle, creates config/packages/jev.yaml and adds TYPESAFE_API_KEY, TYPESAFE_BASE_URL and TYPESAFE_DEFAULT_MODEL to .env. Put the real key in .env.local or in Symfony secrets:

php bin/console secrets:set TYPESAFE_API_KEY
php bin/console jev:models

Without Flex, register the bundle in config/bundles.php:

return [
    // ...
    Jev\Bundle\JevBundle::class => ['all' => true],
];

Configuration

Every option with its default:

jev:
    api_key: '%env(default::TYPESAFE_API_KEY)%'  # checked when a request is sent, never printed
    base_url: 'https://api.typesafe.ai'
    model: 'jev-latest'          # jev-latest, jev-preview or a pinned id such as jev-1.13.0
    timeout: 10.0                # seconds per attempt
    default_client: default
    http_client: ~               # id of a base HttpClientInterface service; a dedicated one when null
    fake: false                  # true: every client becomes JevClientFake (test env)
    retry:
        max_retries: 2
        delay_ms: 500
        multiplier: 2.0
        max_delay_ms: 5000
        jitter: 0.25             # up to 25 % subtracted from each backoff delay
        max_retry_after_ms: 60000
    clients: {}                  # named clients, see below
    validator:
        fail_open: false         # when the API is unavailable: false adds a violation, true accepts
    metrics:
        service: ~               # service implementing Jev\Bundle\Observability\JevMetricsInterface
    workflow_guards: {}          # see "Workflow guards"

The API key is checked when a request is sent, not at container build time. cache:clear and CI builds therefore work without a key. An empty key, a key with a newline or other control character, or a key with surrounding whitespace raises InvalidConfigurationException. The message never contains the key.

Named clients

Named clients inherit every root setting they leave unset:

jev:
    api_key: '%env(TYPESAFE_API_KEY)%'
    clients:
        default: ~
        moderation:
            model: jev-preview
            timeout: 3
            retry: { max_retries: 0 }
        eu:
            base_url: '%env(TYPESAFE_EU_BASE_URL)%'
            api_key: '%env(TYPESAFE_EU_API_KEY)%'

Inject a client by argument name (JevClientInterface $moderationClient). An untargeted JevClientInterface receives default_client. The service ids are jev.client.<name>, and jev.client points to the default client.

Asking questions

use Jev\Bundle\Client\JevClientInterface;
use Jev\Bundle\Model\Evaluation;
use Jev\Bundle\Model\Question\{Choice, Noul, Score};

final readonly class TicketTriage
{
    public function __construct(private JevClientInterface $jev) {}

    public function triage(string $ticket): Triage
    {
        $decision = $this->jev->evaluate(new Evaluation($ticket, [
            'spam'       => Noul::ask('Is this message unsolicited advertising?'),
            'department' => Choice::fromEnum(Department::class, 'Which team should handle this ticket?'),
            'urgency'    => new Score(['Can wait', 'Needs attention this week', 'Needs attention today'], 'How urgent is it?'),
        ]));

        return new Triage(
            isSpam: $decision->noul('spam')->isYes(0.8),
            department: $decision->choice('department')->as(Department::class),
            urgency: $decision->score('urgency')->nearestLevel(),
            confident: $decision->choice('department')->confidence >= 0.7,
        );
    }
}

State can be a string, an array, a \stdClass or a \JsonSerializable. Instructions and criteria accept the same types, e.g. new Noul(['task' => '...', 'examples' => [...]]).
Noul: new Noul(instructions, true: 'what counts as yes', false: 'what counts as no'). You need instructions or at least one criterion.
Choice: new Choice(['billing' => 'Payments and invoices', 'other' => null], 'Which team?'), Choice::of(['a', 'b']) or Choice::fromEnum(MyEnum::class). Enum cases that implement DescribedOption supply their own description. Up to 255 options.
Score: an ordered list of 2 to 10 levels. Level N is the N-th entry, counting from 0.
Numeric ids and labels such as "0" and "1" are always sent as JSON objects. Plain json_encode() turns such PHP arrays into JSON lists, which the API rejects. To force an object for your own state, pass (object) [...].

Decision gives you model (the resolved id, e.g. jev-1.13.0 for jev-latest), usage (inputTokens, outputTokens), requestId (the x-typesafe-request-id header) and typed accessors. Accessors throw AnswerNotFoundException for a missing id and UnexpectedAnswerTypeException for the wrong type. Answer types this version doesn't know are kept as UnknownAnswer (raw payload) rather than failing the whole decision.

Answer	Fields and helpers
`NoulAnswer`	`probability` (of "yes"), `isYes(0.5)`, `isNo(0.5)`
`ChoiceAnswer`	`choice`, `probabilities`, `confidence`, `probability($label)`, `is(...)`, `as(Enum::class)`, `ranked()`
`ScoreAnswer`	`score` (probability-weighted, may fall between levels), `legend`, `probabilities`, `confidence`, `nearestLevel()`, `mostLikelyLevel()`, `criterion($level)`

Per-call overrides: $jev->evaluate($evaluation, new RequestOptions(timeout: 3.0, maxRetries: 0, headers: ['X-Correlation-Id' => $id])). To override the model for a single evaluation, pass it as the third Evaluation argument or use ->withModel('jev-preview').

$jev->models() returns list<ModelCard> (name, description, releaseDate).

Errors and retries

Every exception implements Jev\Bundle\Exception\JevException.

Exception	When
`ApiException` (base: `status`, `errorType`, `detail`, `requestId`, `retryAfter`)	any other non-2xx status
`BadRequestException`	400: semantically invalid request, e.g. `Unknown model: jev`
`AuthenticationException`	401 invalid key, 403 missing key
`NotFoundException`	404
`ValidationException` (`violations`: `list<Violation>`)	422 schema violations
`RateLimitException`	429
`OverloadedException`	529
`ServerException`	other 5xx
`TransportException`, `TimeoutException`	network failure, per-attempt timeout
`InvalidResponseException`	a 2xx body that doesn't match the contract (fields are never silently defaulted)
`InvalidEvaluationException`	a question or evaluation rejected locally before sending
`InvalidConfigurationException`	bad key, URL, client name or configuration

Retries follow the official SDKs. Statuses 408, 429 and 5xx (including 529), connection errors and timeouts are retried up to max_retries times. The server's retry-after-ms header wins, then Retry-After (seconds or an HTTP date), capped at max_retry_after_ms. Otherwise the bundle backs off exponentially with jitter. Each retry carries X-TypeSafe-Retry-Count. Statuses 400, 401, 403, 404 and 422 are never retried.

Validator constraints

use Jev\Bundle\Validator\{JevChoice, JevNoul};

final class CommentInput
{
    #[Assert\NotBlank]
    #[JevNoul(question: 'Is this comment spam or advertising?', max: 0.2)]
    #[JevChoice(question: 'What is the tone?', enum: Tone::class, allowed: [Tone::Friendly, Tone::Neutral], minConfidence: 0.6)]
    public string $body = '';
}

JevNoul adds a violation when P(yes) is above max or below min. Use max for "is this bad?" questions and min for "is this good?" questions. Options: true, false, client, model, failOpen, message, unavailableMessage. Message placeholders: {{ probability }}, {{ limit }}, {{ question }}.
JevChoice takes options (a list, or label => description) or enum. It adds a violation when the choice is not in allowed (an empty list allows every option) or its confidence is below minConfidence. Placeholders: {{ choice }}, {{ confidence }}, {{ allowed }}, {{ limit }}.
null and "" are skipped; combine with NotBlank when needed.
Fail-closed by default. When the API is down, rate-limited or answers badly, the value is rejected with unavailableMessage (code JevNoul::UNAVAILABLE_ERROR). Set failOpen: true on the constraint, or jev.validator.fail_open: true globally, to accept values instead. Configuration errors such as a missing key are always thrown.

Each constraint makes one API call. For several questions about the same value, call the client directly with one Evaluation.

Messenger

use Jev\Bundle\Messenger\EvaluateMessage;

$bus->dispatch(new EvaluateMessage($evaluation, client: 'moderation', correlationId: (string) $comment->getId()));

framework:
    messenger:
        routing:
            Jev\Bundle\Messenger\EvaluateMessage: async

The handler dispatches Jev\Bundle\Event\DecisionMadeEvent (message and decision) through the event dispatcher:

#[AsEventListener]
public function onDecision(DecisionMadeEvent $event): void
{
    if ($event->decision->noul('spam')->isYes(0.9)) {
        $this->comments->hide($event->message->correlationId);
    }
}

Errors that can't succeed later (400, 401/403, 404, 422, invalid evaluation or configuration) are wrapped in UnrecoverableMessageHandlingException, so Messenger doesn't retry them. Rate limits, overloads, 5xx and network errors go through Messenger's retry strategy after the client's own retries. Messages use Messenger's default PHP serializer.

Workflow guards

jev:
    workflow_guards:
        no_spam:
            workflow: comment
            transitions: [publish]
            state_property: body            # property path read with PropertyAccess
            noul:
                question: 'Is this comment spam?'
                max: 0.2                    # block when P(yes) > 0.2 (use min for "must be yes")
            message: 'Blocked as probable spam ({{ probability }}).'
        safe_to_ship:
            workflow: article
            transitions: [publish, feature]
            client: moderation
            choice:
                question: 'Classify this article'
                options: { safe: ~, nsfw: 'Sexual or graphic content', hateful: ~ }
                allowed: [safe]
                min_confidence: 0.8
            fail_open: false                # API unavailable: block (default) or allow

Instead of state_property, the subject can implement Jev\Bundle\Workflow\JevSubjectInterface (jevState(string $guard)). A blocked transition gets a TransitionBlocker with code jev_guard_blocked and parameters guard, reason, request_id, plus the verdict values. If the API is unavailable, the code is jev_guard_unavailable. Guards run on every Workflow::can() call, so results are cached per subject state until the end of the request or message. Rule mistakes, such as an allowed option that isn't in options, fail the container build.

Profiler

With WebProfilerBundle enabled, a Jev panel lists each call: client, requested and resolved model, outcome and HTTP status, request id, duration, attempts and retry reasons, tokens, a truncated state preview, and each question with its answer, confidence and probability distribution. The collector is only registered when the profiler service exists, and it never sees the API key.

Console

php bin/console jev:models
php bin/console jev:ask "BUY CHEAP WATCHES!!!" "Is this message spam?"
php bin/console jev:ask "I was charged twice" "Which team handles this?" -o billing -o shipping -o other
php bin/console jev:ask "Server is down" "How urgent is this?" -l "can wait" -l "this week" -l "right now"
echo "text" | php bin/console jev:ask - "Is this polite?" --client=moderation --model=jev-preview

Logging and metrics

Every call is logged on the jev Monolog channel: info on success, warning per retry, error on failure. The context holds client, operation, model, request_id, status, attempt(s), duration_ms, token counts and, for failures, outcome and error_class. The API key, state and questions are never logged.

For metrics, implement JevMetricsInterface (one counter and one histogram method) and set jev.metrics.service. You can also wrap two closures in CallbackMetrics:

use Jev\Bundle\Observability\JevMetricsInterface;
use Prometheus\CollectorRegistry;

final readonly class PrometheusJevMetrics implements JevMetricsInterface
{
    public function __construct(private CollectorRegistry $registry) {}

    public function counter(string $name, float $value, array $labels): void
    {
        $this->registry->getOrRegisterCounter('app', $name, $name, array_keys($labels))->incBy($value, array_values($labels));
    }

    public function histogram(string $name, float $value, array $labels): void
    {
        $this->registry->getOrRegisterHistogram('app', $name, $name, array_keys($labels))->observe($value, array_values($labels));
    }
}

Emitted series:

jev_requests_total{client,operation,outcome}
jev_request_duration_seconds{client,operation,outcome}
jev_retries_total{client,operation,reason}
jev_tokens_total{client,model,direction}

outcome is success, the HTTP status, timeout, transport_error, invalid_response or invalid_request. For anything more, implement Jev\Bundle\Observability\JevObserverInterface; services are autoconfigured and receive every retry and completed call.

Testing your application

The Flex recipe enables fake: true in the test environment. Every client then becomes one Jev\Bundle\Testing\JevClientFake, which makes no HTTP calls:

use Jev\Bundle\Exception\RateLimitException;
use Jev\Bundle\Model\Evaluation;
use Jev\Bundle\Testing\{FakeDecision, JevClientFake};

public function testSpamIsRejected(): void
{
    $client = static::createClient();
    $jev = static::getContainer()->get(JevClientFake::class);
    $jev->queue(FakeDecision::of(['rule' => FakeDecision::noul(0.97)]));    // #[JevNoul] asks under id "rule"

    $client->request('POST', '/comments', ['body' => 'BUY NOW']);

    self::assertResponseStatusCodeSame(422);
    $jev->assertEvaluatedTimes(1);
    $jev->assertEvaluated(fn (Evaluation $e): bool => $e->state === 'BUY NOW');
}

queue() accepts Decision objects, JevExceptions to throw, or closures fn (Evaluation $e): Decision. Queued items are used in order.
fallback() answers everything once the queue is empty.
withModels() feeds models().
FakeDecision::noul(), ::choice() and ::score() build answers.
Assertions: assertEvaluatedTimes(), assertNothingEvaluated(), assertEvaluated(), assertQueueEmpty(). They report through PHPUnit when it is present.
An empty queue throws a message telling you what to queue.

Validator constraints and workflow guards ask under the question id rule. jev:ask uses question.

Related projects

The official Python SDK and JavaScript SDK by TypeSafe AI. This bundle's retry defaults and error model follow them.
symfony/ai PR #2549 adds a TypeSafe bridge to the Symfony AI Platform. It exposes Jev as a generic AI platform model. This bundle is complementary: it provides Jev-specific typed questions and answers plus Symfony integrations (validator, workflow, Messenger, profiler) that a generic platform bridge doesn't cover.

Development

make install      # downloads tools/composer.phar if missing, then composer install
make verify       # composer validate, PHPUnit, PHPStan (max level), php-cs-fixer
TYPESAFE_API_KEY=... make test-live   # 4 small real API calls

CI runs the suite on PHP 8.4 and 8.5 with Symfony 7.4 and 8.0. The live smoke test is skipped when TYPESAFE_API_KEY is not set. The Flex recipe lives in recipe/, ready for symfony/recipes-contrib.

Support and license

Report bugs through GitHub Issues. Include PHP/Symfony versions, the request id from the exception or profiler, and a minimal example without your API key. See CHANGELOG.md. Licensed under Apache-2.0; see LICENSE.

vbcherepanov / jev-symfony-bundle

Package info

Security

Statistics