Feature: select decision services through Ai and register them from configuration

[jimador]

This PR selects, binds and registers decision services. Workflow code selects a service through Ai.decisions() or Ai.classifications() by family default, registration name, role or a supplied instance, and asks it a whole spec. Inside a workflow using(service) wraps a service in ObservedDecisionService. Spring configuration registers several Jev and prompted services with family defaults and roles. Plain Java builds the same registry by hand. Running a spec against one service, the four service families and their telemetry come from #2080.

Selecting a service and asking

ai.decisions() returns a ServiceSelector<DecisionService>. Each terminal returns the service itself, so ask follows directly. In the support-triage example URGENT and DEPARTMENT are named questions and TRIAGE is DecisionSpec.of(URGENT, DEPARTMENT):

DecisionService byRole = ai.decisions().byRole("support-triage");
DecisionService byName = ai.decisions().named("triage-stub");
DecisionService supplied = ai.decisions().using(stub);

DecisionResponse response = byRole.ask(TICKET, TRIAGE);
PropositionResult urgent = response.answer(URGENT);
ClassificationResult department = response.answer(DEPARTMENT);

defaultService() is the fourth terminal and returns the family default.

classifications() selects from every registered service, because a decision service also classifies. DEPARTMENTS is a ClassificationSpec, shown under Classification only:

ClassificationService router = context.ai().classifications().byRole("routing");
ClassificationResult result = router.classify(TICKET, DEPARTMENTS);

Inside a workflow operation each terminal returns a service bound to that operation. Its provider calls run under the operation's observation, including calls made from another thread, and the ask span is a child of the action span. A name, role or default that the registry cannot resolve throws ServiceSelectionException with a reason and a message that lists the family's services, roles and default and says how to fix the selection.

The operation binding forwards capabilities() and every hook, adds no kind, and implements DelegatingDecisionService. A decorator stacked on the binding therefore routes the way the provider does.

Reading typed answers

Named questions give typed results. Each result keeps answered, inconclusive and failed apart, and a choice also keeps no-match. A Spring Converter maps a response to an application route:

record SupportRoute(String queue, boolean sameDay, Double confidence) {
}

static final class SupportRouteConverter implements Converter<DecisionResponse, SupportRoute> {

    @Override
    public SupportRoute convert(DecisionResponse response) {
        boolean sameDay = switch (response.answer(URGENT)) {
            case PropositionResult.Answered answered -> answered.getAnswer();
            // Without an answer, a person looks at the ticket the same day.
            case PropositionResult.Inconclusive inconclusive -> true;
            case PropositionResult.Failure failure -> true;
        };
        // Confidence is carried for display when the provider reports it. Routing does not read it.
        return switch (response.answer(DEPARTMENT)) {
            case ClassificationResult.Selected selected ->
                new SupportRoute(selected.getCategoryId(), sameDay, selected.getConfidence());
            case ClassificationResult.NoMatch noMatch -> new SupportRoute("general", sameDay, null);
            case ClassificationResult.Inconclusive inconclusive -> new SupportRoute("human-review", sameDay, null);
            case ClassificationResult.Failure failure -> new SupportRoute("retry-later", sameDay, null);
        };
    }
}

A request that failed as a whole carries its reason, and every answer is then a failure with that reason:

FailureReason requestFailure = response.getRequestFailure();   // null when the request did not fail

Evidence is present only when the provider reports it:

if (response.answer(frustration) instanceof RatingResult.Answered rated) {
    RatingScore score = rated.getScore();                     // Jev: expected level index
    List<LevelProbability> distribution = rated.getDistribution();
    Double confidence = rated.getConfidence();
    String level = rated.getSelectedLevelId();                // prompted LLM: the selected level
}

The application stores the response JSON and each answer's provenance:

record AnswerProvenance(String question, String outcome, String modelName, String provider) {
}

record TriageEvidence(
    String responseJson,
    List<AnswerProvenance> answers) {
}

private final JsonMapper mapper = JsonMapper.builder().build();

private final List<TriageEvidence> evidenceStore = new ArrayList<>();

TriageEvidence persist(DecisionResponse response) {
    List<AnswerProvenance> answers = response.getAnswers().stream()
        .map(SupportTriageExampleTest::provenanceOf)
        .toList();
    TriageEvidence evidence = new TriageEvidence(
        mapper.writeValueAsString(response),
        answers);
    evidenceStore.add(evidence);
    return evidence;
}

static AnswerProvenance provenanceOf(DecisionAnswer answer) {
    Object outcome = switch (answer) {
        case DecisionAnswer.Proposition proposition -> proposition.getOutcome();
        case DecisionAnswer.Choice choice -> choice.getOutcome();
        case DecisionAnswer.Rating rating -> rating.getOutcome();
    };
    ModelProvenance model = switch (outcome) {
        case PropositionResult.Answered answered -> answered.getProvenance();
        case PropositionResult.Inconclusive inconclusive -> inconclusive.getProvenance();
        case ClassificationResult.Selected selected -> selected.getProvenance();
        case ClassificationResult.NoMatch noMatch -> noMatch.getProvenance();
        case ClassificationResult.Inconclusive inconclusive -> inconclusive.getProvenance();
        case RatingResult.Answered answered -> answered.getProvenance();
        case RatingResult.Inconclusive inconclusive -> inconclusive.getProvenance();
        // A failure outcome has no model provenance.
        default -> null;
    };
    return new AnswerProvenance(
        answer.getName(),
        outcome.getClass().getSimpleName(),
        model == null ? null : model.getModelName(),
        model == null ? null : model.getProvider());
}

The stored JSON holds no input and reads back to an equal response.

Grounding evidence

The grounding example asks whether a source excerpt supports a claim through the role dice-grounding. An application policy reads the answer kind and its own source rule. It records pTrue with the provenance and a policy version, and it writes the grounding link only for SUPPORTED:

static final PropositionQuestionSpec SUPPORTED =
    Questions.named("supported").proposition("Does the source excerpt support the claim?").build();

static final DecisionSpec GROUNDING = DecisionSpec.of(SUPPORTED);

static final String POLICY_VERSION = "grounding-policy-3";

enum Disposition { SUPPORTED, NOT_SUPPORTED, STALE_SOURCE, INCONCLUSIVE, FAILED }

// The policy reads the answer kind and its own source rule. The probability is recorded and not read.
static Disposition decide(PropositionResult outcome, GroundingFacts facts, String currentSourceRevision) {
    boolean sourceCurrent = facts.sourceRevision().equals(currentSourceRevision);
    return switch (outcome) {
        case PropositionResult.Answered answered when !sourceCurrent -> Disposition.STALE_SOURCE;
        case PropositionResult.Answered answered ->
            answered.getAnswer() ? Disposition.SUPPORTED : Disposition.NOT_SUPPORTED;
        case PropositionResult.Inconclusive inconclusive -> Disposition.INCONCLUSIVE;
        case PropositionResult.Failure failure -> Disposition.FAILED;
    };
}

The revision example asks one choice per scoped candidate and one evidence-strength rating in a single spec, and rechecks candidate versions before it applies a proposal. The application applies each proposal itself.

Kotlin

The DSL builds the spec, and the application handles every outcome:

data class Route(val queue: String, val sameDay: Boolean, val frustrationConfidence: Double?)

val triage = decisionSpec {
    proposition("urgent") { asking("Does the customer need help today?") }
    choice("department") {
        asking("Which team should handle the ticket?")
        option("billing", "Payments, invoices and refunds")
        option("technical", "Errors, outages and integrations")
    }
    rating("frustration") {
        asking("How frustrated is the customer?")
        level("calm")
        level("frustrated")
        level("very-angry")
    }
}

fun route(response: DecisionResponse): Route {
    val urgent = response.answer("urgent") as DecisionAnswer.Proposition
    val department = response.answer("department") as DecisionAnswer.Choice
    val frustration = response.answer("frustration") as DecisionAnswer.Rating
    val sameDay = when (val outcome = urgent.outcome) {
        is PropositionResult.Answered -> outcome.answer
        is PropositionResult.Inconclusive, is PropositionResult.Failure -> true
    }
    val queue = when (val outcome = department.outcome) {
        is ClassificationResult.Selected -> outcome.categoryId
        is ClassificationResult.NoMatch -> "general"
        is ClassificationResult.Inconclusive -> "human-review"
        is ClassificationResult.Failure -> "retry-later"
    }
    // Confidence is optional. It is shown to the agent and does not change the route.
    val confidence = (frustration.outcome as? RatingResult.Answered)?.confidence
    return Route(queue, sameDay, confidence)
}

fun triageTicket(ai: Ai, ticket: String): Route =
    route(ai.decisions().byRole("support-triage").ask(ticket, triage))

Execution is blocking and needs no coroutine scope.

Classification only

An application that only sorts text into categories needs Ai.classifications() and a ClassificationSpec. Select the service by role, as the family default, by registration name, or pass an instance:

ClassificationService byRole = ai.classifications().byRole("ticket-routing");
ClassificationService byDefault = ai.classifications().defaultService();
ClassificationService byName = ai.classifications().named("ticket-classifier");
ClassificationService supplied = ai.classifications().using(new KeywordClassifier());

Describe the categories once:

static final ClassificationSpec DEPARTMENTS = ClassificationSpec.builder()
    .asking("Which team should handle this?")
    .category("billing", "Payments, invoicing, refunds")
    .category("technical", "Bugs, outages, integrations")
    .build();

Classify with the spec and switch over the four outcomes:

static String queueFor(Ai ai, String ticketText) {
    ClassificationService classifier = ai.classifications().byRole("ticket-routing");
    return switch (classifier.classify(ticketText, DEPARTMENTS)) {
        case ClassificationResult.Selected selected -> selected.getCategoryId();
        case ClassificationResult.NoMatch noMatch -> "general";
        case ClassificationResult.Inconclusive inconclusive -> "triage";
        case ClassificationResult.Failure failure -> "triage";
    };
}

CategoryMapping.fromEnum builds the spec from an enum, and map turns a selection into the constant:

enum Department { BILLING, TECHNICAL }

static final CategoryMapping<Department> DEPARTMENT_MAPPING = CategoryMapping.fromEnum(
    Department.class,
    "Which team should handle this?",
    department -> switch (department) {
        case BILLING -> "Payments, invoicing, refunds";
        case TECHNICAL -> "Bugs, outages, integrations";
    });

static Optional<Department> departmentFor(Ai ai, String ticketText) {
    ClassificationService classifier = ai.classifications().byRole("ticket-routing");
    ClassificationResult result = classifier.classify(ticketText, DEPARTMENT_MAPPING.spec());
    return switch (DEPARTMENT_MAPPING.map(result)) {
        case MappedClassificationResult.Selected<Department> selected -> Optional.of(selected.getValue());
        case ClassificationResult.NoMatch noMatch -> Optional.empty();
        case ClassificationResult.Inconclusive inconclusive -> Optional.empty();
        case ClassificationResult.Failure failure -> Optional.empty();
    };
}

In a Spring application the spec can be a bean:

@Configuration(proxyBeanMethods = false)
static class TicketRoutingConfiguration {

    @Bean
    ClassificationSpec departments() {
        return ClassificationSpec.builder()
                .asking("Which team should handle this?")
                .category("billing", "Payments, invoicing, refunds")
                .category("technical", "Bugs, outages, integrations")
                .build();
    }
}

A prompted service that only classifies sets kind: classification on its entry and binds to a classification role. The relevant entries from decision-services-example.yml:

embabel:
  agent:
    platform:
      decisions:
        llm:
          services:
            ticket-classifier:
              llm: gpt-4.1-mini
              kind: classification
  models:
    classification:
      roles:
        ticket-routing: ticket-classifier

That service is not a decision service, so a decision default or role cannot name it, and decisions().named("ticket-classifier") throws ServiceSelectionException with WRONG_CAPABILITY. A decision service answers a ClassificationSpec too, since the spec is a decision spec with one choice question. ClassificationOnlyExampleTest and ClassificationOnlyKotlinExampleTest cover the selectors, the four outcomes and the enum mapping, with the Kotlin spec built by classificationSpec { }.

Spring configuration

Two Jev services, a prompted decision service, a prompted classification-only service and roles in both families:

embabel:
  agent:
    platform:
      models:
        typesafe:
          api-key: ${TYPESAFE_API_KEY}
          services:
            jev:
              model: jev-latest
            jev-fast:
              model: jev-fast
      decisions:
        llm:
          services:
            llm-review:
              llm: gpt-4.1-mini
            ticket-classifier:
              llm: gpt-4.1-mini
              kind: classification
        capture-content: false
  models:
    decision:
      roles:
        support-triage: jev
        dice-revision-review: llm-review
    classification:
      roles:
        dice-revision: llm-review
        ticket-routing: ticket-classifier

The entry key under services is the bean name and the registration name. A service's own name is its model name. Named Jev services share the credential, base URL and response limit under embabel.agent.platform.models.typesafe, and the default typeSafeDecisionService bean stays registered next to them.

With no explicit default, the TypeSafe auto-configuration offers typeSafeDecisionService as the default candidate, so Jev is the default of both families when TypeSafe is configured. Prompted services contribute no candidate. A default resolves to the explicit default, then the single eligible candidate, then the single eligible service. Zero or several of each throw NO_DEFAULT or AMBIGUOUS_DEFAULT at lookup.

Application beans take precedence over configuration:

Application bean	Effect
A DecisionServiceRegistry bean	Replaces the auto-configured registry
A bean named like a Jev or prompted entry	Replaces that entry only. One INFO line names the skipped entry.
A bean named typeSafeDecisionService	Replaces the default Jev service, which then contributes no default candidate. Named Jev services under services still register. With none configured, the TypeSafe auto-configuration does not run and needs no credential.

Startup fails when a default or role names a missing service, when a decision default or role names a classification-only service, when a Jev entry has a blank model or an unknown key, when a Jev and a prompted entry share a key, when a configured key names an application bean that is not a decision or classification service, or when a service's capabilities() throws. Each error names the property or the registration.

Lazy and non-singleton service beans are not registered, and one INFO line names each. One instance under several bean names registers once, under its @Primary name.

The registry is an immutable snapshot taken when the decisionServiceRegistry bean is created. A service created later, such as a per-user service, goes through using(service).

Plain Java

Plain Java builds Jev, stub and no-op services and the registry:

// Every Jev service built by one factory shares its credential, transport and observations.
var jevFactory = new TypeSafeModelFactory(
        TypeSafeClientOptions.defaults(),
        () -> System.getenv("TYPESAFE_API_KEY"),
        observations);

// Plain Java cannot build a prompted LLM service: it needs the platform's LLM operations.
return DecisionServiceRegistry.builder()
        .register("typeSafeDecisionService", jevFactory.build())
        .register("jev", jevFactory.build("jev-latest"))
        .register("jev-fast", jevFactory.build("jev-fast"))
        .register("triage-stub", StubDecisionService.builder("triage-stub").build())
        .register("disabled", new NoOpDecisionService("disabled"))
        .defaultCandidate("typeSafeDecisionService")
        .decisionRole("support-triage", "jev")
        .decisionRole("tests", "triage-stub")
        .classificationRole("dice-revision", "jev")
        // Pass the observation registry the services observe with.
        .observationRegistry(observations)
        .build();

An application with a Spring context can register services from the LlmDecisionServiceFactory bean in a registry it builds.

Diagnosing

build() logs one INFO summary on DecisionServiceRegistry that maps each registration name to its service name, provider, type and capabilities, with the family defaults, roles and candidates. For the YAML above, with two capability lists shortened:

Decision service registry built with 5 services: [typeSafeDecisionService (name jev-latest, provider TypeSafe, type DECISION, capabilities kinds [PROPOSITION, CHOICE, RATING], max questions none, max input characters none), jev (name jev-latest, provider TypeSafe, type DECISION, capabilities ...), jev-fast (name jev-fast, provider TypeSafe, type DECISION, capabilities ...), llm-review (name gpt-4.1-mini, provider OpenAI, type DECISION, capabilities kinds [PROPOSITION, CHOICE, RATING], max questions none, max input characters none), ticket-classifier (name gpt-4.1-mini, provider OpenAI, type CLASSIFICATION)]; decision default: 'typeSafeDecisionService' (default candidate); decision roles: {support-triage=jev, dice-revision-review=llm-review}; classification default: 'typeSafeDecisionService' (default candidate); classification roles: {dice-revision=llm-review, ticket-routing=ticket-classifier}; default candidates: [typeSafeDecisionService]

An unresolved selection throws ServiceSelectionException, described above. Execution, provider and assembler log lines are described in #2080.

embabel.agent.platform.decisions.capture-content: true turns on DecisionContentCapture while the context runs. Spring then logs one WARN at startup on DecisionServiceRegistryConfiguration. Captured lines hold the input and provider output.

Selection and binding

flowchart TD
    Ai["context.ai().decisions() / classifications()"] --> Sel[operation-bound selector]
    Sel -->|"named / byRole / defaultService"| Reg[DecisionServiceRegistry]
    Sel -->|"using(service)"| Wrap["ObservedDecisionService, unless already observed"]
    Reg -->|unresolved| SSE[ServiceSelectionException]
    Reg -->|resolved| Bind[operation binding]
    Wrap --> Bind
    Bind --> Ask["ask: preflight and execution (execution PR)"]

Loading

Configuration

Property	Default	Purpose
embabel.models.decision.default	none	Registration name of the decision family default
embabel.models.decision.roles.<role>	none	Registration name bound to a decision role
embabel.models.classification.default	none	Registration name of the classification family default
embabel.models.classification.roles.<role>	none	Registration name bound to a classification role
embabel.agent.platform.models.typesafe.services.<name>.model	none	Registers a Jev service under <name> for the given model
embabel.agent.platform.decisions.llm.services.<name>.llm	none	Registers a prompted service under <name>, with the entry keys from #2074
embabel.agent.platform.decisions.capture-content	false	Turns on TRACE content capture

Roles and defaults are scoped to one family. A role bound under embabel.models.decision.roles is unknown to classifications().byRole(...).

Compatibility

Every member added to Ai and PlatformServices has a default body, and no abstract member is added to an existing interface. ClassificationService, ModelProvider and the existing LLM and embedding selection are unchanged.

Experimental status

Every type added here carries @ApiStatus.Experimental. The owner is James Dunnam (@jimador). Promotion to stable needs resolution by role, name, default and instance used by a consumer application, the four families working through one registry, a compatibility review of Ai and PlatformServices, privacy checks on logs, tags and exception messages, a runnable consumer proof, and a recorded run against the hosted Jev service. Promotion is revisited at the next release review after the consumer proof.

Docs: the decision-execution reference page gains sections on selecting services, classification only, the registry, configuration, plain Java and the examples, with samples from the tagged tests. The TypeSafe page covers named services.

Stacked on #2080.

[igordayen]

@jimador - PR is very large, could you please try to split into smaller PRs. Thank you

[sonarqubecloud]

❌ The last analysis has failed.

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
90.6% Coverage on New Code
1.1% Duplication on New Code

See analysis details on SonarQube Cloud