Feature: LLM-backed decision and classification services

[jimador]

Decision and classification services answered by a chat model, built on the contracts in #2062. Stacked on #2061, which adds the TypeSafe services for the same contracts.

LlmDecisionServiceFactory builds a DecisionService or a ClassificationService from a model name or from an LlmService the caller already holds. The platform autoconfiguration registers the factory as a bean. Services can also be declared in configuration, and the platform autoconfiguration registers each one as a DecisionService bean named after its key.

classDiagram
    ClassificationService <|-- DecisionService
    LlmDecisionServiceFactory --> DecisionService : decisionService(llm)
    LlmDecisionServiceFactory --> ClassificationService : classificationService(llm)
    LlmDecisionServicesAutoConfiguration --> LlmDecisionServiceFactory : bean
    LlmDecisionServicesAutoConfiguration --> LlmDecisionServicesRegistrar : registers
    LlmDecisionServicesRegistrar --> LlmDecisionServiceFactory : builds configured services
    DecisionService <|.. ObservedDecisionService
    ClassificationService <|.. ObservedClassificationService
    ObservedDecisionService --> LlmDecisionService
    ObservedClassificationService --> LlmClassificationService
    LlmClassificationService --> LlmDecisionService
    LlmDecisionService --> LlmOperations

Loading

flowchart LR
    App[Application] --> Observed[ObservedDecisionService]
    Observed --> Service[LlmDecisionService]
    Service --> Retry[Retry template]
    Retry --> Ops[LlmOperations]
    Ops --> Model[Chat model]

Loading

DecisionService triage = factory.decisionService("small-chat-model");
DecisionService byok = factory.decisionService(validatedLlm);

embabel:
  agent:
    platform:
      decisions:
        llm:
          max-attempts: 4
          services:
            routing:
              llm: fast-chat-model

Behavior:

• A named model is resolved once, when the service is built. Every call uses that exact model.
• A supplied LlmService is used as is, which covers BYOK: validate the key with the provider's BYOK factory and pass the result.
• The model answers with a structured verdict. SELECTED, NO_MATCH and INCONCLUSIVE map to Selected, NoMatch and Inconclusive. TRUE, FALSE and INCONCLUSIVE map to Answered(true), Answered(false) and Inconclusive. Results carry no confidence: the answer schema has no confidence field, and the prompt tells the model not to report one.
• A verdict that breaks the rules, such as an unknown category ID, is Failure(INVALID_RESPONSE). A reply that cannot be read is retried and then becomes Failure(INVALID_RESPONSE). A failed model call is retried and then becomes Failure(UNAVAILABLE). The defaults are 5 attempts counting the first, a 100 ms first wait, a multiplier of 5 and at most 180000 ms between attempts.
• An interruption during the model call or the retry wait throws an unchecked CancellationException caused by the InterruptedException, and leaves the thread's interrupt flag set. The observation records the interrupted outcome.
• Input text goes to the model as the input field of a JSON object, so empty input and input that looks like instructions reach it unchanged.
• The classification prompt puts the request's instructions on a Question: line before the categories.
• Each call records one embabel.ai.decision or embabel.ai.classification observation. The underlying model call keeps its own observation.
• The retry settings bind as @ConfigurationProperties under embabel.agent.platform.decisions.llm and apply to every service the factory builds. The factory takes them as a constructor argument. A setting out of range fails binding, and the cause names the setting.
• LlmDecisionServicesAutoConfiguration in embabel-agent-platform-autoconfigure supplies the factory bean, binds the retry settings and registers the configured services. The factory bean backs off when the application defines its own. Core has no Spring Boot wiring for decisions. An entry names its llm, nothing else. An unknown LLM name or an unknown key under a service in configuration files fails startup, and the error names the property. Keys set through environment variables or system properties are not checked.
• The factory bean is @ConditionalOnMissingBean. Configured services are built through whichever factory bean is present, so an application's own factory builds them too.

No checked exception escapes classify or assess, so Java callers never have to handle one. #2080 gives the TypeSafe services and ask the same convention.

The API is experimental. The factory's public surface is its four builder methods. The reference page, embabel-agent-docs/src/main/asciidoc/reference/llm-decisions/page.adoc, covers configuration, results, failures, interruption and observations.

[igordayen]

@jimador - major blocking concern is on LLmDecisionServiceConfiguration location.
thanks

[igordayen]

@jimador - please resolve merge conflicts.
thanks

[igordayen]

from CODEX:

Findings

• P1 LlmDecisionServiceConfiguration.kt:32 I agree with your blocking comment. LlmDecisionServiceConfiguration is in embabel-agent-api, but it is doing
  Boot-specific configuration work: ConditionalOnMissingBean, Boot Binder, NoUnboundElementsBindHandler, and a BeanDefinitionRegistryPostProcessor. This
  is not just API/SPI code; it is autoconfiguration-style wiring. Keeping it in API means the core platform configuration imports this feature directly
  via AgentPlatformConfiguration.kt:77, instead of letting an autoconfigure module own conditional registration. I’d move this to the appropriate
  autoconfigure/platform-autoconfigure module and leave embabel-agent-api with the factory/service implementation contracts only.

• P2 LlmDecisionService.kt:73 The implementation can throw InterruptedException through ClassificationService.classify / DecisionService.assess, but the
  Feature: implementation of classification and decision service contracts #2062 contracts do not declare it. The PR body calls this out, so it is known, but it is still an awkward Java API: callers cannot catch
  InterruptedException directly around the contract method. I would not ship this behavior until the contract is aligned, or use the same unchecked/
  cancellation convention as TypeSafe until the checked exception is added.

Your other comments make sense as cleanup once the config is moved: using EnableConfigurationProperties, implementing/reusing RetryProperties, naming
retryTemplate, and using Registry spelling are all good consistency fixes.

thanks

[igordayen]

@jimador - just flagging depency on autoconfig outside autoconfig, please consider moving, thanks

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
97.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[igordayen]

@jimador looks good, thanks for addressing inquiries