Interface EmergentJudgeConfig

Configuration for the EmergentJudge.

All LLM interaction is abstracted behind the generateText callback, making the judge model-agnostic and easily testable with mocks.

interface EmergentJudgeConfig {
    judgeModel: string;
    promotionModel: string;
    generateText: ((model, prompt) => Promise<string>);
    generateTextWithSystem?: ((model, system, user) => Promise<string>);
}

Properties

judgeModel promotionModel generateText generateTextWithSystem?

Properties

judgeModel

judgeModel: string

Model ID used for the single-pass creation review. Should be a fast, cost-efficient model since correctness is primarily validated through test cases.

Example

"gpt-4o-mini"

promotionModel

promotionModel: string

Model ID used by both reviewers in the promotion panel. Should be a more capable model than judgeModel since promotion decisions are higher-stakes.

Example

"gpt-4o"

generateText

generateText: ((model, prompt) => Promise<string>)

Callback that invokes an LLM to generate text from a prompt. The judge calls this for creation reviews and promotion panels.

Type declaration

    • (model, prompt): Promise<string>

    • Parameters

      • model: string

        The model ID to use for generation.

      • prompt: string

        The full prompt string to send to the LLM.

      Returns Promise<string>

Returns

The raw text response from the LLM.

Optional generateTextWithSystem

generateTextWithSystem?: ((model, system, user) => Promise<string>)

Optional structured callback that receives a stable system prefix and a candidate-specific user payload separately. When supplied, the judge prefers this path over generateText so hosts can attach provider-level prompt caching (e.g. Anthropic cache_control: ephemeral or OpenAI automatic prefix cache) to the shared rubric. A 10-20 call run on Anthropic sees ~25% judge cost reduction once the ~500-token rubric hits the cache on call 2+.

Hosts that do not care about caching may omit this field; the judge falls back to concatenating system + '\n\n' + user and calling the legacy generateText path, which preserves behavior exactly.

Type declaration

    • (model, system, user): Promise<string>

    • Parameters

      • model: string

        Model ID to use for generation.

      • system: string

        Stable rubric text. Safe to mark cacheable.

      • user: string

        Candidate-specific payload that varies per call.

      Returns Promise<string>

Returns

The raw text response from the LLM.

Settings

Member Visibility

Theme

On This Page

judgeModelpromotionModelgenerateTextgenerateTextWithSystem