Interface ModelPoolEntry

One model a seat can sit on. The record key is the entry's name.

interface ModelPoolEntry {
    provider: string;
    model: string;
    apiKey?: string;
    baseUrl?: string;
    effort?: string;
    thinking?: false | {
        budgetTokens: number;
    };
    maxTokens?: number;
    weight?: number;
    vendor?: string;
}

Properties

provider model apiKey? baseUrl? effort? thinking? maxTokens? weight? vendor?

Properties

provider

provider: string

One of the twelve text provider ids the provider manager builds and the resolver can resolve by name (every one but litellm), checked against that list and never against PROVIDER_DEFAULTS, which also holds media providers.

model

model: string

A plain model id for that provider. A known provider: prefix is rejected.

Optional apiKey

apiKey?: string

Key for this entry. Falls back to a same-provider setDefaultProvider() key, then the provider's environment variable; never to the agency-level apiKey.

Optional baseUrl

baseUrl?: string

Base URL for this entry. Falls back as apiKey does, to a same-provider setDefaultProvider() URL, then the provider's URL variable; an empty or whitespace-only value at any step counts as missing.

Optional effort

effort?: string

Defaults for a seat that sits here and sets none of its own.

Optional thinking

thinking?: false | {
    budgetTokens: number;
}

Default thinking budget for a seat that sits here and sets none of its own.

Type declaration

  • budgetTokens: number

Optional maxTokens

maxTokens?: number

Default output cap for a seat that sits here and sets none of its own. A positive integer.

Optional weight

weight?: number

Relative share for the weighted and round-robin policies. A finite number greater than 0. Default 1.

Optional vendor

vendor?: string

Who trained the model, when it cannot be read from the provider and model id.

Settings

Member Visibility

Theme

On This Page

providermodelapiKeybaseUrleffortthinkingmaxTokensweightvendor