Class SpeechProviderResolver

Central resolver for speech providers (STT, TTS, VAD, wake-word).

Resolution Algorithm

  1. Registration — Providers are registered via register() with a unique id, a kind (stt/tts/vad/wake-word), a numeric priority, and a boolean isConfigured flag.

  2. Filtering — When a consumer calls resolveSTT(), resolveTTS(), resolveVAD(), or resolveWakeWord(), the resolver filters registrations by kind, isConfigured === true and a registered provider instance (provider). A core catalog entry with its keys set but no instance does not resolve.

  3. Requirements matching — Optional ProviderRequirements further filter by streaming, local, and features capabilities from the provider's catalog entry.

  4. Priority ordering — Remaining candidates are sorted by ascending priority (lower number = tried first). Core providers default to 100, extensions to 200, and user-preferred providers get boosted to 50+.

  5. Fallback wrapping — When config.stt.fallback or config.tts.fallback is true and multiple candidates survive, they are wrapped in a FallbackSTTProxy or FallbackTTSProxy that tries each in order, emitting provider_fallback events on failure.

Priority Tiers

Tier Priority Range Source
User-preferred 50–59 config.stt.preferred / config.tts.preferred
Core 100 Built-in provider catalog
Extension 200 Discovered via ExtensionManager

Events

Event Payload When
provider_registered { id, kind, source } A provider is registered via register()

See

Example

// SpeechRuntime builds the providers whose keys are set (DEEPGRAM_API_KEY
// for deepgram-batch) and registers them in its resolver.
const runtime = new SpeechRuntime({ env: process.env, preferredSttProviderId: 'deepgram-batch' });
await runtime.resolver.refresh(); // applies the preferred priorities
const stt = runtime.resolver.resolveSTT();

Hierarchy

  • EventEmitter
    • SpeechProviderResolver

Constructors

  • Creates a new SpeechProviderResolver.

    Parameters

    • Optional config: SpeechResolverConfig

      Optional resolver configuration controlling preferred providers and fallback behaviour for each provider kind.

    • env: Record<string, undefined | string> = process.env

      Environment variable map used to check whether a provider's required API keys are present. Defaults to process.env so unit tests can inject a controlled map without polluting the real environment.

    Returns SpeechProviderResolver

    Example

    // Production: use real env vars
    const resolver = new SpeechProviderResolver({ stt: { fallback: true } });

    // Testing: inject controlled env
    const resolver = new SpeechProviderResolver(undefined, { OPENAI_API_KEY: 'test' });

Methods

  • Register a provider, overwriting any existing registration with the same kind and id. Re-registering at the priority the resolver shows (a preferred boost included), as register({ ...entry, provider }) does, keeps the priority the provider was first registered with.

    Emits a provider_registered event with { id, kind, source } so that listeners (e.g. UI dashboards, logging middleware) can track what's available.

    Parameters

    • reg: ProviderRegistration

      The full registration object including the provider instance, catalog entry, priority, and configuration status.

    Returns void

    Example

    resolver.register({
    id: 'custom-stt',
    kind: 'stt',
    provider: myCustomProvider,
    catalogEntry: { id: 'custom-stt', kind: 'stt', label: 'Custom', envVars: [], local: false, description: '' },
    isConfigured: true,
    priority: 150,
    source: 'extension',
    });
  • List all registrations for a given provider kind, sorted ascending by priority (lower number = higher priority = tried first).

    This returns both configured and unconfigured providers — use .filter(r => r.isConfigured && r.provider) for the ones the resolve methods can return (keys set and an instance registered).

    Parameters

    Returns ProviderRegistration[]

    A new array of registrations sorted by ascending priority.

    Example

    const allSTT = resolver.listProviders('stt');
    const ready = allSTT.filter(r => r.isConfigured && r.provider);
    console.log(`${ready.length} of ${allSTT.length} STT providers ready`);
  • Resolve the best STT provider matching optional ProviderRequirements.

    When config.stt.fallback is true and multiple candidates exist, wraps them in a FallbackSTTProxy that tries providers in priority order. Otherwise returns the single highest-priority match.

    Parameters

    • Optional requirements: ProviderRequirements

      Optional filtering criteria (streaming, local, features, preferredIds).

    Returns SpeechToTextProvider

    The resolved STT provider, possibly wrapped in a FallbackSTTProxy.

    Throws

    When no configured STT provider with an instance matches the requirements.

    See

    Example

    // Simple: best available
    const stt = resolver.resolveSTT();

    // With requirements
    const stt = resolver.resolveSTT({ streaming: true, features: ['diarization'] });

    // With explicit preference ordering
    const stt = resolver.resolveSTT({ preferredIds: ['deepgram-batch', 'openai-whisper'] });
  • Resolve the highest-priority configured VAD provider.

    VAD providers don't support fallback chains because voice activity detection is a real-time frame-by-frame operation where mid-session provider switching would cause state inconsistency.

    Returns SpeechVadProvider

    The resolved VAD provider instance.

    Throws

    When no configured VAD provider with an instance is registered.

    Example

    const vad = resolver.resolveVAD();
    const decision = vad.processFrame(audioFrame);
  • Resolve the highest-priority configured wake-word provider, or null when none is registered.

    Returns null instead of throwing because wake-word detection is an optional feature — many deployments operate without it.

    Returns null | WakeWordProvider

    The resolved wake-word provider, or null if none is available.

    Example

    const wakeWord = resolver.resolveWakeWord();
    if (wakeWord) {
    const detection = await wakeWord.detect(frame, sampleRate);
    }
  • Populate the resolver by registering core providers from the static catalog, optionally discovering extension providers, and applying user-configured preferred priorities.

    The refresh sequence is:

    1. discoverExtensionProviders() — if an ExtensionManager is provided, read the speech providers it lists as active, and drop the ones an earlier refresh read from it that it no longer lists.
    2. registerCoreProviders() — register all built-in providers from the static catalog, marking each as configured/unconfigured based on env vars.
    3. Register the providers read in step 1, at priority 200.
    4. applyPreferredPriorities() — boost priority for providers listed in the user's config.stt.preferred / config.tts.preferred arrays.

    Parameters

    • Optional extensionManager: any

      Optional ExtensionManager (its speech provider registries are read through getRegistry(kind).listActive()), or any object exposing getDescriptorsByKind(kind). Uses any type because importing the ExtensionManager type here would create a circular dependency.

    Returns Promise<void>

    Example

    const runtime = new SpeechRuntime({ env: process.env });
    await runtime.resolver.refresh(extensionManager);
    // The manager's speech providers now resolve beside the runtime's own.
  • Registers an explicit scoped factory. Legacy catalog metadata cannot satisfy this API.

    Parameters

    • adapter: ScopedVoiceAdapter<keyof VoiceAdapterValues>

    Returns void

  • Reads typed descriptors without triggering legacy environment discovery or creating providers.

    Parameters

    • manager: {
          getVoiceAdapters(): ScopedVoiceAdapter<keyof VoiceAdapterValues>[];
      }

    Returns void

  • Validates language, PCM format, immutable models/voices, options and every fallback before construction.

    Parameters

    • rawScope: VoiceScopeContext
    • rawAgent: {
          name: string;
          prompt: string;
          language: string;
          reasoning: {
              modelId: string;
              modelRevision: string;
              protocol: {
                  id: "openai-chat-completions";
                  version: 1;
              };
              usageUnits: {
                  input_token: "token";
                  output_token: "token";
                  reasoning_token: "token";
                  cached_input_token: "token";
                  reasoningInOutput: boolean;
              };
              attemptLimits: {
                  maxAttempts: number;
                  perAttempt: {
                      input_token: number;
                      output_token: number;
                      reasoning_token: number;
                      cached_input_token: number;
                      request: number;
                  };
              };
              cancellation: "native_abort" | "process_exit" | "unsupported";
              provider: "local";
              manifestDigest: string;
              fallbacks: ({
                  modelId: string;
                  modelRevision: string;
                  protocol: {
                      id: "openai-chat-completions";
                      version: 1;
                  };
                  usageUnits: {
                      input_token: "token";
                      output_token: "token";
                      reasoning_token: "token";
                      cached_input_token: "token";
                      reasoningInOutput: boolean;
                  };
                  attemptLimits: {
                      maxAttempts: number;
                      perAttempt: {
                          input_token: number;
                          output_token: number;
                          reasoning_token: number;
                          cached_input_token: number;
                          request: number;
                      };
                  };
                  cancellation: "native_abort" | "process_exit" | "unsupported";
                  provider: "local";
                  manifestDigest: string;
              } | {
                  modelId: string;
                  modelRevision: string;
                  protocol: {
                      id: "openai-chat-completions";
                      version: 1;
                  };
                  usageUnits: {
                      input_token: "token";
                      output_token: "token";
                      reasoning_token: "token";
                      cached_input_token: "token";
                      reasoningInOutput: boolean;
                  };
                  attemptLimits: {
                      maxAttempts: number;
                      perAttempt: {
                          input_token: number;
                          output_token: number;
                          reasoning_token: number;
                          cached_input_token: number;
                          request: number;
                      };
                  };
                  cancellation: "native_abort" | "process_exit" | "unsupported";
                  provider: "registered_endpoint";
                  endpointRef: string;
                  endpointRevision: number;
                  credentialRef?: string;
              })[];
          } | {
              modelId: string;
              modelRevision: string;
              protocol: {
                  id: "openai-chat-completions";
                  version: 1;
              };
              usageUnits: {
                  input_token: "token";
                  output_token: "token";
                  reasoning_token: "token";
                  cached_input_token: "token";
                  reasoningInOutput: boolean;
              };
              attemptLimits: {
                  maxAttempts: number;
                  perAttempt: {
                      input_token: number;
                      output_token: number;
                      reasoning_token: number;
                      cached_input_token: number;
                      request: number;
                  };
              };
              cancellation: "native_abort" | "process_exit" | "unsupported";
              provider: "registered_endpoint";
              endpointRef: string;
              endpointRevision: number;
              credentialRef?: string;
              fallbacks: ({
                  modelId: string;
                  modelRevision: string;
                  protocol: {
                      id: "openai-chat-completions";
                      version: 1;
                  };
                  usageUnits: {
                      input_token: "token";
                      output_token: "token";
                      reasoning_token: "token";
                      cached_input_token: "token";
                      reasoningInOutput: boolean;
                  };
                  attemptLimits: {
                      maxAttempts: number;
                      perAttempt: {
                          input_token: number;
                          output_token: number;
                          reasoning_token: number;
                          cached_input_token: number;
                          request: number;
                      };
                  };
                  cancellation: "native_abort" | "process_exit" | "unsupported";
                  provider: "local";
                  manifestDigest: string;
              } | {
                  modelId: string;
                  modelRevision: string;
                  protocol: {
                      id: "openai-chat-completions";
                      version: 1;
                  };
                  usageUnits: {
                      input_token: "token";
                      output_token: "token";
                      reasoning_token: "token";
                      cached_input_token: "token";
                      reasoningInOutput: boolean;
                  };
                  attemptLimits: {
                      maxAttempts: number;
                      perAttempt: {
                          input_token: number;
                          output_token: number;
                          reasoning_token: number;
                          cached_input_token: number;
                          request: number;
                      };
                  };
                  cancellation: "native_abort" | "process_exit" | "unsupported";
                  provider: "registered_endpoint";
                  endpointRef: string;
                  endpointRevision: number;
                  credentialRef?: string;
              })[];
          };
          voice: {
              id: string;
              revision: string;
          };
          stt: {
              providerId: string;
              modelId: string;
              modelRevision: string;
              mode: "streaming" | "batch";
              sampleRate: number;
              codec: "pcm_s16le" | "pcm_f32le" | "opus" | "mp3" | "wav";
              optionsRef?: string;
              credentialRef?: string;
          };
          tts: {
              providerId: string;
              modelId: string;
              modelRevision: string;
              mode: "streaming" | "batch";
              sampleRate: number;
              codec: "pcm_s16le" | "pcm_f32le" | "opus" | "mp3" | "wav";
              optionsRef?: string;
              credentialRef?: string;
          };
          vad: {
              providerId: string;
              modelId: string;
              modelRevision: string;
              threshold: number;
              optionsRef?: string;
          };
          endpoint: {
              providerId: string;
              modelId: string;
              modelRevision: string;
              optionsRef?: string;
          };
          providerPolicy: {
              mode: "provider" | "local" | "hosted";
              providerIds: string[];
              purposes: ("stt" | "tts" | "vad" | "endpoint" | "reasoning")[];
              fallback: "none" | "allowlisted";
          };
          limits: {
              maxDurationSeconds: number;
              setupTimeoutMs: number;
              reconnectGraceMs: number;
              silenceTimeoutMs: number;
              cleanupTimeoutMs: number;
              maxTurns: number;
          };
          allowedToolRefs: string[];
          transcriptPolicy: {
              mode: "none";
          } | {
              mode: "delivery";
              deliveryTtlSeconds: number;
          } | {
              mode: "retained";
              retentionSeconds: number;
          };
          webhookRef: null | string;
          version: 1;
          agentId: string;
          scopeId: string;
          revision: number;
          state: "active" | "archived";
          createdAt: number;
          updatedAt: number;
      }
      • name: string
      • prompt: string
      • language: string
      • reasoning: {
            modelId: string;
            modelRevision: string;
            protocol: {
                id: "openai-chat-completions";
                version: 1;
            };
            usageUnits: {
                input_token: "token";
                output_token: "token";
                reasoning_token: "token";
                cached_input_token: "token";
                reasoningInOutput: boolean;
            };
            attemptLimits: {
                maxAttempts: number;
                perAttempt: {
                    input_token: number;
                    output_token: number;
                    reasoning_token: number;
                    cached_input_token: number;
                    request: number;
                };
            };
            cancellation: "native_abort" | "process_exit" | "unsupported";
            provider: "local";
            manifestDigest: string;
            fallbacks: ({
                modelId: string;
                modelRevision: string;
                protocol: {
                    id: "openai-chat-completions";
                    version: 1;
                };
                usageUnits: {
                    input_token: "token";
                    output_token: "token";
                    reasoning_token: "token";
                    cached_input_token: "token";
                    reasoningInOutput: boolean;
                };
                attemptLimits: {
                    maxAttempts: number;
                    perAttempt: {
                        input_token: number;
                        output_token: number;
                        reasoning_token: number;
                        cached_input_token: number;
                        request: number;
                    };
                };
                cancellation: "native_abort" | "process_exit" | "unsupported";
                provider: "local";
                manifestDigest: string;
            } | {
                modelId: string;
                modelRevision: string;
                protocol: {
                    id: "openai-chat-completions";
                    version: 1;
                };
                usageUnits: {
                    input_token: "token";
                    output_token: "token";
                    reasoning_token: "token";
                    cached_input_token: "token";
                    reasoningInOutput: boolean;
                };
                attemptLimits: {
                    maxAttempts: number;
                    perAttempt: {
                        input_token: number;
                        output_token: number;
                        reasoning_token: number;
                        cached_input_token: number;
                        request: number;
                    };
                };
                cancellation: "native_abort" | "process_exit" | "unsupported";
                provider: "registered_endpoint";
                endpointRef: string;
                endpointRevision: number;
                credentialRef?: string;
            })[];
        } | {
            modelId: string;
            modelRevision: string;
            protocol: {
                id: "openai-chat-completions";
                version: 1;
            };
            usageUnits: {
                input_token: "token";
                output_token: "token";
                reasoning_token: "token";
                cached_input_token: "token";
                reasoningInOutput: boolean;
            };
            attemptLimits: {
                maxAttempts: number;
                perAttempt: {
                    input_token: number;
                    output_token: number;
                    reasoning_token: number;
                    cached_input_token: number;
                    request: number;
                };
            };
            cancellation: "native_abort" | "process_exit" | "unsupported";
            provider: "registered_endpoint";
            endpointRef: string;
            endpointRevision: number;
            credentialRef?: string;
            fallbacks: ({
                modelId: string;
                modelRevision: string;
                protocol: {
                    id: "openai-chat-completions";
                    version: 1;
                };
                usageUnits: {
                    input_token: "token";
                    output_token: "token";
                    reasoning_token: "token";
                    cached_input_token: "token";
                    reasoningInOutput: boolean;
                };
                attemptLimits: {
                    maxAttempts: number;
                    perAttempt: {
                        input_token: number;
                        output_token: number;
                        reasoning_token: number;
                        cached_input_token: number;
                        request: number;
                    };
                };
                cancellation: "native_abort" | "process_exit" | "unsupported";
                provider: "local";
                manifestDigest: string;
            } | {
                modelId: string;
                modelRevision: string;
                protocol: {
                    id: "openai-chat-completions";
                    version: 1;
                };
                usageUnits: {
                    input_token: "token";
                    output_token: "token";
                    reasoning_token: "token";
                    cached_input_token: "token";
                    reasoningInOutput: boolean;
                };
                attemptLimits: {
                    maxAttempts: number;
                    perAttempt: {
                        input_token: number;
                        output_token: number;
                        reasoning_token: number;
                        cached_input_token: number;
                        request: number;
                    };
                };
                cancellation: "native_abort" | "process_exit" | "unsupported";
                provider: "registered_endpoint";
                endpointRef: string;
                endpointRevision: number;
                credentialRef?: string;
            })[];
        }
      • voice: {
            id: string;
            revision: string;
        }
        • id: string
        • revision: string
      • stt: {
            providerId: string;
            modelId: string;
            modelRevision: string;
            mode: "streaming" | "batch";
            sampleRate: number;
            codec: "pcm_s16le" | "pcm_f32le" | "opus" | "mp3" | "wav";
            optionsRef?: string;
            credentialRef?: string;
        }
        • providerId: string
        • modelId: string
        • modelRevision: string
        • mode: "streaming" | "batch"
        • sampleRate: number
        • codec: "pcm_s16le" | "pcm_f32le" | "opus" | "mp3" | "wav"
        • Optional optionsRef?: string
        • Optional credentialRef?: string
      • tts: {
            providerId: string;
            modelId: string;
            modelRevision: string;
            mode: "streaming" | "batch";
            sampleRate: number;
            codec: "pcm_s16le" | "pcm_f32le" | "opus" | "mp3" | "wav";
            optionsRef?: string;
            credentialRef?: string;
        }
        • providerId: string
        • modelId: string
        • modelRevision: string
        • mode: "streaming" | "batch"
        • sampleRate: number
        • codec: "pcm_s16le" | "pcm_f32le" | "opus" | "mp3" | "wav"
        • Optional optionsRef?: string
        • Optional credentialRef?: string
      • vad: {
            providerId: string;
            modelId: string;
            modelRevision: string;
            threshold: number;
            optionsRef?: string;
        }
        • providerId: string
        • modelId: string
        • modelRevision: string
        • threshold: number
        • Optional optionsRef?: string
      • endpoint: {
            providerId: string;
            modelId: string;
            modelRevision: string;
            optionsRef?: string;
        }
        • providerId: string
        • modelId: string
        • modelRevision: string
        • Optional optionsRef?: string
      • providerPolicy: {
            mode: "provider" | "local" | "hosted";
            providerIds: string[];
            purposes: ("stt" | "tts" | "vad" | "endpoint" | "reasoning")[];
            fallback: "none" | "allowlisted";
        }
        • mode: "provider" | "local" | "hosted"
        • providerIds: string[]
        • purposes: ("stt" | "tts" | "vad" | "endpoint" | "reasoning")[]
        • fallback: "none" | "allowlisted"
      • limits: {
            maxDurationSeconds: number;
            setupTimeoutMs: number;
            reconnectGraceMs: number;
            silenceTimeoutMs: number;
            cleanupTimeoutMs: number;
            maxTurns: number;
        }
        • maxDurationSeconds: number
        • setupTimeoutMs: number
        • reconnectGraceMs: number
        • silenceTimeoutMs: number
        • cleanupTimeoutMs: number
        • maxTurns: number
      • allowedToolRefs: string[]
      • transcriptPolicy: {
            mode: "none";
        } | {
            mode: "delivery";
            deliveryTtlSeconds: number;
        } | {
            mode: "retained";
            retentionSeconds: number;
        }
      • webhookRef: null | string
      • version: 1
      • agentId: string
      • scopeId: string
      • revision: number
      • state: "active" | "archived"
      • createdAt: number
      • updatedAt: number
    • options: VoiceOptionsStore
    • network: VoiceNetworkDependencies = {}

    Returns Promise<FrozenVoicePlan>

  • Resolves a scoped batch provider through the same registered options, credentials and request budget.

    Type Parameters

    • K extends "batch-stt" | "batch-tts"

    Parameters

    • kind: K
    • providerId: string
    • rawScope: VoiceScopeContext
    • selection: VoiceAdapterSelection
    • budget: IAttemptAuthorizer
    • network: VoiceNetworkDependencies = {}
    • Optional onSupplierUsage: VoiceSupplierUsageSink

    Returns Promise<OpenedVoiceAdapter<K>>