Interface VisionPipelineConfig

Configuration for the VisionPipeline.

All fields are optional — the factory function createVisionPipeline auto-detects available providers and fills in sensible defaults.

Example

const config: VisionPipelineConfig = {
  strategy: 'progressive',
  ocr: 'paddle',
  handwriting: true,
  documentAI: true,
  embedding: true,
  cloudProvider: 'openai',
  cloudModel: 'gpt-4o',
  confidenceThreshold: 0.8,
  preprocessing: { grayscale: true, sharpen: true },
};
interface VisionPipelineConfig {
    strategy: VisionStrategy;
    ocr?: "paddle" | "tesseract" | "none";
    handwriting?: boolean;
    documentAI?: boolean;
    embedding?: boolean;
    cloudProvider?: string;
    cloudModel?: string;
    cloudApiKey?: string;
    cloudBaseUrl?: string;
    cloudFallbackProviders?: FallbackProviderEntry[];
    confidenceThreshold?: number;
    preprocessing?: VisionPreprocessingConfig;
}

Properties

strategy ocr? handwriting? documentAI? embedding? cloudProvider? cloudModel? cloudApiKey? cloudBaseUrl? cloudFallbackProviders? confidenceThreshold? preprocessing?

Properties

strategy

strategy: VisionStrategy

How to combine tiers.

Default

'progressive'

Optional ocr

ocr?: "paddle" | "tesseract" | "none"

OCR engine for Tier 1 text extraction.

  • 'paddle' — PaddleOCR (via ppu-paddle-ocr). Best accuracy for most scripts.
  • 'tesseract' — Tesseract.js. Wider language support, slightly lower accuracy.
  • 'none' — Skip OCR entirely (useful for photo-only pipelines).

Default

'paddle' (if installed), else 'tesseract' (if installed), else 'none'

Optional handwriting

handwriting?: boolean

Enable handwriting recognition via TrOCR (@huggingface/transformers). Only triggered when OCR confidence is low and content appears handwritten.

Default

false

Optional documentAI

documentAI?: boolean

Enable document understanding via Florence-2 (@huggingface/transformers). Produces a DocumentLayout of the lines of text it reads, each with its box.

Default

false

Optional embedding

embedding?: boolean

Enable CLIP image embeddings (@huggingface/transformers). Runs in parallel with other tiers — does not affect text extraction.

Default

false

Optional cloudProvider

cloudProvider?: string

Cloud vision LLM provider name for Tier 3 fallback. Must match a provider known to generateText() (e.g. 'openai', 'anthropic', 'gemini'). When unset, cloud vision is disabled.

Optional cloudModel

cloudModel?: string

Cloud model override. When unset, the provider's default vision model is used.

Example

'gpt-4o', 'claude-sonnet-4-6', 'gemini-2.5-flash'

Optional cloudApiKey

cloudApiKey?: string

API key for the cloud vision provider. When unset, the provider's key is read from its environment variable (OPENAI_API_KEY and so on), as generateText does. createVisionPipeline refuses a key, and a base URL, that comes with no cloudProvider and no cloudModel naming a provider (openai:gpt-4o): it would go to whichever provider the environment names.

Optional cloudBaseUrl

cloudBaseUrl?: string

Base URL for the cloud vision provider, such as a proxy or a local OpenAI-compatible server. When unset, the provider's default is used. Needs a named provider, as cloudApiKey does. An empty or whitespace-only value is unset: it needs no provider, and the provider's base-URL variable, then its default, apply as with none.

Optional cloudFallbackProviders

cloudFallbackProviders?: FallbackProviderEntry[]

The providers the cloud tier may try, in order, after cloudProvider fails: generateText's fallbackProviders. Default [], so the image goes to cloudProvider alone and never to another vendor whose key is in the environment. A listed provider uses its own key from the environment, never cloudApiKey or cloudBaseUrl.

Optional confidenceThreshold

confidenceThreshold?: number

Minimum confidence to accept an OCR result without escalating to cloud. Only applies to 'progressive' strategy — if OCR confidence is below this threshold, the pipeline escalates to the next tier.

Default

0.7

Optional preprocessing

preprocessing?: VisionPreprocessingConfig

Image preprocessing options applied before any tier runs. Uses sharp for resizing, grayscale conversion, sharpening, and normalization.

Settings

Member Visibility

Theme

On This Page

strategyocrhandwritingdocumentAIembeddingcloudProvidercloudModelcloudApiKeycloudBaseUrlcloudFallbackProvidersconfidenceThresholdpreprocessing