Class UrlLoader

An IDocumentLoader that fetches a remote URL and delegates parsing to the appropriate registered loader based on the response Content-Type.

Supported content types

Content-Type Delegates to
text/html HtmlLoader (registry)
application/pdf PdfLoader (registry)
Everything else Plain UTF-8 text

With fetchDocument, the answer's media type picks the loader: text/html and application/xhtml+xml go to the .html loader, application/pdf to the .pdf loader, anything else is read as UTF-8 text, and metadata.source is the answer's last address after redirects. An HTML, XHTML or PDF answer the loader source has no loader for is refused with an error, so an HTML or XHTML answer is never returned as raw markup.

Example

const registry = new LoaderRegistry();
const urlLoader = new UrlLoader(registry);

// Register so the registry also dispatches URLs via canLoad checks.
// (Optional — UrlLoader can be used standalone too.)

if (urlLoader.canLoad('https://example.com/report.pdf')) {
  const doc = await urlLoader.load('https://example.com/report.pdf');
  console.log(doc.format); // 'pdf'
}

An address a person gave a server

const loaders = {
  getLoader: (extension: string) =>
    extension === '.html' ? new HtmlLoader() : extension === '.pdf' ? new PdfLoader() : undefined,
};
const loader = new UrlLoader(loaders, {
  fetchDocument: (url) => guardedFetch(url, { maxBytes: 2 * 1024 * 1024, deadlineMs: 10_000 }),
});
const doc = await loader.load(address);

Implements

Implements

Constructors

constructor

Methods

canLoad load

Properties

supportedExtensions

Constructors

constructor

Methods

canLoad

  • canLoad(source): boolean

  • Returns true when source is a string that starts with http:// or https://.

    Buffer sources are always rejected — raw bytes cannot be a URL.

    Parameters

    • source: string | Buffer<ArrayBufferLike>

      Absolute file path, URL string, or raw bytes.

    Returns boolean

load

  • load(source, options?): Promise<LoadedDocument>

  • Fetch source over HTTP/HTTPS and return a LoadedDocument.

    The response body is buffered in memory and then handed to the appropriate sub-loader according to the Content-Type header:

    • text/html → fetched as text, passed to the HTML loader as a Buffer.
    • application/pdf → fetched as bytes, passed to the PDF loader as a Buffer.
    • Anything else → returned as plain text with format 'text' and source metadata set to the URL.

    With fetchDocument, the answer it gives is handed on instead: HTML and XHTML to the HTML loader, PDF to the PDF loader, anything else as plain text, with source metadata set to the answer's last address.

    Parameters

    • source: string | Buffer<ArrayBufferLike>

      HTTP/HTTPS URL string.

    • Optional options: LoadOptions

      Optional load hints forwarded to the delegated loader.

    Returns Promise<LoadedDocument>

    A promise resolving to the LoadedDocument.

    Throws

    When source is a Buffer (URLs must be strings).

    Throws

    When the HTTP request fails (network error or non-2xx status), or whatever fetchDocument throws.

    Throws

    When the answer is a PDF and the loader source has no .pdf loader, or, with fetchDocument, HTML or XHTML and it has no .html loader.

Properties

Readonly supportedExtensions

supportedExtensions: string[] = []

URLs have no file extension so this array is always empty.

Routing to this loader must be performed via canLoad rather than the registry's extension-based lookup.

Settings

Member Visibility

Theme

On This Page

constructorcanLoadloadsupportedExtensions