Class FuzzyIndex

A ranked search over a caller's documents. Each document is a set of named fields, a text or a list of short texts each, and each field has the weight the index was made with. A query is read in NFC and cut into words by the tokenizer, and each word, folded to lower case with its accents dropped, matches a word of a document that is:

• the same word, which counts 1;
• a word it begins, which counts 0.75;
• one edit away (a letter added, dropped or changed, or two letters side by side swapped: withinOneEdit), where both words have editMinLength letters or more, which counts 0.5.

A document's score is sum over the query's words q of (max over its fields f and their words t of weight(f) * kind(q, t)): each query word's best match, its kind times the weight of the field it is in, summed over the query's words, each counted once. With match: 'all' (the default) every query word must match somewhere in the document; with 'any', at least one. Hits come highest score first; a tie keeps the order the documents were added in, a document added again under its id counting as added last.

A hit holds each field of the document with its matched words marked, and up to passages sentences or lines of the passage fields that hold a match: those that hold the most query words, then the earliest, shown in the order of the text, each clipped to about passageWidth characters around its first match. A document matched only in another field shows the opening of its text, unmarked. A mark is a whole word at UTF-16 offsets into the text as the index holds it, in NFC; the marks of a text are in order and never overlap.

A search reads the words near each query word's, not every word the index holds: it looks a query word up in two orders of the index's words, by how they begin and by how they end. It reads the words that begin with the query word's first half, the query word itself and the words it begins among them, the words that end with its second half, and the one word with the two letters on either side of its middle swapped; a query word under editMinLength letters reads only the words it begins. The first search after a document is added brings the two orders up to date. The fields of the hits a search answers are then read again to mark them.

Pure and synchronous: it reads and writes nothing outside itself and imports nothing but the library's word rule, so a page can hold it. It is saved with FuzzyIndex.toJSON and restored with FuzzyIndex.fromJSON.

Example

const index = new FuzzyIndex({ fields: { title: 100, keywords: 10, body: 1 } });
index.addDocument({
  id: 'n1',
  fields: { title: 'Enclosure samples', keywords: ['factory'], body: 'The samples leave the factory on Friday.' },
});
const [hit] = index.search('firday'); // "Friday", one swap away in the body: 0.5 times 1
hit.passages; // [{ text: 'The samples leave the factory on Friday.', marks: [{ start: 33, end: 39 }] }]

const restored = FuzzyIndex.fromJSON(JSON.parse(JSON.stringify(index.toJSON())));
Copy