Function lexicalTokens

  • lexicalTokens(text): string[]

  • The words of a text: read in NFC and in lower case, each a letter or a digit and the letters, digits and marks after it, in order.

    Parameters

    • text: string

    Returns string[]

Settings

Member Visibility

Theme