@lancedb/lancedb • Docs

---

@lancedb/lancedb / tokenize

Function: tokenize()

function tokenize(query, options?): Promise<FtsToken[]>

Tokenize a full-text search query using an explicit tokenizer.

This does not require a table or FTS index. The tokenizer options match Index.fts.

Parameters

• query: string

• options?: Partial<TokenizeOptions>

Returns

Promise<FtsToken[]>