Package version:

Enumeration KnownTokenizerNames

Known values of LexicalTokenizerName that the service accepts.

Enumeration Members

Classic EdgeNGram Keyword Letter Lowercase MicrosoftLanguageStemmingTokenizer MicrosoftLanguageTokenizer NGram PathHierarchy Pattern Standard UaxUrlEmail Whitespace

Enumeration Members

Classic

Classic: "classic"

Grammar-based tokenizer that is suitable for processing most European-language documents. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/standard/ClassicTokenizer.html

EdgeNGram

EdgeNGram: "edgeNGram"

Tokenizes the input from an edge into n-grams of the given size(s). See https://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/ngram/EdgeNGramTokenizer.html

Keyword

Keyword: "keyword_v2"

Emits the entire input as a single token. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/core/KeywordTokenizer.html

Letter

Letter: "letter"

Divides text at non-letters. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/core/LetterTokenizer.html

Lowercase

Lowercase: "lowercase"

Divides text at non-letters and converts them to lower case. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/core/LowerCaseTokenizer.html

MicrosoftLanguageStemmingTokenizer

MicrosoftLanguageStemmingTokenizer: "microsoft_language_stemming_tokenizer"

Divides text using language-specific rules and reduces words to their base forms.

MicrosoftLanguageTokenizer

MicrosoftLanguageTokenizer: "microsoft_language_tokenizer"

Divides text using language-specific rules.

NGram

NGram: "nGram"

Tokenizes the input into n-grams of the given size(s). See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/ngram/NGramTokenizer.html

PathHierarchy

PathHierarchy: "path_hierarchy_v2"

Tokenizer for path-like hierarchies. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/path/PathHierarchyTokenizer.html

Pattern

Pattern: "pattern"

Tokenizer that uses regex pattern matching to construct distinct tokens. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/pattern/PatternTokenizer.html

Standard

Standard: "standard_v2"

Standard Lucene analyzer; Composed of the standard tokenizer, lowercase filter and stop filter. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/standard/StandardTokenizer.html

UaxUrlEmail

UaxUrlEmail: "uax_url_email"

Tokenizes urls and emails as one token. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/standard/UAX29URLEmailTokenizer.html

Whitespace

Whitespace: "whitespace"

Divides text at whitespace. See http://lucene.apache.org/core/4_10_3/analyzers-common/org/apache/lucene/analysis/core/WhitespaceTokenizer.html

Settings

Member Visibility