Explanation of searchable fields in Atlan¶
The full model reference provides details of every metadata attribute available in Atlan. Alongside each attribute is a listing of the search-specific fields for that attribute and its type.
Field types¶
We use the following conventions to delineate the nuances of how different attributes are indexed, and what this means for searching. Note that in the SDKs, every field that exists in Atlan is enumerated. This avoids you needing to remember the exact name of the field (which can vary depending on how it is indexed, even for the same field) or making typos in strings you send in the query.
With the fluent search abstraction, you no longer need to consider how you want to search a particular attribute — the underlying implementation is chosen based on the search predicate you use when writing your query. However, for those curious about the mapping, the following are the different index field that we use in Elastic to handle different kinds of searches.
Keyword¶
Keyword fields are used for content that you would usually search using exact values. (In particular, for term-level queries.)
They are generally not used for full-text search where you would expect your input to provide results that are fuzzy-matched using synonyms, both plural/singular versions of words, matching any or multiple words in a multi-word search input, and so on.
In the SDKs, fields that have a keyword index will be one of these types:
KeywordField- have only a keyword indexKeywordTextField— have both a keyword and text indexKeywordTextStemmedField— have a keyword, text, and stemmed text index
Text¶
Text fields are used for content that you would usually search using full-text queries.
These fields are passed through an analyzer to convert them into an individual list of terms that can be searched. This provides the ability to fuzzy-match using synonyms, both plural/singular versions of words, matching one or many words in a multi-word search input, and so on. (Including scoring each result differently based on how many of these aspects it matches compared to other results.)
In the SDKs, fields that have a text index will be one of these types:
TextField— have only a text indexKeywordTextField— have both a keyword and text indexKeywordTextStemmedField— have a keyword, text, and stemmed text index
The stemmed text index has been further analyzed for fuzzy matching through full text searches.
Date¶
Date fields are used for timestamps. We use the numeric milliseconds-since-the-epoch representation, so searches should be against such long numbers rather than string-formatted dates.
You will most commonly search these using term-level queries, in particular range queries.
In the SDKs, fields that have a date-based index will be one of these types:
NumericField— have only a numeric indexNumericRankField— have both a numeric index and are rank-boostable
Boolean¶
Boolean fields are used for content that can only be either true or false. You should only try to match these two values when searching on these attributes, using a term query.
In the SDKs, fields that have a boolean-based index will only ever be of the type BooleanField.
Float¶
Float fields are used for content that is numeric and includes decimals.
You will most commonly search these using term-level queries, in particular range queries.
There are non-obvious nuances to float fields
For example, the values +0.0 and -0.0 are considered different values. See the linked Elasticsearch documentation for full details of the nuances.
In the SDKs, fields that have a date-based index will be one of these types:
NumericField— have only a numeric indexNumericRankField— have both a numeric index and are rank-boostable
Rank feature¶
Rank feature fields are used to boost documents in search results.
You cannot directly query, sort, or aggregate on a rank feature field, but you can use them in rank feature queries to boost the relevance score of results based on the values of this field.
In the SDKs, fields that have a rank-boostable index will only ever be of the type NumericRankField.