Create custom metadata¶
Like other objects in the SDK that you can create, custom metadata implements the builder pattern. This allows you to progressively build-up the structure you want to create.
There are limits to the number of custom metadata properties you can create
Atlan currently preserves details of custom metadata in its audit log. This allows Atlan to retain an audit trail of actions users took on custom metadata on each asset, even if the custom metadata definition itself is deleted.
However, this also places an upper limit on the number of custom metadata properties you can (structurally) define in Atlan. Even if you delete the custom metadata definitions, any that you have previously defined will still take up "space" within this limit.
More details
By default this is ~1000 properties. Note that this limit applies only to the structural definition of the properties themselves, not the values captured for assets. If you see an error like the following, it means you have reached this limit:
{
"errorCode": "ATLAS-500-00-001",
"errorMessage": "Unable to push entity audits to ES",
"errorCause": "[{type=mapper_parsing_exception, reason=failed to parse, caused_by={type=illegal_argument_exception, reason=Limit of total fields [1000] has been exceeded while adding new fields [5]}}]"
}
You will need to contact Atlan support to extend this threshold if you reach it.
Build minimal object needed¶
For example, to create a custom metadata structure to capture RACI assignments:
| Build custom metadata definition for creation | |
|---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 | |
- When creating the custom metadata structure, you must provide a name (
RACIin this example). - You can then add as many attributes to that structure as you want.
- Each attribute must have a name.
- Each attribute must have a type.
- If the type is
AtlanCustomAttributePrimitiveType.OPTIONSthen you must also specify the enumeration that defines the valid values for this attribute (in this example the type is not an enumeration, so this isnulland could even be left out as in the subsequent lines). - You must also specify whether the attribute allows multiple values to be captured on it (
true) or only a single value (false). - You can also provide a custom logo for the custom metadata by providing a URL to an image. The second argument controls whether this custom metadata is editable via the UI — for
falseit is editable via the UI, while fortruethe custom metadata will only be editable via APIs (including via SDK). - Or you can use an emoji as the custom icon for the custom metadata.
- Or you can use a built-in icon for the custom metadata. The second argument controls the color to use for the icon.
- As with all other builder patterns, you must
build()the object you've defined.
| Build custom metadata definition for creation | |
|---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | |
- When creating the custom metadata structure, you must provide a name (
RACIin this example). - You can then add as many attributes to that structure as you want.
- Each attribute must have a name.
- Each attribute must have a type.
- If the type is
AtlanCustomAttributePrimitiveType.OPTIONSthen you must also specify the enumeration that defines the valid values for this attribute (in this example none are enumerations, so this is the default value for the argument:None). - You can also specify whether the attribute allows multiple values to be captured on it (
True) or only a single value (False) (the default). - You can also provide a custom logo for the custom metadata by providing a URL to an image. The second argument controls whether this custom metadata is editable via the UI — for
locked=Falseit is editable via the UI, while forlocked=Truethe custom metadata will only be editable via APIs (including via SDK). - Or you can use an emoji as the custom icon for the custom metadata.
| Build custom metadata definition for creation | |
|---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 | |
- When creating the custom metadata structure, you must provide a name (
RACIin this example). - You can then add as many attributes to that structure as you want.
- Each attribute must have a name.
- Each attribute must have a type.
- If the type is
AtlanCustomAttributePrimitiveType.OPTIONSthen you must also specify the enumeration that defines the valid values for this attribute (in this example the type is not an enumeration, so this isnulland could even be left out as in the subsequent lines). - You must also specify whether the attribute allows multiple values to be captured on it (
true) or only a single value (false). - You can also provide a custom logo for the custom metadata by providing a URL to an image. The second argument controls whether this custom metadata is editable via the UI — for
falseit is editable via the UI, while fortruethe custom metadata will only be editable via APIs (including via SDK). - Or you can use an emoji as the custom icon for the custom metadata.
- Or you can use a built-in icon for the custom metadata. The second argument controls the color to use for the icon.
- As with all other builder patterns, you must
build()the object you've defined.
| POST /api/meta/types/typedefs | |
|---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | |
- All custom metadata structural definitions must be specified within the
businessMetadataDefsarray. - Each structural definition must be defined with a category set to
BUSINESS_METADATA. - Whatever name you provide for the structural definition will be replaced by a hashed-string generated name by the back-end.
- Within the structural definition, you need to define each attribute inside the
attributeDefsarray. - You should leave the
nameof the attribute as an empty string. This will be generated by the back-end. - Instead, provide the name for the attribute (as it should appear in the UI) to the
displayName. - There are various properties of each attribute that define how the attribute is validated and will behave in the UI. You can use the
multiValueSelectto specify whether to allow multiple values for this attribute on a single asset (in this example multiple values will be allowed). - One of the properties that must be specified is the type of the custom attribute. In this example, we use an Atlan-specific custom type to capture users.
- For each attribute that is to be newly created and added to the structure, set the
isNewtotrue. - Specify the name of the custom metadata structure, as it should appear in the UI, to the
displayName. - Finally, set options on the custom metadata structure to control its appearance — specifically the icon that should be used in the UI.
Create the custom metadata from the object¶
Now that the object is built, this customMetadataDef object will have the required information for Atlan to create it.
You can then actually create the custom metadata definition in Atlan by calling the create() method on the object itself:
| Create the custom metadata definition | |
|---|---|
14 | |
- The
create()operation will actually create the custom metadata definition within Atlan, including all the attributes that were defined as part of it.
| Create the custom metadata definition | |
|---|---|
33 34 35 36 | |
- The
typedef.create()operation will actually create the custom metadata definition within Atlan, including all the attributes that were defined as part of it.
| Create the custom metadata definition | |
|---|---|
14 | |
- The
create()operation will actually create the custom metadata definition within Atlan, including all the attributes that were defined as part of it.
Creation implicit in step above
The actual creation of the custom metadata structure is implicit in the example above.
Now that the custom metadata structure has been created, you can set its values per asset.
Limit applicability of an attribute¶
You can also limit the assets the custom metadata applies to in Atlan. Anywhere you create an attribute definition, you can:
| Limit applicability of an attribute | |
|---|---|
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
- We still recommend creating the attribute using the
Attribute.of()factory method. This ensures all required settings are configured based on the type of the attribute. - You can then clone the attribute into a mutable form using
toBuilder(). - Set the
optionson this clone to change its applicability. You can use thetoBuilder()on the options themselves to get a mutable clone of the options that have already been setup by theAttributeDef.of()factory method. -
By default, the
AttributeDef.of()method will ensure a custom metadata attribute applies to all assets. To limit its applicability, you need to remove these "grants" by clearing out:- Connections the custom metadata attribute applies to (by default, all assets in all connections that existed when the attribute was created will be capable of using this custom metadata attribute).
- Asset types the custom metadata attribute applies to (by default, all asset types will be capable of using this custom metadata attribute).
- Glossaries the custom metadata attribute applies to (by default, all objects in a glossary that existed when the attribute was created will be capable of using this custom metadata attribute).
- Glossary asset types the custom metadata applies to (by default, glossaries, terms and categories will be capable of using this custom metadata attribute).
-
You can chain any number of
applicableConnection()calls to specify thequaliedNames of connections. The custom metadata attribute will only be available to assets within these connections.To use all connections
To select all connections, instead chain
.applicableConnections(Connection.getAllQualifiedNames()). -
You can chain any number of
applicableAssetType()calls to specify the types of assets for the custom metadata attribute. The custom metadata attribute will only be available to assets of these types, within the connections specified in the line above.To use all asset types
To select all asset types, instead chain
.applicableAssetTypes(AttributeDefOptions.ALL_ASSET_TYPES). -
You can chain any number of
applicableGlossary()calls to specify thequalifiedNames of glossaries. The custom metadata attribute will only be available to assets within these glossaries.To use all glossaries
To select all glossaries, instead chain
.applicableGlossaries(Glossary.getAllQualifiedNames()). -
You can chain any number of
applicableGlossaryType()calls to specify the types of glossary assets for the custom metadata attribute. The custom metadata attribute will only be available to glossary assets of these types, within the glossaries specified in the line above.To use all glossary asset types
To select all glossary asset types, instead use
.applicableGlossaryTypes(AttributeDefOptions.ALL_GLOSSARY_TYPES). -
You then need to build all of these options.
- And finally you need to build the changes back into the attribute definition. You can then use the attribute definition (
responsiblein this example) as you would any other attribute definition, for example passing it to the chained.attributeDef()as part ofCustomMetadataDef.creator()shown earlier.
Coming soon
| Limit applicability of an attribute | |
|---|---|
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | |
- We still recommend creating the attribute using the
Attribute.of()factory method. This ensures all required settings are configured based on the type of the attribute. - You can then clone the attribute into a mutable form using
toBuilder(). - Set the
optionson this clone to change its applicability. You can use thetoBuilder()on the options themselves to get a mutable clone of the options that have already been setup by theAttributeDef.of()factory method. -
By default, the
AttributeDef.of()method will ensure a custom metadata attribute applies to all assets. To limit its applicability, you need to remove these "grants" by clearing out:- Connections the custom metadata attribute applies to (by default, all assets in all connections that existed when the attribute was created will be capable of using this custom metadata attribute).
- Asset types the custom metadata attribute applies to (by default, all asset types will be capable of using this custom metadata attribute).
- Glossaries the custom metadata attribute applies to (by default, all objects in a glossary that existed when the attribute was created will be capable of using this custom metadata attribute).
- Glossary asset types the custom metadata applies to (by default, glossaries, terms and categories will be capable of using this custom metadata attribute).
-
You can chain any number of
applicableConnection()calls to specify thequaliedNames of connections. The custom metadata attribute will only be available to assets within these connections.To use all connections
To select all connections, instead chain
.applicableConnections(Connection.getAllQualifiedNames()). -
You can chain any number of
applicableAssetType()calls to specify the types of assets for the custom metadata attribute. The custom metadata attribute will only be available to assets of these types, within the connections specified in the line above.To use all asset types
To select all asset types, instead chain
.applicableAssetTypes(AttributeDefOptions.ALL_ASSET_TYPES). -
You can chain any number of
applicableGlossary()calls to specify thequalifiedNames of glossaries. The custom metadata attribute will only be available to assets within these glossaries.To use all glossaries
To select all glossaries, instead chain
.applicableGlossaries(Glossary.getAllQualifiedNames()). -
You can chain any number of
applicableGlossaryType()calls to specify the types of glossary assets for the custom metadata attribute. The custom metadata attribute will only be available to glossary assets of these types, within the glossaries specified in the line above.To use all glossary asset types
To select all glossary asset types, instead use
.applicableGlossaryTypes(AttributeDefOptions.ALL_GLOSSARY_TYPES). -
You then need to build all of these options.
- And finally you need to build the changes back into the attribute definition. You can then use the attribute definition (
responsiblein this example) as you would any other attribute definition, for example passing it to the chained.attributeDef()as part ofCustomMetadataDef.creator()shown earlier.
Coming soon
Applicability is combined across connections and glossaries
If you specify both connections (and asset types) and glossaries (and glossary types), then the custom metadata attribute will be available across both data assets in those connections and glossary objects in those glossaries. In other words, these options are not mutually exclusive, but are combined.