Skip to content

Manage data products

Create a new data product

1.8.4 1.8.1

To create a new data product:

Create a data product
1
2
3
4
5
6
7
8
9
FluentSearch assets = Table.select() // (1)
    .where(Table.CERTIFICATE_STATUS.eq(CertificateStatus.VERIFIED))
    .where(Table.ATLAN_TAGS.eq("Marketing"))
    .build();
DataProduct dp = DataProduct.creator("Marketing Influence", // (2)
        DataDomain.refByQualifiedName("default/domain/marketing"), // (3)
        assets) // (4)
    .build(); // (5)
AssetMutationResponse response = dp.save(); // (6)
  1. When defining a data product, you must define the assets within it. These are defined through a search, so that the assets included can be automatically managed. In this example, we are selecting all verified tables that have a tag of Marketing.
  2. You must provide a human-readable name for your data product.
  3. You must also provide the domain in which the data product should exist.
  4. And finally the search that was defined earlier, to define which assets to include in the data product.
  5. You then need to build the object.
  6. And then you can save() the object you've built to create the new data product in Atlan.
Create a data product
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import DataProduct, Table
from pyatlan.model.fluent_search import CompoundQuery, FluentSearch
from pyatlan.model.enums import CertificateStatus

client = AtlanClient()

assets = (
  FluentSearch()
  .where(CompoundQuery.active_assets())
  .where(CompoundQuery.asset_type(Table))
  .where(Table.CERTIFICATE_STATUS.eq(CertificateStatus.VERIFIED))
  .where(Table.ATLAN_TAGS.eq("Marketing"))
).to_request()  # (1)

product = DataProduct.create(
  name="Marketing Influence",  # (2)
  assets=assets,  # (3)
  domain_qualified_name="default/domain/marketing"  # (4)
)

response = client.asset.save(product)  # (5)
  1. When defining a data product, you must define the assets within it. These are defined through a search, so that the assets included can be automatically managed. In this example, we are selecting all verified tables that have a tag of Marketing.
  2. You must provide a human-readable name for your data product.
  3. You must provide the index search request that was defined earlier, to define which assets to include in the data product.
  4. You must also provide the domain in which the data product should exist.
  5. And then you can save() the object you've built to create the new data product in Atlan.
Create a data product
1
2
3
4
5
6
7
8
9
val assets = Table.select() // (1)
    .where(Table.CERTIFICATE_STATUS.eq(CertificateStatus.VERIFIED))
    .where(Table.ATLAN_TAGS.eq("Marketing"))
    .build()
val dp = DataProduct.creator("Marketing Influence",  // (2)
        DataDomain.refByQualifiedName("default/domain/marketing"),  // (3)
        assets) // (4)
    .build() // (5)
val response = dp.save() // (6)
  1. When defining a data product, you must define the assets within it. These are defined through a search, so that the assets included can be automatically managed. In this example, we are selecting all verified tables that have a tag of Marketing.
  2. You must provide a human-readable name for your data product.
  3. You must also provide the domain in which the data product should exist.
  4. And finally the search that was defined earlier, to define which assets to include in the data product.
  5. You then need to build the object.
  6. And then you can save() the object you've built to create the new data product in Atlan.
POST /api/meta/entity/bulk
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
{
  "entities": [
    {
      "typeName": "DataProduct", // (1)
      "attributes": {
        "name": "Marketing Influence", // (2)
        "qualifiedName": "default/product/marketingInfluence", // (3)
        "dataProductAssetsDSL": "{\"query\": {\"attributes\": [\"schemaName\", \"databaseName\"], \"dsl\": {\"from\": 0, \"size\": 100, \"aggregations\": {}, \"track_total_hits\": true \"post_filter\": {\"term\": {\"databaseName.keyword\": {\"value\": \"ATLAN_SAMPLE_DATA\"}}}, \"query\": {\"term\": {\"__typeName.keyword\": {\"value\": \"Schema\"}}}, \"sort\": [{\"__guid\": {\"order\": \"asc\"}}]}, \"relationAttributes\": []}}"
          // (4)
      },
      "relationshipAttributes": {
        "dataDomain": {
          "typeName": "DataDomain",
          "guid": "218c8144-dc39-43a5-b0c0-9eeb4d11e74a" // (5)
        },
      }
    }
  ]
}
  1. The typeName must be exactly DataProduct.
  2. Human-readable name for your data product.
  3. You must provide a qualifiedName, following the pattern: default/product/<lowerCamelCaseName>.
  4. You must provide the DSL that defines the assets to include in the data product, as an embedded JSON string.
  5. You must specify the relationship for the data domain the product exists within.

Retrieve a data product

1.8.4 1.8.1

To retrieve a data product, use its qualified name:

Retrieve a data product by its qualifiedName
1
DataProduct dp = DataProduct.get("default/product/marketingInfluence"); // (1)
  1. If no exception is thrown, the returned object will be a non-null data domain.
Retrieve an data product by its qualifiedName
1
2
3
4
5
6
7
8
9
from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import DataProduct

client = AtlanClient()

product = client.asset.get_by_qualified_name(
    asset_type=DataProduct,  # (1)
    qualified_name="default/product/marketingInfluence"
)
  1. If no exception is thrown, the returned object will be non-null and of the type requested.
Retrieve a data product by its qualifiedName
1
val dp = DataProduct.get("default/product/marketingInfluence") // (1)
  1. If no exception is thrown, the returned object will be a non-null data domain.
GET /api/meta/entity/uniqueAttribute/type/DataProduct/attr:qualifiedName=default/product/marketingInfluence&ignoreRelationships=false&minExtInfo=false
1
// (1)
  1. In the case of retrieving an asset, all necessary information is included in the URL of the request. There is no payload for the body of the request.

Update a data product

1.8.4 1.8.1

To update a data product:

Update a data product
1
2
3
4
5
DataProduct dp = DataProduct.updater("default/product/marketingInfluence", // (1)
        "Marketing Influence")
    .userDescription("Now with a description!") // (2)
    .build(); // (3)
AssetMutationResponse response = dp.save(); // (4)
  1. Use the updater() method to update a data product, providing the qualifiedName and name of the data product.
  2. You can chain onto the updater any other enrichment, for example changing the product's description.
  3. You then need to build the object.
  4. You can then save() the object you've built to update the data product in Atlan.
Update a data product
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
from pyatlan.client.atlan import AtlanClient
from pyatlan.model.assets import DataProduct

client = AtlanClient()

product = DataProduct.create_for_modification( # (1)
    qualified_name="default/product/marketingInfluence", # (2)
    name="Marketing Influence", # (3)
)
product.user_description = "Now with a description!" # (4)

response = client.asset.save(product) # (5)
  1. Use the create_for_modification() method to update a data product.
  2. You must provide the qualifiedName of the data product.
  3. You must provide the name of the data product.
  4. You can then add on any other updates, such as changing the user description of the data product.
  5. To update the data product in Atlan, call the save() method with the object you've built.
Update a data product
1
2
3
4
5
val dp = DataProduct.updater("default/product/marketingInfluence", // (1)
        "Marketing Influence")
    .userDescription("Now with a description!") // (2)
    .build() // (3)
val response = dp.save() // (4)
  1. Use the updater() method to update a data product, providing the qualifiedName and name of the data product.
  2. You can chain onto the updater any other enrichment, for example changing the product's description.
  3. You then need to build the object.
  4. You can then save() the object you've built to update the data product in Atlan.
POST /api/meta/entity/bulk
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
{
  "entities": [
    {
      "typeName": "DataProduct", // (1)
      "attributes": {
        "name": "Marketing Influence", // (2)
        "qualifiedName": "default/product/marketingInfluence", // (3)
        "userDescription": "Now with a description!" // (4)
      }
    }
  ]
}
  1. The typeName must be exactly DataProduct.
  2. Human-readable name for your data product.
  3. You must provide the the qualifiedName of the data product to update.
  4. You can add on any other updates, such as changing the user description of the data product.

Delete a data product

1.8.4 1.8.1

Soft-delete (archive)

To soft-delete, or archive, a data product:

Delete a data product
1
AssetDeletionResponse response = DataProduct.delete("218c8144-dc39-43a5-b0c0-9eeb4d11e74a"); // (1)
  1. To archive a data product in Atlan, call the DataProduct.delete() method with the GUID of the data product.
Delete a data product
1
2
3
4
5
from pyatlan.client.atlan import AtlanClient

client = AtlanClient()

client.asset.delete_by_guid("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") # (1)
  1. To archive a data product in Atlan, call the asset.delete_by_guid() method with the GUID of the data product.
Delete a data product
1
val response = DataProduct.delete("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") // (1)
  1. To archive a data product in Atlan, call the DataProduct.delete() method with the GUID of the data product.
DELETE /api/meta/entity/bulk?guid=218c8144-dc39-43a5-b0c0-9eeb4d11e74a&deleteType=SOFT
1
// (1)
  1. All the details for deleting the data product are specified in the URL directly. Note that you must provide the GUID of the data product to delete it.

Hard-delete (purge)

To permanently delete (purge) a data product:

Delete a data product
1
AssetDeletionResponse response = DataProduct.purge("218c8144-dc39-43a5-b0c0-9eeb4d11e74a"); // (1)
  1. To permanently delete a data product in Atlan, call the DataProduct.purge() method with the GUID of the data product.
Delete a product
1
2
3
4
5
from pyatlan.client.atlan import AtlanClient

client = AtlanClient()

client.asset.purge_by_guid("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") # (1)
  1. To permanently delete a data product in Atlan, call the asset.purge_by_guid() method with the GUID of the data product.
Delete a data product
1
val response = DataProduct.purge("218c8144-dc39-43a5-b0c0-9eeb4d11e74a") // (1)
  1. To permanently delete a data product in Atlan, call the DataProduct.purge() method with the GUID of the data product.
DELETE /api/meta/entity/bulk?guid=218c8144-dc39-43a5-b0c0-9eeb4d11e74a&deleteType=PURGE
1
// (1)
  1. All the details for deleting the data product are specified in the URL directly. Note that you must provide the GUID of the data product to delete it.