Traverse lineage¶
Retrieve lineage¶
To fetch lineage, you need to request lineage from Atlan from a particular starting point:
Retrieve lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 |
|
- Build a request for lineage with the starting point for your lineage retrieval (the GUID of an asset). If you already have an asset, you can also instead run
requestLineage()
on the asset to directly build the same request. - You can specify how far you want lineage to be fetched using
depth()
. A depth of1
will only fetch immediate upstream and downstream assets, while2
will also fetch the immediate upstream and downstream assets of those assets, and so on. The default value of1000000
will fetch upstream and downstream assets up to 1,000,000 hops away (basically all lineage). - You can fetch only upstream assets or only downstream assets. In the list API, you cannot access both directions at the same time.
- You can specify how many results to include per page of results (defaults to 10).
- You can also specify any extra attributes you want to include in each asset in the resulting list.
- To include details about which asset is upstream and downstream of which other asset, set
immediateNeighbors
totrue
. (Without this, all downstream assets will be listed in breadth-first order, but you will not know specifically which asset is downstream of which other asset.) - You can then directly stream the results from the request. These will be lazily-fetched and paged automatically.
- A normal Java
Stream
is created, so you can apply any stream-based operations to it (filtering, mapping, collecting, or doing something for each result as in this example).
Retrieve lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
- Build a request for lineage by specifying the parameters on the constructor.
- The starting point for lineage must be the GUID of an asset.
- You can specify how far you want lineage to be fetched using
depth
. A depth of1
will only fetch immediate upstream and downstream assets, while2
will also fetch the immediate upstream and downstream assets of those assets, and so on. The default value of1000000
will fetch upstream and downstream assets up to 1,000,000 hops away (basically all lineage). - You can fetch only upstream assets or only downstream assets. In the list API, you cannot access both directions at the same time.
- You can specify how many results to include per page of results (defaults to 10).
- You can also specify any extra attributes you want to include in each asset in the resulting list.
- The
immediate_neighbors
parameter, when set toTrue
, includes direct upstream and downstream connections for each asset, enabling detailed lineage traversal. - Call the
asset.get_lineage_list()
method using therequest
to actually retrieve the lineage details from Atlan. - Iterate through the results
Retrieve lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 |
|
- Build a request for lineage with the starting point for your lineage retrieval (the GUID of an asset). If you already have an asset, you can also instead run
requestLineage()
on the asset to directly build the same request. - You can specify how far you want lineage to be fetched using
depth()
. A depth of1
will only fetch immediate upstream and downstream assets, while2
will also fetch the immediate upstream and downstream assets of those assets, and so on. The default value of1000000
will fetch upstream and downstream assets up to 1,000,000 hops away (basically all lineage). - You can fetch only upstream assets or only downstream assets. In the list API, you cannot access both directions at the same time.
- You can specify how many results to include per page of results (defaults to 10).
- You can also specify any extra attributes you want to include in each asset in the resulting list.
- To include details about which asset is upstream and downstream of which other asset, set
immediateNeighbors
totrue
. (Without this, all downstream assets will be listed in breadth-first order, but you will not know specifically which asset is downstream of which other asset.) - You can then directly stream the results from the request. These will be lazily-fetched and paged automatically.
- A normal Java
Stream
is created, so you can apply any stream-based operations to it (filtering, mapping, collecting, or doing something for each result as in this example).
POST /api/meta/lineage/list | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
- The starting point for lineage must be the GUID of an asset.
- You can specify how far you want lineage to be fetched using
depth()
. A depth of1
will only fetch immediate upstream and downstream assets, while2
will also fetch the immediate upstream and downstream assets of those assets, and so on. A value of1000000
will fetch upstream and downstream assets up to 1,000,000 hops away (basically all lineage). - You can fetch only upstream assets (
INPUT
) or only downstream assets (OUTPUT
). In the list API, you cannot access both directions at the same time. - You can specify the starting point for a page of results (you must provide a value:
0
will start at the first result). - You can specify how many results to include per page of results (you must provide a value: we suggest starting at
10
). - You can also specify any extra attributes you want to include in each asset in the resulting list.
Traverse lineage¶
The new lineage list API returns results in breadth-first order. So you can traverse the lineage by progressing through the result list in the order they are returned, even across multiple pages of results.
Downstream assets¶
To traverse downstream assets in lineage:
Traverse downstream lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
- Specify the GUID of an asset for the starting point. (Or from an asset itself, use
requestLineage()
to start the same builder.) - Request the
DOWNSTREAM
direction. - If you want to understand specifically which assets are downstream from which other assets, set
immediateNeighbors
totrue
. - You can then stream the results from the request. The pages will be lazily-fetched in the background, as-needed.
- With streams, you can apply additional filters over the results (in this example any processes in the results will be ignored).
- With streams, you can also limit the total number of results you want to process — independently from page size of retrievals. With lazy-fetching of the results, this will ensure you only retrieve the number of pages required to complete the stream.
- Of course, you still need to actually do something with those remaining results.
- If
immediateNeighbors
istrue
, each asset will have a list of downstream lineage references populated in.getImmediateDownstream()
. - You can, for example, retrieve the GUID of each of these downstream references to see the assets that are immediately downstream from the asset you are iterating through in the lineage results.
Traverse downstream lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
- Specify the GUID of an asset for the starting point.
- Request the
DOWNSTREAM
direction. - If you want to understand specifically which assets are downstream from which other assets, set
immediate_neighbors
toTrue
. - Call the
get_lineage_list
method using therequest
to get the results - You can then iterate through all of the results. The pages will be lazily-fetched in the background as-needed, and each result looped through.
- Do something with the result.
- If
immediate_neighbors
isTrue
, each asset will have a list of downstream lineage references populated in.immediate_downstream
. - You can, for example, retrieve the GUID of each of these downstream references to see the assets that are immediately downstream from the asset you are iterating through in the lineage results.
Traverse downstream lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
- Specify the GUID of an asset for the starting point. (Or from an asset itself, use
requestLineage()
to start the same builder.) - Request the
DOWNSTREAM
direction. - If you want to understand specifically which assets are downstream from which other assets, set
immediateNeighbors
totrue
. - You can then stream the results from the request. The pages will be lazily-fetched in the background, as-needed.
- With streams, you can apply additional filters over the results (in this example any processes in the results will be ignored).
- With streams, you can also limit the total number of results you want to process — independently from page size of retrievals. With lazy-fetching of the results, this will ensure you only retrieve the number of pages required to complete the stream.
- Of course, you still need to actually do something with those remaining results.
- If
immediateNeighbors
istrue
, each asset will have a list of downstream lineage references populated in.getImmediateDownstream()
. - You can, for example, retrieve the GUID of each of these downstream references to see the assets that are immediately downstream from the asset you are iterating through in the lineage results.
POST /api/meta/lineage/list | |
---|---|
1 2 3 4 5 6 7 8 9 |
|
- Specify the GUID of an asset for the starting point.
- Request the
OUTPUT
(downstream) direction.
Upstream assets¶
To traverse upstream assets in lineage:
Traverse upstream lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
- Specify the GUID of an asset for the starting point. (Or from an asset itself, use
requestLineage()
to start the same builder.) - Request the
UPSTREAM
direction. - If you want to understand specifically which assets are upstream from which other assets, set
immediateNeighbors
totrue
. - You can then stream the results from the request. The pages will be lazily-fetched in the background, as-needed.
- With streams, you can apply additional filters over the results (in this example any processes in the results will be ignored).
- With streams, you can also limit the total number of results you want to process — independently from page size of retrievals. With lazy-fetching of the results, this will ensure you only retrieve the number of pages required to complete the stream.
- Of course, you still need to actually do something with those remaining results.
- If
immediateNeighbors
istrue
, each asset will have a list of upstream lineage references populated in.getImmediateUpstream()
. - You can, for example, retrieve the GUID of each of these upstream references to see the assets that are immediately upstream from the asset you are iterating through in the lineage results.
Traverse upstream lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
- Specify the GUID of an asset for the starting point.
- Request the
UPSTREAM
direction. - If you want to understand specifically which assets are upstream from which other assets, set
immediate_neighbors
toTrue
. - Call the
get_lineage_list
method using therequest
to get the results - You can then iterate through all of the results. The pages will be lazily-fetched in the background as-needed, and each result looped through.
- Do something with the result.
- If
immediate_neighbors
isTrue
, each asset will have a list of upstream lineage references populated in.immediate_upstream
. - You can, for example, retrieve the GUID of each of these upstream references to see the assets that are immediately upstream from the asset you are iterating through in the lineage results.
Traverse upstream lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
- Specify the GUID of an asset for the starting point. (Or from an asset itself, use
requestLineage()
to start the same builder.) - Request the
UPSTREAM
direction. - If you want to understand specifically which assets are upstream from which other assets, set
immediateNeighbors
totrue
. - You can then stream the results from the request. The pages will be lazily-fetched in the background, as-needed.
- With streams, you can apply additional filters over the results (in this example any processes in the results will be ignored).
- With streams, you can also limit the total number of results you want to process — independently from page size of retrievals. With lazy-fetching of the results, this will ensure you only retrieve the number of pages required to complete the stream.
- Of course, you still need to actually do something with those remaining results.
- If
immediateNeighbors
istrue
, each asset will have a list of upstream lineage references populated in.getImmediateUpstream()
. - You can, for example, retrieve the GUID of each of these upstream references to see the assets that are immediately upstream from the asset you are iterating through in the lineage results.
POST /api/meta/lineage/list | |
---|---|
1 2 3 4 5 6 7 8 9 |
|
- Specify the GUID of an asset for the starting point.
- Request the
INPUT
(upstream) direction.
Filter lineage¶
You can also filter the information fetched through lineage. This can help improve performance of your code by limiting the results it will fetch to only those you require.
Retrieve only active assets
In most cases for lineage you only care about active assets. By filtering to only active assets, you can improve the performance of lineage retrieval by as much as 10x. (The new FluentLineage
interface will do this automatically, unless you explicitly request the inclusion of archived assets.)
Not possible to filter by custom metadata
You currently cannot filter lineage based on the values of custom metadata.
Limit assets in response¶
You can limit the assets you will see in the response through entity filters. These restrict what assets will be included in the results, but still traverse all of the lineage:
Limit assets in response | |
---|---|
1 2 3 4 5 6 |
|
- Build the request as you would above, or request it directly from an asset. Because this operation will directly request lineage for the asset in Atlan, you must provide it an
AtlanClient
through which to connect to the tenant. -
Add one or more
includeInResults
to the request before sending it to Atlan. Each of these defines criteria for which assets should be filtered for inclusion in the results, in this example only assets with a verified certificate will be included. The criterion itself is composed of:- The field by which you want to filter (
Asset.CERTIFICATE_STATUS
in this example). - A fixed member within that field that builds lineage filters, called
.inLineage
. - The operator you want to use to compare values for that field in order to determine whether or not an asset matches (
.eq()
in this example). - The value you want to compare against using that operator (
CertificateStatus.VERIFIED
in this example).
- The field by which you want to filter (
-
Optionally, you can use
includesCondition
in your lineage request to specify whether theincludeInResults
criteria should be combined with AND (default) or if any matching is sufficient (OR). - When you then stream the results, only those assets that match the filter criteria will be included in the response.
- You can then collect them (standard stream operation) to give a complete list, across pages, of those assets that match the criteria.
Limit assets in response | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
- Build the request as you would above, or request it directly from an asset.
-
Add one or more
include_in_results
to the request before sending it to Atlan. Each of these defines criteria for which assets should be filtered for inclusion in the results, in this example only assets with a verified certificate will be included. The criterion itself is composed of:- The field by which you want to filter (
Asset.CERTIFICATE_STATUS
in this example). - A fixed member within that field that builds lineage filters, called
.in_lineage
. - The operator you want to use to compare values for that field in order to determine whether or not an asset matches (
.eq()
in this example). - The value you want to compare against using that operator (
CertificateStatus.VERIFIED
in this example).
- The field by which you want to filter (
-
Optionally, you can use
.includes_condition
in your lineage request to specify whether theincludes_in_results
criteria should be combined with AND (default) or if any matching is sufficient (OR). - Use the
request
to get theresponse
that can be used to iterate through the assets. - Iterate through the assets and use them as you will.
Limit assets in response | |
---|---|
1 2 3 4 5 6 |
|
- Build the request as you would above, or request it directly from an asset. Because this operation will directly request lineage for the asset in Atlan, you must provide it an
AtlanClient
through which to connect to the tenant. -
Add one or more
includeInResults
to the request before sending it to Atlan. Each of these defines criteria for which assets should be filtered for inclusion in the results, in this example only assets with a verified certificate will be included. The criterion itself is composed of:- The field by which you want to filter (
Asset.CERTIFICATE_STATUS
in this example). - A fixed member within that field that builds lineage filters, called
.inLineage
. - The operator you want to use to compare values for that field in order to determine whether or not an asset matches (
.eq()
in this example). - The value you want to compare against using that operator (
CertificateStatus.VERIFIED
in this example).
- The field by which you want to filter (
-
Optionally, you can use
includesCondition
in your lineage request to specify whether theincludeInResults
criteria should be combined with AND (default) or if any matching is sufficient (OR). - When you then stream the results, only those assets that match the filter criteria will be included in the response.
- You can then collect them (standard stream operation) to give a complete list, across pages, of those assets that match the criteria.
POST /api/meta/lineage/list | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
- Build the request as you would above, but add
entityFilters
to the request before sending it to Atlan. -
You can specify any number of criteria to include in your filters, and whether they should all apply (condition of
AND
) or only any one of them need apply (condition ofOR
) for a resulting asset to be included. Each filter criterion is a combination of:- The name of the field in Atlan to use for filtering the results
- The value of that field that should be compared against for filtering
- An operator that defines how that comparison should be done to be considered a match
Limit lineage traversal¶
You can also limit how much of the lineage is traversed. You can do this both at an asset-level and a relationship-level:
Limit lineage traversal | |
---|---|
1 2 3 4 5 6 7 8 |
|
-
Provide your conditions to the
whereAsset
andwhereRelationship
of the request. This will ensure that once an asset (or relationship) is found in lineage traversal that does not match the conditions, further lineage traversal beyond that asset (or relationship) will not be done.In this example, that means that once we hit an archived or soft-deleted asset (or relationship) in the lineage, we will not look for any further downstream lineage from that archived or soft-deleted asset (or relationship). (In other words, we will limit the lineage results to only active assets by short-circuiting traversal when we hit an archived or soft-deleted asset or relationship.)
FluentLineage.ACTIVE constant
Note that the
FluentLineage.ACTIVE
example here is a predefined filter constant. If you look at its code, it is equivalent to writing any other lineage filter:Asset.STATUS.inLineage.eq(AtlanStatus.ACTIVE)
When you request lineage directly on an asset, as in the example above, by default only active assets and relationships are included. (In other words, the filters by
FluentLineage.ACTIVE
are applied by default when using theAsset.lineage()
request style.) -
Optionally, you can use
assetsCondition
in your lineage request to specify whether thewhereAsset
criteria should be combined with AND (default) or if any matching is sufficient (OR). -
Optionally, you can use
relationshipsCondition
in your lineage request to specify whether thewhereRelationship
criteria should be combined with AND (default) or if any matching is sufficient (OR). -
When you then fetch the results and iterate through them, not only are those assets that match the filter criteria the only ones included in the response, but the traversal is likely to run significantly faster as well by entirely skipping any further downstream traversal through the assets that do not match.
-
You can continue to process the results from there as you would with any stream: filtering, mapping, running something for each result, or in this example collecting them into a list.
Limit lineage traversal | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
-
Provide your conditions to the
where_assets
andwhere_relationships
of FluentLineage. This will ensure that once an asset (or relationship) is found in lineage traversal that does not match the conditions, further lineage traversal beyond that asset (or relationship) will not be done.In this example, that means that once we hit an archived or soft-deleted asset (or relationship) in the lineage, we will not look for any further downstream lineage from that archived or soft-deleted asset (or relationship). (In other words, we will limit the lineage results to only active assets by short-circuiting traversal when we hit an archived or soft-deleted asset or relationship.)
FluentLineage.ACTIVE constant
Note that the
FluentLineage.ACTIVE
example here is a predefined filter constant. If you look at its code, it is equivalent to writing any other lineage filter:Asset.STATUS.in_lineage.eq(EntityStatus.ACTIVE)
When you request lineage directly on an asset, as in the example above, by default only active assets and relationships are included. (In other words, the filters by
FluentLineage.ACTIVE
are applied by default when using theAsset.lineage()
request style.) -
Optionally, you can use
assets_condition
in your lineage request to specify whether thewhere_assets
criteria should be combined with AND (default) or if any matching is sufficient (OR). -
Optionally, you can use
relationships_condition
in your lineage request to specify whether thewhere_relationships
criteria should be combined with AND (default) or if any matching is sufficient (OR). -
Use the
request
to get theresponse
that can be used to iterate through the assets. -
Iterate through the assets and use them as you will.
Limit lineage traversal | |
---|---|
1 2 3 4 5 6 7 8 |
|
-
Provide your conditions to the
whereAsset
andwhereRelationship
of the request. This will ensure that once an asset (or relationship) is found in lineage traversal that does not match the conditions, further lineage traversal beyond that asset (or relationship) will not be done.In this example, that means that once we hit an archived or soft-deleted asset (or relationship) in the lineage, we will not look for any further downstream lineage from that archived or soft-deleted asset (or relationship). (In other words, we will limit the lineage results to only active assets by short-circuiting traversal when we hit an archived or soft-deleted asset or relationship.)
FluentLineage.ACTIVE constant
Note that the
FluentLineage.ACTIVE
example here is a predefined filter constant. If you look at its code, it is equivalent to writing any other lineage filter:Asset.STATUS.inLineage.eq(AtlanStatus.ACTIVE)
When you request lineage directly on an asset, as in the example above, by default only active assets and relationships are included. (In other words, the filters by
FluentLineage.ACTIVE
are applied by default when using theAsset.lineage()
request style.) -
Optionally, you can use
assetsCondition
in your lineage request to specify whether thewhereAsset
criteria should be combined with AND (default) or if any matching is sufficient (OR). -
Optionally, you can use
relationshipsCondition
in your lineage request to specify whether thewhereRelationship
criteria should be combined with AND (default) or if any matching is sufficient (OR). -
When you then fetch the results and iterate through them, not only are those assets that match the filter criteria the only ones included in the response, but the traversal is likely to run significantly faster as well by entirely skipping any further downstream traversal through the assets that do not match.
-
You can continue to process the results from there as you would with any stream: filtering, mapping, running something for each result, or in this example collecting them into a list.
POST /api/meta/lineage/list | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
|
-
Provide your conditions to the
entityTraversalFilters
of the request. This will ensure that once an asset is found in lineage traversal that does not match the conditions, further lineage traversal beyond that asset will not be done.In this example, that means that once we hit an archived of soft-deleted asset in the lineage, we will not look for any further downstream lineage from that archived or soft-deleted asset. (In other words, we will limit the lineage results to only active assets by short-circuiting traversal when we hit an archived or soft-deleted asset.)
Limit asset details¶
You can also limit the details for each asset returned by lineage:
Limit asset details | |
---|---|
1 2 3 4 5 6 7 8 9 10 |
|
- Build the request as above, but chain as many
includeOnResults
as you like to specify the attributes you want to include on each asset in the lineage. - You can also decide whether to include or exclude Atlan tags and assigned business terms, but to do this you must first conver the fluent lineage request into a
LineageListRequest
. You can do this by chainingtoRequestBuilder()
. - You can then use
excludeAtlanTags(false)
to ensure that Atlan tags are included on each asset in lineage. - You can also use
excludeMeanings(false)
to ensure that assigned business terms are included on each asset in lineage. - You then need to call
fetch()
on theLineageListRequest
to actually run the lineage request. Because this operation will directly request lineage for the asset in Atlan, you must provide it anAtlanClient
through which to connect to the tenant. - You can then stream and further transform or collect the results from the request, directly.
Limit asset details | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
- Build the request as above, but chain as many
include_on_results
as you like to specify the attributes you want to include on each asset in the lineage. - You can use
exclude_atlan_tags(False)
to ensure that Atlan tags are included on each asset in lineage. - You can use
exclude_meanings(False)
to ensure that assigned business terms are included on each asset in lineage. - You then need to call
get_lineage_list()
with theLineageListRequest
to actually run the lineage request. - You can then iterate through and further transform or collect the results from the request.
Limit asset details | |
---|---|
1 2 3 4 5 6 7 8 9 10 |
|
- Build the request as above, but chain as many
includeOnResults
as you like to specify the attributes you want to include on each asset in the lineage. - You can also decide whether to include or exclude Atlan tags and assigned business terms, but to do this you must first conver the fluent lineage request into a
LineageListRequest
. You can do this by chainingtoRequestBuilder()
. - You can then use
excludeAtlanTags(false)
to ensure that Atlan tags are included on each asset in lineage. - You can also use
excludeMeanings(false)
to ensure that assigned business terms are included on each asset in lineage. - You then need to call
fetch()
on theLineageListRequest
to actually run the lineage request. Because this operation will directly request lineage for the asset in Atlan, you must provide it anAtlanClient
through which to connect to the tenant. - You can then stream and further transform or collect the results from the request, directly.
POST /api/meta/lineage/list | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
- Build the request as above, but add as many field names as you like to specify the attributes you want to include on each asset in the lineage.
- You can use
"excludeClassifications": false
to ensure that Atlan tags are included on each asset in lineage. - You can use
"excludeMeanings": false
to ensure that assigned business terms are included on each asset in lineage.
Original API¶
Deprecated and removed
The original lineage API was previously deprecated, and now no longer exists in the latest releases of the SDKs. It is slower, does not support paging, and will not receive any enhancements. We would therefore strongly recommend using the newer API (described above); however, the original API is described here for completeness.
Retrieve lineage (deprecated)
To fetch lineage, you need to request lineage from Atlan from a particular starting point:
Retrieve lineage | |
---|---|
1 2 3 4 5 6 7 8 |
|
- Build a
LineageRequest
to specify the starting point for your lineage retrieval. - The starting point for lineage must be the GUID of an asset.
-
You can specify how far you want lineage to be fetched using
depth()
. A depth of1
will only fetch immediate upstream and downstream assets, while2
will also fetch the immediate upstream and downstream assets of those assets, and so on. The default value of0
will fetch all upstream and downstream assets.If you expect extensive lineage, change the default!
The default value of
0
can result in a long-running API call with a very large response payload. If you expect your lineage to be extensive, you may want to try smaller depths first. -
You can fetch only upstream assets, only downstream assets, or lineage in both directions.
-
Decide whether to include processes in the response.
Use
true
if you want to use the SDK's traversal helpersCurrently the SDK's traversal logic only works when this is set to
true
. Unless you want to code your own traversal logic, sethideProcess
totrue
. -
If
allowDeletedProcess
is set totrue
andhideProcess
is set tofalse
then deleted (archived) processes will also be included in the response. - Build the request.
- Call the
fetch()
method to actually retrieve the lineage details from Atlan.
Retrieve lineage | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
- Build a
LineageRequest
to specify the starting point for your lineage retrieval. - The starting point for lineage must be the GUID of an asset.
-
You can specify how far you want lineage to be fetched using
depth
. A depth of1
will only fetch immediate upstream and downstream assets, while2
will also fetch the immediate upstream and downstream assets of those assets, and so on. The default value of0
will fetch all upstream and downstream assets.If you expect extensive lineage, change the default!
The default value of
0
can result in a long-running API call with a very large response payload. If you expect your lineage to be extensive, you may want to try smaller depths first. -
You can fetch only upstream assets, only downstream assets, or lineage in both directions.
-
Decide whether to include processes in the response.
Use
True
if you want to use the SDK's traversal helpersCurrently the SDK's traversal logic only works when this is set to
True
. Unless you want to code your own traversal logic, sethide_process
toTrue
. -
If
allow_deleted_process
is set toTrue
andhide_process
is set toFalse
then deleted (archived) processes will also be included in the response. - Call the
asset.get_lineage()
method to actually retrieve the lineage details from Atlan.
POST /api/meta/lineage/getlineage | |
---|---|
1 2 3 4 5 6 7 |
|
- The starting point for lineage must be the GUID of an asset.
-
You can specify how far you want lineage to be fetched using
depth
. A depth of1
will only fetch immediate upstream and downstream assets, while2
will also fetch the immediate upstream and downstream assets of those assets, and so on. The default value of0
will fetch all upstream and downstream assets.If you expect extensive lineage, change the default!
The default value of
0
can result in a long-running API call with a very large response payload. If you expect your lineage to be extensive, you may want to try smaller depths first. -
You can fetch only upstream assets, only downstream assets, or lineage in both directions.
- Decide whether to include processes in the response.
- If
allowDeletedProcess
is set totrue
andhideProcess
is set tofalse
then deleted (archived) processes will also be included in the response.
Traverse lineage (deprecated)
To assist with traversal of the lineage, the SDK provides some helper methods.
Downstream assets (deprecated)
To retrieve assets immediately downstream from the originally-requested asset:
Retrieve downstream assets | |
---|---|
9 10 11 |
|
- The
getDownstreamAssetGuids()
method will return the GUIDs of assets that are immediately downstream. - The
getDownstreamAssets()
method will return the asset objects for the assets that are immediately downstream. - The
getDownstreamProcessGuids()
method will return the GUIDs of the processes that run immediately downstream.
Retrieve downstream assets | |
---|---|
14 15 16 |
|
- The
get_downstream_asset_guids()
method will return the GUIDs of assets that are immediately downstream. - The
get_downstream_assets()
method will return the asset objects for the assets that are immediately downstream. - The
get_downstream_process_guids()
method will return the GUIDs of the processes that run immediately downstream.
POST /api/meta/lineage/getlineage | |
---|---|
1 2 3 4 5 |
|
- The starting point for lineage must be the GUID of an asset.
- A depth of
1
will only fetch immediate upstream and downstream assets. - A direction of
OUTPUT
will fetch only downstream assets.
Upstream assets (deprecated)
To retrieve assets immediately upstream from the originally-requested asset:
Retrieve upstream assets | |
---|---|
9 10 11 |
|
- The
getUpstreamAssetGuids()
method will return the GUIDs of assets that are immediately upstream. - The
getUpstreamAssets()
method will return the asset objects for the assets that are immediately upstream. - The
getUpstreamProcessGuids()
method will return the GUIDs of the processes that run immediately upstream.
Retrieve upstream assets | |
---|---|
14 15 16 |
|
- The
get_upstream_asset_guids()
method will return the GUIDs of assets that are immediately upstream. - The
get_upstream_assets()
method will return the asset objects for the assets that are immediately upstream. - The
get_upstream_process_guids()
method will return the GUIDs of the processes that run immediately upstream.
POST /api/meta/lineage/getlineage | |
---|---|
1 2 3 4 5 |
|
- The starting point for lineage must be the GUID of an asset.
- A depth of
1
will only fetch immediate upstream and downstream assets. - A direction of
INPUT
will fetch only upstream assets.
Depth-first traversal (deprecated)
You might want to traverse more than only the immediate upstream or downstream assets. To retrieve all assets that are downstream from the originally-requested asset, across multiple degrees of separation, using a depth-first search traversal:
Retrieve all downstream assets | |
---|---|
9 10 |
|
-
The
getAllDownstreamAssetGuidsDFS()
method will return the GUIDs of all assets that are downstream.The first GUID will always be the GUID of the asset used as the starting point for lineage, so even if there is no downstream lineage this will still return a list with a single GUID.
The traversal will be in depth-first order downstream. This means after the GUID for the starting point, the list will contain GUIDs of assets immediately downstream. These will be followed by the assets that are immediately downstream from those assets, and so on. (The deeper you get into the list, the further downstream you will be in lineage from the starting point.)
-
The
getAllDownstreamAssetsDFS()
method will return the asset objects for all assets that are downstream.The first asset object will always be the object for the asset used as the starting point for lineage, so even if there is no downstream lineage this will still return a list with a single asset.
The traversal will be in depth-first order downstream. This means after the asset for the starting point, the list will contain assets of assets immediately downstream. These will be followed by the assets that are immediately downstream from those assets, and so on. (The deeper you get into the list, the further downstream you will be in lineage from the starting point.)
Retrieve all upstream assets | |
---|---|
9 10 |
|
-
The
getAllUpstreamAssetGuidsDFS()
method will return the GUIDs of all assets that are upstream.The first GUID will always be the GUID of the asset used as the starting point for lineage, so even if there is no upstream lineage this will still return a list with a single GUID.
The traversal will be in depth-first order upstream. This means after the GUID for the starting point, the list will contain GUIDs of assets immediately upstream. These will be followed by the assets that are immediately upstream from those assets, and so on. (The deeper you get into the list, the further upstream you will be in lineage from the starting point.)
-
The
getAllUpstreamAssetsDFS()
method will return the asset objects for all assets that are upstream.The first asset object will always be the object for the asset used as the starting point for lineage, so even if there is no upstream lineage this will still return a list with a single asset.
The traversal will be in depth-first order upstream. This means after the asset for the starting point, the list will contain assets of assets immediately upstream. These will be followed by the assets that are immediately upstream from those assets, and so on. (The deeper you get into the list, the further upstream you will be in lineage from the starting point.)
Retrieve all downstream assets | |
---|---|
14 15 |
|
-
The
get_all_downstream_asset_guids_dfs()
method will return the GUIDs of all assets that are downstream.The first GUID will always be the GUID of the asset used as the starting point for lineage, so even if there is no downstream lineage this will still return a list with a single GUID.
The traversal will be in depth-first order downstream. This means after the GUID for the starting point, the list will contain GUIDs of assets immediately downstream. These will be followed by the assets that are immediately downstream from those assets, and so on. (The deeper you get into the list, the further downstream you will be in lineage from the starting point.)
-
The
get_all_downstream_assets_dfs()
method will return the asset objects for all assets that are downstream.The first asset object will always be the object for the asset used as the starting point for lineage, so even if there is no downstream lineage this will still return a list with a single asset.
The traversal will be in depth-first order downstream. This means after the asset for the starting point, the list will contain assets of assets immediately downstream. These will be followed by the assets that are immediately downstream from those assets, and so on. (The deeper you get into the list, the further downstream you will be in lineage from the starting point.)
Retrieve all upstream assets | |
---|---|
14 15 |
|
-
The
get_all_upstream_asset_guids_dfs()
method will return the GUIDs of all assets that are upstream.The first GUID will always be the GUID of the asset used as the starting point for lineage, so even if there is no upstream lineage this will still return a list with a single GUID.
The traversal will be in depth-first order upstream. This means after the GUID for the starting point, the list will contain GUIDs of assets immediately upstream. These will be followed by the assets that are immediately upstream from those assets, and so on. (The deeper you get into the list, the further upstream you will be in lineage from the starting point.)
-
The
get_all_upstream_assets_dfs()
method will return the asset objects for all assets that are upstream.The first asset object will always be the object for the asset used as the starting point for lineage, so even if there is no upstream lineage this will still return a list with a single asset.
The traversal will be in depth-first order upstream. This means after the asset for the starting point, the list will contain assets of assets immediately upstream. These will be followed by the assets that are immediately upstream from those assets, and so on. (The deeper you get into the list, the further upstream you will be in lineage from the starting point.)
Multiple API calls
You may either need to make multiple API calls using the approaches above, or retrieve all downstream lineage and in your code traverse the returned relationships.
More details on what all means here
Keep in mind that when we say "all" above we mean all assets that are found in the response. If you have modified your request parameters to limit the lineage (for example, through depth()
or direction()
) then this will only traverse what is found in the response — not necessarily all lineage in Atlan.