Packages and workflows introduction¶
Operations on workflows.
In Atlan, packages define the workflows you can run to retrieve metadata from various sources. Within this section the individual packages that are currently supported through the SDKs are listed with examples.
Workflows run asynchronously
This means the helper method to run a workflow will return immediately, before the workflow itself has finished running. If you want to wait until the workflow is finished you'll need to use other helper methods to check the status and wait accordingly.
Block until workflow completion¶
To block until the workflow has completed running:
Block until workflow has completed | |
---|---|
1 2 3 |
|
- Every package returns a
Workflow
object, from which you canrun()
the workflow. This call will return almost immediately with some metadata about the workflow run — it will not wait until the workflow has completed running. -
There is a
monitorStatus()
method on the response of a workflow run that you can use to wait until the workflow has completed. When this method finally returns, it will give the state of the workflow when it completed (for example, success or failure).The method comes in two variations:
- one that takes an slf4j logger (in this example) and will log its status periodically
- and another that takes no arguments and does not do any logging
Block until workflow has completed | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
- Each package returns a
Workflow
object, which you can subsequently pass to therun()
method of the workflow client. This call will return almost immediately with some metadata about the workflow run — it will not wait until the workflow has completed running. -
Use the
monitor()
method on the workflow client to wait until the workflow has completed. When this method returns, it provides the final state of the workflow, indicating whether it was successful or failed.The method comes in two variations:
- one that takes a logger (in this example) and will log its status periodically.
- and another that takes no arguments and does not do any logging.
Block until workflow has completed | |
---|---|
1 2 3 |
|
- Every package returns a
Workflow
object, from which you canrun()
the workflow. This call will return almost immediately with some metadata about the workflow run — it will not wait until the workflow has completed running. -
There is a
monitorStatus()
method on the response of a workflow run that you can use to wait until the workflow has completed. When this method finally returns, it will give the state of the workflow when it completed (for example, success or failure).The method comes in two variations:
- one that takes an slf4j logger (in this example) and will log its status periodically
- and another that takes no arguments and does not do any logging
Update workflow source credentials¶
To update workflow source credentials for example, for Snowflake:
Update workflow source credentials | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
- You can retrieve the workflow credential object by providing its
GUID
. - You must specify the authentication type of the credential.
- You must provide the sensitive details such as the
username
,password
, andextra
when updating credentials. This behavior aligns with the Atlan workflow config update UI. - Build the minimal
Credential
object. - Now, use the
update()
method of theCredential
object to update this new credentials in Atlan after initially testing it for successful validation.
Update workflow source credentials | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
|
- You can retrieve the workflow credential object by providing its
GUID
. - You must specify the authentication type of the credential.
- You must provide the sensitive details such as the
username
,password
, andextras
when updating credentials. This behavior aligns with the Atlan workflow config update UI. - Now, pass the
credential
object to thetest_and_update()
method to update this new credentials in Atlan after initially testing it to confirm its successful validation.
Update workflow source credentials | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
- You can retrieve the workflow credential object by providing its
GUID
. - You must specify the authentication type of the credential.
- You must provide the sensitive details such as the
username
,password
, andextra
when updating credentials. This behavior aligns with the Atlan workflow config update UI. - Build the minimal
Credential
object. - Now, use the
update()
method of theCredential
object to update this new credentials in Atlan after initially testing it for successful validation.
Update workflow configuration¶
To update workflow configuration for example, for Snowflake:
Coming soon
Update workflow configuration | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
- Find workflows by their type using the
find_by_type()
method of the workflow client and provide the prefix for one of the packages. In this example, we do this for theSnowflake
package. You can also specify the maximum number of resulting workflows you want to retrieve as results. - Retrieve the workflow template and specific task that you need to update.
- Update the specific workflow parameter. In this example,
we're enabling lineage for the
Snowflake
workflow. - Convert the workflow search result object to a workflow object
and pass that to the
update()
method to actually perform the workflow update in Atlan.
Coming soon