OpenAPISpecLoader

Atlan University

See it in action in our code samples course.

This content has moved!

We have moved our samples to a separate, dedicated site: https://solutions.atlan.com.

This document is no longer being maintained.

Updates (or creates, if they do not exist) API specs and paths in Atlan from an OpenAPI (v3) definition.

Expects an OpenAPI JSON specification file, accessible via URL.

The OpenAPISpecLoader will attempt to process the spec in an idempotent way:

• Only updating assets that already exist
• Creating any assets that do not yet exist

Does not delete assets

However, note that it will not currently delete assets that no longer appear in the spec (feel free to extend the sample with this, if needed!)

Configuration

The following variables are expected to run the loader:

ATLAN_BASE_URL

Specifying the URL for your Atlan tenant.

ATLAN_API_KEY

Providing the API token to use when accessing Atlan.

API_SPEC_URL

Providing the URL to the JSON specification. For instance, for the Swagger pet store example this would be: https://petstore3.swagger.io/api/v3/openapi.json.

API_NAME

Providing a unique name for the connection under which you want to store the API assets for this specification. For example, this could be something like public or test.

You can also load from a local file

You can also provide a local filename for the API_SPEC_URL to load from a local file. You might want to opt for this approach if the OpenAPI spec file is otherwise more complicated to reach (for example, behind some additional authentication layer or proxy).

Logging

By default, only messages at INFO and above will be logged (to the console). You can change this level by copying the main/resources/log4j2.xml from the source repo and modifying it to give:

• more detail (DEBUG will log every API request and response, including their full payloads)
• less detail (WARN will only print warnings and errors, ERROR only errors).

Running the loaders locally

You can run the loaders locally on any machine with network access to your Atlan tenant (and the location where your API spec is hosted, if you are not using a local spec file).

When run in this mode, the loader will read the input Excel file locally on the machine running the loader. If you want to change any of the settings, you can simply prepend the command with the environment variables.

From repo cloneFrom downloaded jar files

You can run the OpenAPISpecLoader from a clone of the samples repository:

You will need a JDK installed to run this way, as the code must be compiled locally before it can be executed.

Run OpenAPISpecLoader from a clone of the GitHub repo

API_SPEC_URL=https://petstore3.swagger.io/api/v3/openapi.json \
API_NAME=PetStore \
./gradlew OpenAPISpecLoader # (1)!

1. Within the root directory of your local clone of the repo, run the Gradle task for the OpenAPISpecLoader.

   Overriding the configuration

   Note that you can specify any optional overrides as environment variables before running the reporter. You can either do this in the same line where you run the command, or by exporting them first for example using export API_SPEC_URL=https://petstore3.swagger.io/api/v3/openapi.json.

You can run the OpenAPISpecLoader by downloading the pre-compiled jar files for both:

• 
• 

With this approach you only need a JRE installed, as you will be using pre-compiled code.

Run OpenAPISpecLoader from pre-compiled jar files

API_SPEC_URL=https://petstore3.swagger.io/api/v3/openapi.json \
API_NAME=PetStore \
java \
    -cp atlan-java-...-jar-with-dependencies.jar:atlan-java-samples-...-jar-with-dependencies.jar \
    com.atlan.samples.loaders.OpenAPISpecLoader # (1)!

1. Within the directory containing the downloaded jar files, run the java command providing at least two arguments:

   1. The two jar files, separated by a :.
   2. The canonical classname of the OpenAPISpecLoader.

   Overriding the configuration

   Note that you can specify any optional overrides as environment variables before running the reporter. You can either do this in the same line where you run the command, or by exporting them first for example using export API_SPEC_URL=https://petstore3.swagger.io/api/v3/openapi.json.

   (Optional) If you want to override the logging settings, also add -Dlog4j.configurationFile=./log4j2.xml just before the canonical classname, pointing to your modified version of the log4j2.xml file.

   Replace the ... with proper version numbers

   Don't forget to replace the ... in the -cp argument's values with proper version numbers, or running this will not work.

Example output

12:13:22.746 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - Retrieving configuration and context...
12:13:22.752 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - Loading definitions from: https://petstore3.swagger.io/api/v3/openapi.json
12:13:24.759 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - Looking for existing API connection named: PetStore
12:13:28.025 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ... none found, creating a new API connection...
12:13:30.656 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ... created connection: default/api/1688987608
12:13:54.158 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - Upserting APISpec: default/api/1688987608/Swagger Petstore - OpenAPI 3.0
12:13:54.955 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ... created APISpec: default/api/1688987608/Swagger Petstore - OpenAPI 3.0
12:13:54.956 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - Creating an APIPath for each path defined within the spec: 13
12:13:54.958 [main] INFO  com.atlan.util.AssetBatch - ... upserting next batch of (13) APIPaths...
12:13:57.025 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/pet
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/pet/findByStatus
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/pet/findByTags
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/pet/{petId}
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/pet/{petId}/uploadImage
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/store/inventory
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/store/order
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/store/order/{orderId}
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/user
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/user/createWithList
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/user/login
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/user/logout
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... created: APIPath: default/api/1688987608/Swagger Petstore - OpenAPI 3.0/user/{username}
12:13:57.026 [main] INFO  com.atlan.samples.loaders.OpenAPISpecLoader - ...... updated: APISpec: default/api/1688987608/Swagger Petstore - OpenAPI 3.0

More details

• The first line will always tell you the location of the OpenAPI spec the loader is processing.
• The loader will only attempt to load information from within the OpenAPI spec, and will not delete anything that already exists in Atlan but is no longer present in the spec.

---