Okta, Google, and others provide user authentication and have native support for Open ID Connect (OIDC) for all of their customers. This step-by-step guide will explain how to enable OIDC for your Amundsen installation.
You can find existing Amundsen docs at:
We rely on verdan/flaskoidc for the OIDC setup in Amundsen. So, make sure to install flaskoidc for each Amundsen service. i.e., frontend, metadata, and search. You can find the repo at:
Step 1 — Create an OIDC Application
I will assume in the rest of this guide that you have created a client/application in your OIDC provider and have the Client ID and Client Secret handy.
Important: While creating a client in your OIDC provider, you will be asked to provide a Redirect URI. For that, make sure to use https://YOUR_AMUNDSEN_DOMAIN/auth
Step 2 — Use OIDC Config for your amundsen/frontend service
Nothing complicated here. Make sure you are using the correct OidcConfig for your frontend service.
You can do this by setting the following available environment variable.
Or, if you are using a custom config class for your frontend service, make sure to inherit your custom config with the OidcConfig.
The reason to use OidcConfig is that it sets the following two methods that are needed to get the end-to-end authentication within Amundsen:
- REQUEST_HEADERS_METHOD (https://www.amundsen.io/amundsen/authentication/oidc/#setting-up-request-headers)
This method ensures your frontend service passes the correct token in the request to other services when calling their APIs.
- AUTH_USER_METHOD (https://www.amundsen.io/amundsen/authentication/oidc/#setting-up-auth-user-method)
This configuration method gets the user information from the session and serializes that information into a User object.
Step 3 — Get a Discovery Document endpoint
To simplify OIDC implementations and increase flexibility, OpenID Connect allows the use of a “Discovery document,” a JSON document found at a well-known location containing key-value pairs which provide details about the OpenID Connect provider’s configuration, including the URIs of the authorization, token, revocation, userinfo, and public-keys endpoints.
The following are the discovery document endpoints of the commonly used OIDC providers:
Okta (Auth Server ID): https://[YOUR_OKTA_DOMAIN]/oauth2/[AUTH_SERVER_ID]/.well-known/openid-configuration
If you are using your custom OIDC provider, you should expose a discovery endpoint that follows the specification defined at: https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderMetadata
Step 4 — Configure Amundsen to use OIDC Auth
Following environment variables MUST be set to get the OIDC working.
Note: You will need to set the following environment variables for each service. i.e., frontend, metadata, and search. (In our community helm chart, I have made the common settings to be set only once. I will explain how to set these for helm in a later section of this guide)
The name of the OIDC provider, like google, okta, keycloak, etc. I have verified this package only for google, okta, and keycloak. Would you mind making sure to open a new issue if any of your OIDC providers is not working? (Even better: contribute it upstream to the open-source project, I am very proactive in reviewing and merging the patches and new features 😉)
The following Scopes are required to make your client works with the OIDC provider, separated by a space.
You will get the Client ID and Client Secret values after creating a new application in your OIDC provider in Step 1.
This is the endpoint that your OIDC provider hits to authenticate against your request. This is what you set as one of your redirect URI in the OIDC provider client’s settings.
This is the discovery endpoint that you will get in Step 3 of this guide.
Step 5 — Verify your setup
This is it! Now is the time to start your Amundsen services and verify if things are working for you. By this time, you should be able to see your OIDC provider’s login screen when you try to access your Amundsen instance. ☀
Please note that only setting up OIDC:
- does not enable User Profiles in Amundsen UI.
- does not ingest users in your database. If you don’t have the users in your metadata proxy database, this will fail to load the Users’ related information, like owners, frequent users, user profiles, bookmarks, etc.
Please refer to the below optional sections to enable any of the above two mentioned items.
Optional: Enable User Profiles in Amundsen
User Profile is disabled by default in Amundsen UI. This can be enabled via a frontend configuration.
Make sure you enable the “indexUsers” settings, it will look like this afterward:
Please note that you need to build the frontend manually after making this change. This also means that you will need to push your custom Docker image to Dockerhub or your custom registry if you are using Docker images for your deployment.
Optional: Ingest Users in metadata proxy
Preferably, users should be ingested to your metadata database via a databuilder job or a cronjob.
In cases where the databuilder job is not an option to ingest Users into the database, you can leverage USER_DETAIL_METHODof the metadata service configuration class to get (or even create users) in runtime from any third-party/custom system.
This custom function takes user_id as a parameter and returns a dictionary consisting of user details’ fields defined in UserSchema.
Example basic usage:
The above basic code will ensure that your Amundsen UI will not break, even if you do not have Users ingested in your database.
- It is highly recommended to cache this function’s response to avoid any performance issues.
- Once set, this function will be called to get the user details every time a user object is requested.
- This function is implemented for Neo4j Proxy and Apache Atlas Proxy only when writing this guide.
If you are looking for a way to CREATE a user during this process in your database, then you can also do that using the USER_DETAIL_METHOD
The logic should look like this:
- Check if the user exists in your database → if yes, return the user details
- If the user is not already in the database → get user information from OIDC provider, or any other third-party system → Store in your database → Return the newly created user’s details
1: If you are using Google as the OIDC Provider
- You MUST enable the People API for your Google Application.
- Update the scopes defined in Step 4, and add https://www.googleapis.com/auth/directory.readonly.
- The scopes should look like this now
- And finally, the USER_DETAIL_METHOD in metadata config should look something like this.
2: If you are using Okta as the OIDC Provider
- Under your Okta application, you MUST grant permission for the following scopes
- Update the scopes environment variable defined in Step 4, and add the ones you new assigned to your application in Okta. The scopes should look like this now:
- And finally,USER_DETAIL_METHOD in the metadata config should look something like this.
Optional: Configure OIDC using Helm
Mount the OIDC secrets in the Kubernetes cluster by passing in the required values through the Helm --set ... function. (use comma-separated values to set multiple values)
Make sure to enable the OIDC support:
Once you enable the OIDC using the flag above, the helm chart automatically sets the Flask’s wrapper module and class to each Amundsen service. That means you do not need to set the following explicitly:
The following common values for OIDC need to be set only once and not for each service separately. The default values set in our helm are:
You can change the values accordingly, for example:
Finally, you need to set the client id and secret for each service that you got when you created the OIDC application in Step 1 of this guide.
Stemma ❤ Amundsen
Hopefully, this guide helps you in setting up OIDC with Amundsen. I am a founding engineer at Stemma, which provides a managed version of Amundsen with certain additions:
- Managed Amundsen with encryption, backup/restore, OIDC integration, and much more straight out of the box.
- Intelligence — common join and filter conditions, lineage, slack integration, and much more
- Deep experience in ensuring adoption within your organization.