Stemma is based on Amundsen, which makes transitioning from Amundsen to Stemma simple. In fact, other companies (such as iRobot) have already successfully migrated from Amundsen to Stemma.
Migration from Amundsen involves 3 main steps:
- Migrating existing metadata from Amundsen to Stemma
- Ongoing metadata ingestion
- Cut over
1 - Migrating existing metadata from Amundsen to Stemma
In this step, you want to take all the existing metadata from Amundsen and load it into Stemma. The metadata imported includes, but is not limited to:
- Table and column descriptions
- Tags
- Ownership information
In order to import this metadata, you can upload a dump of your Amundsen neo4j metadata into Stemma. You can find complete documentation on that on the Stemma docs site. After import, your Stemma instance will look very similar to your existing Amundsen instance, as shown below. It won’t however be periodically updated. For periodic updates, see the next step below.
2 - Ongoing metadata ingestion
Now that you have ingested existing Amundsen metadata into Stemma, it’s time to configure Stemma to ingest metadata updates on an ongoing basis. Once that's configured, you’ll no longer have to write Python databuilder jobs to ingest data.
Information ingested in this step includes:
- Table and column names
- Linked issues (JIRA tickets)
- Frequent users
- Lineage, if it exists
In Stemma's Admin interface, you can provide credentials to your data sources to allow Stemma access to extract metadata on an ongoing basis. Stemma will do the work to “upsert” metadata updates coming on an ongoing basis into the pre-loaded data from Amundsen.
3 - Cut Over
At this point, you have both Amundsen and Stemma running side-by-side. During this stage, we recommend sharing access to Stemma with your power users and getting their feedback to ensure all of their use-cases are supported. Usually one week of overlap to obtain feedback and an additional week to incorporate feedback, if applicable, is sufficient.
When all looks good, simply redirect the URL of your internal Amundsen to the Stemma URL.
Summary
To summarize, it’s super easy and common to transition from Amundsen to Stemma. Companies like iRobot have successfully migrated from Amundsen to Stemma and many others are in the process.
“We chose Amundsen as our data catalog because of its focus on automation and its supportive community. Since then we have moved to Stemma as our data catalog. I am thrilled to see Stemma make automated data catalogs widely available." - Michelle Gulen, iRobot
