🚀 Improved Python and Bash scripting, API-based repository migration, and automatic pipeline renaming

This week, we're bringing you powerful new capabilities that enhance flexibility and streamline your workflows in Maia. From dedicated script runners to seamless repository migrations and improved pipeline management, these updates are designed to make your data operations more efficient and adaptable. For a full list of recent changes, check our changelog updates.

💻 Script Pushdown component and runner

We're excited to announce an upgrade that allows you to run Python and Bash scripts in a dedicated runner when using a Hybrid SaaS deployment.

This new feature provides allows you to:

• Have a dedicated script runner that is scaled appropriately for these workloads, keeping your Maia runner scaled for average task loads.
• Deploy this script runner as part of your Maia runner deployment.
• Use your existing process to load libraries.
• Migrate large script workloads from Matillion ETL when upgrading to Maia without adjusting for runner architecture.

As part of this change, we've also introduced sizing options for both script runners and Maia runners.

The Script Pushdown component supersedes the Bash Pushdown component. The Python Script and Python Pushdown components are not affected. Python Script can still be used for smaller workloads that do not impact your Maia runner, and Python Pushdown is still used for Full SaaS deployments.

Read the Script Pushdown component guide to learn more.

🔄 Migrate your project’s repository using our API

The new Migrate Repository API endpoints gives you more flexibility than ever. You can now change the repository that your Maia project is based on.

Whether you’ve department switched Git providers or you need to move away from a hosted repository, the Maia API supports migrations from hosted Git to GitHub, GitLab, and Azure, as well as between GitHub, GitLab, and Azure providers. This means you can adapt to organizational changes without losing your project progress or having to start from scratch.

Read our Repository Migration API guide to get started.

✍️ Automatically update pipeline references

When renaming a pipeline in Designer, you can now ask Maia to update all references to the renamed pipeline in other pipelines in your project—for example, if you rename an orchestration pipeline that’s referred to in a Run Orchestration component in another pipeline.

This update streamlines project management by automatically updating all pipeline references, eliminating the need to manually track down and update these references, saving time, and reducing the risk of broken links when reorganizing your pipelines.

💬 We'd love to hear from you!

Let us know how these new features are improving your workflows—we're all ears! Feel free to add any comments or questions below.

‍