This week, we're excited to introduce powerful new API capabilities for shared pipelines, enhanced variable support for Maia AI agents, and Terraform for Streaming runners and pipelines, making your data workflows more automated and intelligent.

🔌 New Public API endpoints for shared pipelines

We're excited to announce the availability of a new set of public API endpoints, allowing you to manage the full lifecycle of your shared pipelines programmatically.

What you can do:

• Publish versions: Publish new shared pipeline artifact versions directly from CI/CD pipelines
• List and filter: List and filter published versions by source project or specific pipeline
• Retrieve details: Retrieve detailed information about any version, including the latest by default
• Enable or disable versions: Enable or disable specific versions to support rollback and controlled rollouts
• Browse pipelines: Browse all shared pipelines in an account, including their latest enabled version and description

See the API documentation and how-to guides for further details:

• Shared pipeline artifacts API endpoints
• Understanding shared pipeline artifacts
• Publishing shared pipeline artifacts with the API

🤖 Maia AI agent update: Improved variable support

We're happy to introduce system variable and project variable support, expanding your Maia AI agent's understanding of the variables available in your project.

Why is this helpful?

When building pipelines, knowing which variables are already defined—and what system variables are available at runtime—saves time and prevents duplication. Maia AI agents can now check what exists before creating something new.

What's new:

• System variables reference: Maia AI agents now know the available system variables (${sysvar.})—environment defaults, pipeline context, component metadata, and more. Ask your Maia AI agent which variables are available for your platform and it'll guide you to the right one.
• Project variable awareness: Maia AI agents can list your project-level variables, showing their name, type, behavior, and default value—so it can reference existing configuration instead of creating redundant pipeline variables.

How do I use it?

Maia AI agents will automatically use system variables and check your project variables when building pipelines—no special prompting needed. For more control, you can use context files to define standards like which project variables to use, helping Maia AI agents stay consistent across your pipelines.

🏗️ Public Terraform provider now available for Streaming runners and pipelines

Creating multiple Streaming runners and pipelines just got significantly more scalable! The new public Terraform provider eliminates the need to manually create runners and pipelines one at a time through the UI—a process that can be extremely time-consuming when dealing with dozens of databases.

With this Terraform provider, you can now programmatically create, update, and delete Streaming runners and pipelines at scale. This is particularly valuable for organizations managing large numbers of data sources, transforming what was once a lengthy, repetitive manual process into an automated, efficient workflow.

The provider includes comprehensive examples and covers both runner and pipeline configuration. While client-side and local infrastructure requirements may vary, this solution provides a solid foundation that can be customized to meet your specific needs.

Check out the Streaming pipelines documentation to get started.

💬 We'd love to hear from you!

Let us know how these new features are improving your workflows—we're all ears! Feel free to add any comments or questions below.

Want to get involved? Join the Matillion Community to stay up to date, share feedback, and help shape our product roadmap for future innovations.

‍

Autonomous Migration for Legacy ETL Platforms

New capability converts legacy ETL pipelines from 14 platforms to modern cloud data warehouses — compressing multi-quarter migration programs into weeks.

Denver and Manchester, UK – March 26 2026 — Matillion today announced the public preview of Migration Agent, a new capability within Maia, its flagship AI Data Automation platform, that autonomously converts legacy ETL pipelines into native, warehouse-optimized pipelines on Snowflake, Databricks, and Amazon Redshift.

Migration Agent addresses one of the most persistent and costly challenges in enterprise data modernization: the inability to move off legacy ETL platforms without committing to expensive, multi-quarter consulting engagements. With Migration Agent, teams can convert pipelines from 14 legacy platforms — including Informatica PowerCenter, Alteryx, IBM DataStage, SSIS, Oracle ODI, SAS Enterprise Guide, and dbt — with no manual rewrite and no GSI dependency.

How it works

Unlike conventional migration tools that replicate legacy logic into a new environment, Migration Agent performs a predictable, structured conversion. Maia parses the original transformation logic, dependency graphs, and pipeline metadata, then reconstructs each pipeline as a native, warehouse-optimized ELT pipeline. Unsupported or ambiguous constructs are flagged explicitly for human review, ensuring transparency and correctness throughout.

Engineers move from builders, manually recreating pipeline logic step by step — to managers, inspecting generated pipelines that already preserve the original system's logic and structure.

Read the full news article here.

This week, we're thrilled to roll out several updates designed to enhance pipeline flexibility, improve collaboration, and make your experience with Maia smoother than ever. Check out the changelog updates for the full details.

🖥️🖥️ Shared pipeline usability improvements and text mode for grid variables

We're excited to announce a series of enhancements that will allow shared pipeline creators to provide greater clarity and usability to their shared pipeline consumers! These improvements include:

• The ability to provide additional context to variables through optional display names
• Reorder variables based on their priority
• Assert if variables are required or optional
• Display variable descriptions to the shared pipeline consumer.

We have also now added text mode for grid variables in our Run Orchestration, Run Transformation, and Run Shared Pipeline components, which will help users quickly set a large number of values.

ℹ️ Improved webhook notification support

Pipeline failure notifications just got a lot more flexible. Alongside email and Slack, you can now route pipeline failure notifications to any webhook endpoint, meaning you can connect your Data Productivity Cloud agent notifications to services like Microsoft Teams, ServiceNow, or any internal tooling that accepts a webhook.

How it works

When setting up a notification, select Webhook as your delivery method. Provide a URL and a name for the webhook, then build your payload using a simple template.

You can customize the payload using any of the available template variables:

• ${pipelineName} - Pipeline name
• ${status} - Execution status
• ${finishedAt} - Completion timestamp
• ${pipelineExecutionId} - Execution ID
• ${projectId} - Project ID
• ${accountId} - Account ID

This update gives teams the flexibility to pass the context they need to downstream systems—not just a generic alert.

Read the documentation for more details.

🤖 Maia update: documentation search

You can now ask Maia questions about the Data Productivity Cloud, and it will search our official online documentation in real time to provide accurate, up-to-date answers.

What's new:

When you ask "How do I..." or "What is...", Maia now actively browses our latest documentation to find the specific technical details you need.

Why is this helpful?

• Always up to date: As soon as new documentation is published for a feature, Maia has access to it.
• Source of truth: By pulling directly from the docs, Maia provides reliable, technical guidance on how different parts of the platform work together.
• Efficiency: No more switching tabs to search the docs yourself—Maia does the digging and summarizes the answer right in your chat.

How do I use it?

There's nothing to toggle—just ask! Try asking Maia specific technical questions like:

• "How do I set up a schedule?"
• "How do I manage secrets?"

Give the new search capabilities a try and let us know what you think!

📄 Maia Update: Custom Connector now supports pagination

We've rolled out pagination support for Maia’s Custom Connectors, making it easier than ever to handle APIs that return data across multiple pages—which is most of them.

This update unlocks several key capabilities:

• Maia will suggest what pagination method is needed and the recommended configuration
• The ability to pull large datasets across multiple pages
• Simpler, more scalable connector builds for complex APIs

🌲 Flattening for unstructured data across Flex and Custom Connectors

Flattening for unstructured data across Flex connectors and custom connectors is now live in the Data Productivity Cloud.

What this means:

• Users can now flatten nested JSON structures while configuring these connectors
• Reduces the need for manual post-processing or workarounds
• Makes Flex and custom connectors far more viable for enterprise-grade API ingestion

🔩 Re-run from pipeline run history

Three new re-run actions are now available to help with running pipelines:

1. Re-run pipeline (top left button) - re-runs the entire scheduled or API-triggered execution from the beginning.
2. Play button (per step) - re-runs that individual step only, as a new standalone execution.
3. Play with arrow (per step) - re-runs from that step onwards, continuing execution from that point in the pipeline as a new standalone execution.

Read the documentation to learn more about pipeline observability features.

🔧 Enhanced project APIs for automated configuration management

You can now manage project variables, environment overrides, and project provisioning directly through the Data Productivity Cloud REST API. This means teams can automate configuration at scale—setting up projects, applying environment-specific settings, and managing credentials without manual UI interaction. Whether you're provisioning new workspaces or keeping environments in sync, the API now gives you full control over your project management workflows.

Read the documentation to learn more about these enhanced API capabilities.

🔒 IP allow list now available for Enterprise customers

Enterprise customers can now enhance their security posture by restricting account access to only trusted IP addresses. When enabled, any request from an unauthorized IP address—whether through the UI or API—receives a 403 error, providing robust network-level access control.

Users with the "Manage IP Allow List" permission can configure this feature from Profile & Account → IP Allow List, where they can add individual IPs or CIDR ranges, enable or disable entries individually, and search by IP, range, or description. The system supports both IPv4 and IPv6 addresses and includes built-in safeguards to help prevent admin lockout by auto-detecting and pre-populating the current user's public IP when adding the first address.

This feature is particularly valuable for organizations with strict security requirements, as it applies to both UI and API access across the entire account. Remember to include any IPs used by API clients, automation, or CI/CD tooling before enabling restrictions.

Read the full documentation for setup instructions, supported formats, and troubleshooting guidance.

📊 Richer, role-based sample data in onboarding

When you start a new project with Maia, you'll now get larger, more realistic sample datasets tailored to your job role—with over 1,000 rows of fact data alongside relevant dimension tables. This means you can explore pipeline features with meaningful data that reflects your real-world use case, whether you're in sales, marketing, finance, operations, or any other function.

Previously, onboarding generated just 3–4 small tables with 10–20 rows, taking 2–3 minutes. Now you get a rich dataset in under a minute, getting you to Maia faster and making an impact from the very first interaction.

⌨️ Improved "Add to Canvas" usability

You can now navigate the Add to Canvas modal entirely via keyboard:

• Search for your component as usual and press Enter to add the top result to the canvas
• Or search, then navigate the filtered results using the Up/Down arrow keys, select your component, and press Enter to add it

This update keeps your hands on the keys and your focus on the logic, making the process of building pipelines faster and more intuitive.

💬 We'd love to hear from you!

Let us know how these new features are improving your workflows—we're all ears! Feel free to add any comments or questions below.

Want to get involved? Join the Matillion Community to stay up to date, share feedback, and help shape our product roadmap for future innovations.

This week, we're excited to introduce updates that will streamline your data workflows and expand your pipeline capabilities. We've enhanced system-level variable access and expanded file loading support for Databricks users. For a full list of recent changes, be sure to check our changelog updates.

📊 Environment defaults as system variables

We've added the ability to reference project environment defaults, including role, database, warehouse, and schema, as system-level variables. This eliminates the need to create separate variables when [Environment default] was unavailable, such as in SQL and Python scripts.

📦 File load components now available for Databricks

5 new file load components are now enabled for Databricks, making it easier to bring file-based data straight into your pipelines.

New file load components for Databricks:

• CSV Load
• JSON Load
• AVRO Load
• ORC Load
• Parquet Load

These components allow you to load data from files in your source location directly into Databricks tables, with automatic handling of schema inference and table creation where applicable.

Supported from agent version: 11.154.0.

💬 We'd love to hear from you!

Let us know how these new features are improving your workflows—we're all ears! Feel free to add any comments or questions below.

Want to get involved?

Join the Matillion Community to stay up to date, share feedback, and help shape our product roadmap for future innovations

This week, we're excited to share several powerful updates that enhance security, streamline your workflow, and improve operational visibility. From environment-level credential management to quality-of-life improvements in pipeline building, these features are designed to make your Maia Foundation experience more efficient and secure. For a full list of recent changes, be sure to check our changelog updates.

🔐 Environment-level secrets and OAuths

We've released environment-level secrets and OAuths, delivering enhanced security and simplified credential management across your data pipelines. This major update provides several key benefits:

• Granular control over sensitive credentials: Secret mapping in the vault is now bound to specific environments. This reduces the risk of accidentally modifying production secrets or OAuths.
• Simplified cross-environment credentials: Secret definitions and OAuths automatically adjust based on environment.
• Ability to edit existing secrets and OAuths: You can now update all secrets or OAuths, making credential rotation much easier.
• Consolidated Public API: We've simplified credential management into a single connections endpoint, moving toward unified management of all credentials under one umbrella.

🎯 Copying and pasting components made smoother

We've ironed out a friction point in the pipeline building process to make your workflow even smoother. Previously, when copying and pasting multiple components, the newly pasted items were not automatically selected.

Now, all pasted components are automatically pre-selected, allowing for immediate movement the moment they hit the canvas. This small change reduces manual effort and keeps your momentum going while building complex pipelines, making the development process more intuitive and efficient.

🚀 Pipeline run history now includes artifact versions

We've shipped an update to the pipeline run history view that makes operational triage faster and more efficient for data teams.

Key improvements include:

• Artifact version is now visible - You can see exactly which version was executed. When you're debugging a failure, you'll immediately know whether it happened before or after a recent change, without having to cross-reference Git history.
• "Triggered by" replaces "Started by" - The column now shows what fired the run (a schedule, a manual run by a user, or the API).
• Time columns are cleaner - Started at, Ended at, and Duration are now separate columns.
• Full folder paths are included in the pipeline detail.

The combination of artifact version and schedule trigger means you can answer two of the most common questions — which version of the pipeline ran and what triggered it — directly from the run history list, without clicking into individual runs.

👀 Artifacts tab more detailed than ever

The Artifacts tab in your project now shows the name of the user who created each artifact and, if you have your own Git repository linked to the Maia Foiundation, the Git commit hash used to create it.

This update makes it easier to trace your artifacts and cross-reference them with your Git history for more context.

💬 We'd Love to Hear From You!

Let us know how these new features are improving your workflows—we're all ears! Feel free to add any comments or questions below.

Want to get involved?

Join the Matillion Community to stay up to date, share feedback, and help shape our product roadmap for future innovations.

This week, we're excited to bring you several powerful updates that enhance automation capabilities, improve pipeline building workflows, and keep you connected with Maia's intelligent assistance. For a full list of recent changes, be sure to check our changelog updates.

🔀 New Data Transfer component

We have rolled out a brand new native Data Transfer component, designed to deliver a smoother overall experience while using the component.

What's changed:

• Improved integration with Maia: Enhanced support for pipeline building
• New source added: Microsoft Exchange is now available as a source

Important to know:

• Any pipelines using the old Data Transfer component will continue to work. The new component is available in the new components panel.

This enhancement is included in Agent version 11.183.0. For detailed information on how to use the updated component, check out the documentation.

📣 Hybrid Kubernetes options globally available

You can now choose between Kubernetes deployment options (EKS or AKS) when setting up a hybrid agent in the Maia Foundation. This open-source container platform integration provides greater flexibility and control for organizations already invested in Kubernetes infrastructure.

Key benefits:

• Public repository access: New templates, guides, and pre-deployment check scripts
• Auto-scaling capabilities: Dynamic resource management
• Leverage existing expertise: Use your established Kubernetes infrastructure
• Enhanced control: Greater flexibility for future developments and customizations

Read our deployment guide to get started, and check out our public repository of templates, guides, and pre-deploy check scripts.

🚀 New API endpoint: Create Secret

We're excited to announce the new Create Secret endpoint is now live! This powerful API endpoint supports Full SaaS secret creation in the vault, eliminating the need to manually enter credentials through the UI when setting up new environments.

This enhancement is perfect if you want to fully automate the creation of Full SaaS environments without requiring cloud credentials for data ingestion. By streamlining the secret management process, you can now achieve complete automation in your environment setup workflows.

Check out the API Reference documentation to get started with this new endpoint.

🔔 Maia: Background notifications are here

We're happy to introduce background notifications for Maia in Designer — so you never miss a response or approval request while multitasking.

Previously, if you switched to another browser tab while Maia was working, you had no way of knowing when it finished or needed your input. Now, Maia comes to you with three helpful notification features:

• Tab title indicator: A dot (•) appears in the browser tab title when Maia finishes processing or needs your approval, so you can spot it at a glance.
• Audio chime: A subtle notification sound plays to get your attention without being intrusive.
• Browser notification popup: A system-level notification appears, letting you click straight back to the conversation.

To enable these notifications, simply open the settings menu in the top right of the Maia chat and toggle Notifications on. You'll be prompted to grant browser notification permission the first time.

🎯 Add tables and views to the Designer canvas

We are streamlining the way you build pipelines with an important update to our Add to Canvas functionality. For transformation pipelines, you can now select a table or view directly from your warehouse schema and have the corresponding component automatically added to your workspace.

This update creates a direct path from your data source to your design by solving the challenge of navigating the array of available components, which can be daunting, especially for new users.

What's new:

• Pre-load validation: Before committing to the canvas, you can now inspect metadata and request a data sample. This ensures you are selecting the exact data you need before you even start building.
• Streamlined workflow: By removing the guesswork of component selection, we've made the process of loading warehouse data faster and more accessible for everyone. By default, we select all the table columns so the component is ready to use.

To access this feature, clicking the + on the Designer canvas, then select the new Warehouse data tab. We've also introduced the ability to search by schema name making it easier to find the right data.

💬 We'd Love to Hear From You!

Let us know how these new features are improving your workflows—we're all ears! Feel free to add any comments or questions below.

Want to get involved?

Join the Matillion Community to stay up to date, share feedback, and help shape our product roadmap for future innovations.