✨ This week in Maia: Pending pipeline visibility and iterator concurrency limits

Welcome back to the Maia New Features Blog! This week, we're excited to bring you updates that give you greater control and visibility over your pipeline operations. We've introduced enhanced status tracking so you can see exactly what's happening with your pipeline steps, plus new concurrency controls for iterators to help you optimize performance. For a full list of recent changes, be sure to check our changelog updates.

🚦 New “Pending” status for pipelines

When checking a pipeline’s status using your pipeline run history, task history, or the Maia API, you can now see the difference between a pipeline step that's actively executing and one that's waiting for a runner slot.

Steps will only show as Running when they're actively executing. Before that point, they'll show as Pending, so you'll always know whether something is genuinely in flight or just waiting. As tasks are picked up, pending tasks move to running in real time. This makes it easier to diagnose whether a slowdown was a real execution issue or just queue pressure.

Read the documentation to learn more about viewing step execution status.

🔁 Maximum concurrency limits for iterators

You can now set a maximum number of concurrent iterations on any iterator in Maia, giving you more flexible control over how pipelines are executed. This allows you to:

• Control concurrency: Previously, you had to choose between full concurrency of the available runner capacity or fully sequential runs. Now you can set a sensible concurrency level that gives you optimal performance without overwhelming your downstream systems or data warehouse.
• Import Matillion ETL pipelines with sensible defaults: Apply a sensible max concurrent iterations when importing pipelines, ensuring there are no unintended consequences of Maia’s greater scalability.

For customers upgrading from Matillion TL to Maia specifically, we are introducing a default of 16 when importing a pipeline into Maia. You can be change this setting or completely turn off the limit when setting your import preferences.

⚠️ Watch out for iterator stacking—limits are multiplicative, not additive. Stacked iterators can hit full runner capacity quickly if not configured carefully.

💬 We'd love to hear from you!

Let us know how these new features are improving your workflows—we're all ears! Feel free to add any comments or questions below.

Want to get involved?

Join the Matillion Community to stay up to date, share feedback, and help shape our product roadmap for future innovations.

‍