What Are Data Connectors?

TL;DR:

Data connectors are the pre-built integration components that move data between a source system (a SaaS API, database, or file store) and a target destination such as a cloud data warehouse. Each connector handles the authentication, pagination, rate limiting, and schema mapping required to read from one system and write to another reliably. They exist so engineers do not have to write and maintain custom extraction code for every source they need to integrate.

Modern data platforms ship hundreds of connectors out of the box, but managing them at scale has become its own engineering burden. Agentic AI is now shifting that work from manual connector configuration to autonomous execution.

What a Data Connector Actually Does

A data connector is often described as a simple bridge between two systems. In practice it is a piece of engineering that has to manage the messy reality of how source systems expose their data, and it has to keep doing so as those systems change underneath it.

Every connector is responsible for a defined set of jobs. It authenticates against the source, usually through OAuth, API keys, or database credentials. It reads the data, which means handling pagination on APIs that return results in pages, and respecting the rate limits those APIs enforce. It maps the source structure to a format the destination can store. It tracks what has already been moved, so that incremental runs pull only new or changed records rather than re-extracting everything.

When any of those responsibilities is handled badly, the pipeline downstream of the connector breaks. A connector that does not track state correctly will duplicate data. A connector that cannot adapt to a renamed field will fail the moment the source changes.

Sources and Destinations

The clearest way to understand a connector is to look at the two ends it joins. At one end is the source, the system that holds the data you want. At the other is the destination, the system you want that data to live in. A connector's job is to read from the first and load into the second without losing fidelity along the way.

Sources are diverse and largely outside your control. They include SaaS applications, operational databases, event streams, and file stores, each exposing its data in a different way and on its own terms. Destinations are far more consolidated. In most modern stacks the destination is a cloud data warehouse such as Snowflake, Databricks, or Amazon Redshift, where data is centralized for transformation and analysis.

This direction matters. A connector is not a generic two-way pipe. It is built to extract from a specific kind of source and deliver into a specific kind of destination, and the engineering on the source side is almost always where the difficulty lives.

Types of Data Connectors

Connectors are usually grouped by the kind of system they integrate with, because each category brings its own technical demands.

Application connectors integrate with SaaS platforms through their APIs. These include CRMs such as Salesforce, marketing tools such as HubSpot, support systems, and finance platforms. The main challenge here is that every API is different, and APIs change without warning.

Database connectors read from operational databases such as PostgreSQL, MySQL, and Microsoft SQL Server. For these sources, the question is usually how to capture changes efficiently. Reliable replication often depends on log-based Change Data Capture, which reads directly from the database transaction log rather than repeatedly querying the tables themselves.

File and storage connectors move data from object stores and file systems, including Amazon S3, Azure Blob, and SFTP locations. These handle formats such as CSV, JSON, and Parquet, where the structure is often inconsistent from file to file.

Custom connectors cover the systems that no pre-built option supports. Traditionally, building one meant writing and owning extraction code from scratch, which is exactly the cost that pre-built connectors were meant to remove.

Batch and Streaming Connectors

Beyond the type of source, connectors also differ in how they move data. The right choice depends on how fresh the data needs to be at the destination.

Batch connectors extract data on a schedule, pulling a defined set of records at set intervals. This is the common pattern for most analytics work, where a periodic refresh is sufficient and the simplicity of scheduled runs is an advantage. Most application and file connectors operate this way.

Streaming connectors capture changes continuously rather than on a schedule. They are built to read changes from a source database as they happen and write them to cloud storage or a warehouse with minimal delay, which suits use cases where stale data carries a real cost. Streaming connectors lean heavily on Change Data Capture, since reading the transaction log is what makes near-real-time replication possible without overloading the source.

The Connector Maintenance Problem

Pre-built connectors solved the original problem, which was that writing extraction code by hand for every source did not scale. They introduced a second problem in its place.

A large data team can depend on hundreds of individual connections, and each one is a small contract with a system the team does not control. Source APIs deprecate endpoints. Authentication methods change. A field is renamed at the source, and that schema drift silently breaks the mapping. None of this is visible until a pipeline fails and the data stops arriving.

This is the connector management burden. The work is no longer writing connectors. It is configuring, scheduling, monitoring, and repairing a sprawling estate of them, and that work consumes engineering capacity that should be going toward higher-value transformation and modeling.

How Connectors Fit Into the Modern Data Stack

In a modern ELT architecture, connectors sit at the front of the pipeline. They handle the extract and load stages, pulling raw data from each source and writing it into the warehouse before any transformation happens. The earlier ETL model often transformed data in transit, but the modern pattern loads raw data first and transforms it inside the destination.

This means connectors are the foundation the rest of the stack is built on. If the connector layer is unreliable, every downstream model and report inherits that unreliability. Getting data ingestion right at the connector level is therefore a prerequisite for trustworthy analytics, not an afterthought.

Autonomous Connectors with Maia

Maia is the first AI Data Automation platform, and it changes how the connector layer is managed. Traditional connectors require an engineer to configure each source and then keep it working. Maia interprets the intent behind a request and manages the connection lifecycle itself, which removes much of the manual overhead that the connector estate normally generates.

Maia provides a broad library of pre-built data connectors across applications, databases, and storage systems, so common sources are supported out of the box. Where a source has no existing connector, agentic AI can build a custom connector for any REST API that returns JSON. You point Maia at the API's documentation, or upload its specification as a PDF, YAML, or JSON file, and Maia reads the endpoints, applies the right authentication, and configures pagination, without an engineer writing extraction code by hand.

The more important shift is in maintenance. When a source changes its structure, a traditional connector fails and waits for someone to investigate. Maia detects the change, diagnoses what broke, and surfaces the precise fix, which keeps data flowing without an engineer having to trace the failure manually. Connector management stops being a standing tax on the team and becomes something the platform handles.