SingleStore Pipelines
Native streaming ingestion built into the SingleStore engine. Load continuously from Kafka, object storage, or files using a single SQL statement, transform on ingest, and query new data while it is still loading.
Product Description
Overview
SingleStore Pipelines is the native data ingestion capability built directly into the SingleStore database engine. It streams data continuously from files, object storage, and Apache Kafka without requiring external tooling such as Kafka Connect or a separate stream processing framework. Pipelines are created with a single SQL statement, can transform records on ingest, and allow newly arriving data to be queried immediately while loading continues.
Highlights
Streaming load from files, object storage, and Apache Kafka
Configured with one SQL statement, no external ingestion tooling required
Optional transformation applied on ingest
New data queryable while ingestion is still in progress
Parallel, resilient, and low-latency by design, built into the database engine
Supported Cloud Infrastructure
Amazon Web Services, Google Cloud Platform, and Microsoft Azure through SingleStore Helios, and any environment where SingleStore Self-Managed is deployed, including private cloud and on-premises.
