Pipeline Designer Innovation Release

The Pipeline Designer is a visual interface in Hybrid Manager (HM) for building and managing AI Database (AIDB) data pipelines. It provides a guided, no-code workflow for creating pipelines that process data through AI and ML transformation steps, optionally storing the results in a vector knowledge base for retrieval-augmented generation (RAG).

With Pipeline Designer you can:

• Create single-step or multi-step data pipelines through a visual wizard.

• Build knowledge bases with embedded vector storage for semantic search.

• Configure processing modes (Live, Background, or On Demand).

• Monitor pipeline status, record counts, and error conditions.

• Test knowledge base retrieval with built-in query tools.

• Use HM-hosted models, HM external inference service proxies, AIDB-native models, or built-in on-database models.

• Run pipelines on primary HM clusters, secondary HM locations, or self-managed Postgres instances.

How Pipeline Designer relates to AIDB

Pipeline Designer is a graphical front end for the AIDB extension. Every pipeline you create through the designer translates directly to AIDB SQL calls aidb.create_pipeline, PipelineStepOperation, PipelineAutoProcessingMode, and related functions running in your Postgres database. Pipeline Designer doesn't introduce its own conceptual vocabulary. It presents AIDB pipelines, knowledge bases, and models through a visual interface. For the underlying concepts, see the AIDB documentation.

Pipeline Designer manages only the pipelines it creates. Pipelines created directly through SQL (outside the Pipeline Designer UI) don't appear in the designer interface. See VPU and permissions for details on why this separation exists.

Prerequisites

Before creating your first pipeline, complete the setup steps in Getting started. The setup checklist covers extension installation, cluster configuration, platform roles, model registration, and source table permissions. The Getting started page also includes example source tables you can use to try Pipeline Designer straight away.

Pipeline Designer provides a subset of the full AIDB feature set. Key limitations include volume-based sources and multi-pipeline knowledge bases not yet being supported in the UI. For the full list, see Limitations.

Supported cluster types

Pipeline Designer supports the following cluster configurations:

• HM-managed clusters (Primary/Standby Replication (PSR) and Postgres Distributed (PGD)/AHA): Clusters provisioned and managed by Hybrid Manager on the primary location.

• Secondary HM locations: HM-managed clusters running on additional HM locations in a multi-location deployment.

• Self-managed Postgres instances: Single-instance Postgres servers registered with HM through the EDB Postgres AI agent (beacon-agent), with AIDB installed. External PGD clusters and Cloud Native Postgres (CNP) clusters aren't supported.

The minimum supported Postgres major version is 16. For details on how execution works across these cluster types, see Executing pipelines.

What's next

• Getting started: Set up your environment and create your first pipeline.
• Concepts: Learn about pipeline structure, multi-step design constraints, and processing mode selection.
• Creating pipelines: Walk through the pipeline creation wizard.
• Knowledge bases: Understand vector storage, monitoring, and querying.
• External inference services: Register external model providers.
• Executing pipelines: Run pipelines on different cluster types.
• VPU and permissions: Understand the permission model.
• Limitations: Design constraints and current limitations.