Knowledge bases reference v7

Reference for knowledge base (KB) functions and views. For guide-style documentation, see Knowledge bases.

Catalog views

aidb.knowledge_bases

Also accessible as aidb.kbs. Lists all knowledge bases in the database with their full configuration.

aidb.knowledge_base_metrics

Also accessible as aidb.kbm. Aggregated stats for each knowledge base — pipeline count, embedding row count, and the worst pipeline status. For per-pipeline progress (backlog, errors, source / destination counts), see aidb.pipeline_metrics.

Example

SELECT * FROM aidb.kbm;

             name             | pipelines | embeddings |  status
------------------------------+-----------+------------+----------
 public.product_catalog_kb    |         1 |       1842 | UpToDate
 public.customer_feedback_kb  |         2 |        915 | UpToDate
(2 rows)

The legacy view names aidb.knowledge_base_stats and aidb.kbstat are kept as aliases of aidb.knowledge_base_metrics for backward compatibility.

Types

aidb.DistanceOperator

Vector distance function used during retrieval.

aidb.PipelineDataFormat

Format of data in a pipeline source or volume.

aidb.PipelineAutoProcessingMode

Auto-processing mode for a knowledge base pipeline.

Knowledge base functions

To change a pipeline's auto-processing mode, use aidb.update_pipeline.

aidb.retrieve_key

Returns the source record keys and distances for the top matching embeddings, without fetching source data. In part_ids, also returns the IDs of the matching parts for each step in a multi-step pipeline. For example, if a single source record is split into 10 parts by a ChunkText step, then part_ids indicates which chunk matched.

Parameters

Returns

Example

SELECT * FROM aidb.retrieve_key('public.customer_feedback_kb', 'late delivery', topk=>3);

 key  |      distance      | part_ids |   pipeline_name
------+--------------------+----------+------------------
 1042 | 1.1931772046185758 | {0,0}    | feedback_eu_pipe
 318  | 1.1980633963685476 | {2,0}    | feedback_na_pipe
 1057 | 1.2150919866080878 | {2,0}    | feedback_eu_pipe
(3 rows)

aidb.retrieve_text

Returns source text and distances for the top matching embeddings by joining the embeddings table with the source table. In part_ids, also returns the IDs of the matching parts for each step in a multi-step pipeline. For example, if a single source record is split into 10 parts by a ChunkText step, then part_ids indicates which chunk matched. The return column intermediate_steps contains the actual intermediate step output.

Parameters

Returns

intermediate_steps is a JSON array with one entry per step that has intermediate storage enabled, each shaped like:

{"value": "...", "operation": "ChunkText", "step_order": 1, "destination_table": "public.pipeline_my_pipeline_step_1"}

Example

SELECT * FROM aidb.retrieve_text('public.pipeline_products', 'waterproof jacket', 3);

  key  |                value                | distance           | part_ids | pipeline_name | intermediate_steps
-------+-------------------------------------+--------------------+----------+---------------+--------------------
 19337 | Men Stripes Waterproof Shell Jacket | 0.2994317672742334 | {0}      | products      | []
 55018 | Women All-Weather Hiking Anorak     | 0.3804609668507203 | {0}      | products      | []
(2 rows)

aidb.delete_knowledge_base

Cascades: deletes all pipelines attached to the knowledge base, drops the vector table, and removes the registry entry. Doesn't delete underlying source tables.

To remove a single pipeline from a multi-pipeline KB while keeping the KB and the other pipelines' vectors intact, use aidb.delete_pipeline instead. Deleting the last attached pipeline also drops the KB.

Parameters

Example

SELECT aidb.delete_knowledge_base('public.customer_feedback_kb');

Volume management functions

aidb.create_volume

Creates an AIDB volume from a PGFS storage location. The volume is a foreign table that exposes the storage location's contents as rows and can be used as a knowledge base data source or queried directly with SQL. See External storage for setting up the PGFS storage location first.

Parameters

Example

SELECT aidb.create_volume('pdf_volume', 'my_s3_location', '/', 'Pdf');

aidb.volumes

View that lists all AIDB volumes in the database. Returns schema, volume, storage_location, and path.

Example

SELECT * FROM aidb.volumes;

aidb.delete_volume

Deletes an AIDB volume.

Parameters

Example

SELECT aidb.delete_volume('pdf_volume');