Pipelines retrievers (deprecated) reference v1.3

Suggest edits
Important

The use of the term "retrievers" has been deprecated in version 4.0 and later. It is replaced by the term "knowledge base" and users should consult the knowledge_bases reference content. For backwards compatibility, the retrievers API is still available, but it is recommended to use the knowledge base API for new implementations. This section is covers the deprecated functions only. For the new API, see knowledge_bases.

aidb.RetrieverSourceDataFormat

No longer available. Use aidb.PipelineDataFormat instead.

Functions

aidb.create_retriever_for_table (deprecated)

Replaced by aidb.create_table_knowledge_base

Creates a retriever for a given table.

Parameters

ParameterTypeDefaultDescription
nameTEXTRequiredName of the retriever
model_nameTEXTRequiredName of the model to use
source_tableregclassRequiredName of the table to use as source
source_data_columnTEXTRequiredColumn name in source table to use
source_data_typeaidb.PipelineDataFormatRequiredType of data in that column ("Text"."Image","PDF")
source_key_columnTEXT'id'Column to use as key to reference the rows
vector_tableTEXTNULL
vector_data_columnTEXT'embeddings'
vector_key_columnTEXT'id'
topkINTEGER1
distance_operatoraidb.distanceoperator'L2'
optionsJSONB'{}'::JSONBOptions
index_typeTEXT'vector'Type of index to use for the vector table.
Index_types

  • If index_type is set to vector, the system will automatically create a hnsw index on the vector table based on the distance operator used in the retriever. This is the default index type. The vector index type is only able to support 2000 dimensions or less. If more dimensions are needed, the index type should be set to disabled.

    distance_operatorindex_type
    L2vector_l2_ops
    InnerProductvector_ip_ops
    Cosinevector_cosine_ops
    L1vector_l1_ops

  • If index_type is set to ivfflat, the system will create a IVFFlat index on the vector table.

  • If index_type is set to disabled, no index will be created.

Example

SELECT aidb.create_retriever_for_table(
               name => 'test_retriever',
               model_name => 'simple_model',
               source_table => 'test_source_table',
               source_data_column => 'content',
               source_data_type => 'Text',
       );

aidb.create_retriever_for_volume (deprecated)

Replaced by aidb.create_volume_knowledge_base

Creates a retriever for a given PGFS volume.

Parameters

ParameterTypeDefaultDescription
nameTEXTRequiredName of the retriever
model_nameTEXTRequiredName of the model
source_volume_nameTEXTRequiredName of the volume
vector_tableTEXTNULLName of the vector table
vector_data_columnTEXT'embeddings'Name of the vector column
vector_key_columnTEXT'id'Name of the key column
topkINTEGER1Number of results to return
distance_operatoraidb.distanceoperator'L2'Distance operator
optionsJSONB'{}'::JSONBOptions
index_typeTEXT'vector'Type of index to use for the vector table.
Index_types

  • If index_type is set to vector, the system will automatically create a hnsw index on the vector table based on the distance operator used in the retriever. This is the default index type. The vector index type is only able to support 2000 dimensions or less. If more dimensions are needed, the index type should be set to disabled.

    distance_operatorindex_type
    L2vector_l2_ops
    InnerProductvector_ip_ops
    Cosinevector_cosine_ops
    L1vector_l1_ops

  • If index_type is set to ivfflat, the system will create a IVFFlat index on the vector table.

  • If index_type is set to disabled, no index will be created.

Example

SELECT aidb.create_retriever_for_volume(
               name => 'demo_vol_retriever',
               model_name => 'simple_model',
               source_volume_name => 'demo_bucket_vol'
       );

aidb.set_retriever_auto_processing (deprecated)

Replaced by aidb.set_auto_knowledge_base

Sets the auto-processing mode for this knowledge base.

Parameters

ParameterTypeDefaultDescription
retriever_nameTEXTName of retriever for which to enable auto-embedding
modeaidb.PipelineAutoProcessingModeDesired auto-processing mode

Example

SELECT aidb.set_retriever_auto_processing('test_retriever', 'Live');
SELECT aidb.set_retriever_auto_processing('test_retriever', 'Disabled');

aidb.delete_retriever (deprecated)

Replaced by aidb.delete_knowledge_base

Deletes only the retriever's configuration from the database.

Parameters

ParameterTypeDefaultDescription
retriever_nameTEXTName of retriever to delete

Example

select aidb.delete_retriever('test_retriever');
Output
 delete_retriever
------------------

(1 row)
← Prev

Pipelines PGFS reference

↑ Up

Pipelines reference