Milvus

The Milvus component is a data component that allows users to build and search vector datasets. It can carry out the following tasks:

#Release Stage

Alpha

#Configuration

The component definition and tasks are defined in the definition.json and tasks.json files respectively.

#Setup

In order to communicate with Milvus, the following connection details need to be provided. You may specify them directly in a pipeline recipe as key-value pairs within the component's setup block, or you can create a Connection from the Integration Settings page and reference the whole setup as setup: ${connection.<my-connection-id>}.

FieldField IDTypeNote
Milvus URL Endpoint (required)urlstringFill in your Milvus public URL endpoint with port number, e.g http://3.25.202.142:19530.
Milvus Username (required)usernamestringFill in your Milvus username.
Milvus Password (required)passwordstringFill in your Milvus password.

#Supported Tasks

Perform a vector search on a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_VECTOR_SEARCH
Collection Name (required)collection-namestringThe name of the collection to perform vector search on.
Partition Namepartition-namestringThe name of the partition to vector search the data from.
Vector (required)vectorarray[number]An array of dimensions for the vector search.
Vector Field (required)vector-fieldstringThe name of the field to perform vector search on.
Fieldsfieldsarray[string]The fields to return in the data. If empty then all fields will be returned.
Limit (required)limitintegerThe limit of the data to return.
FilterfilterstringThe properties filter to be applied to the data with milvus scalar filter, please refer to filter-search.
OffsetoffsetintegerThe offset of the data to return.
Grouping Fieldgrouping-fieldstringThe name of the field to group the data by, please refer to Grouping-search.
Search Parameterssearch-paramsobjectThe search parameters to be applied to the data with milvus search parameters, please refer to Search-parameters.
OutputIDTypeDescription
ResultresultobjectResult of the vector search operation.
StatusstatusstringVector search status.
Output Objects in Vector Search

Result

FieldField IDTypeNote
DatadataarrayThe points returned from the vector search operation.
IDsidsarrayThe ids returned from the vector search operation.
MetadatametadataarrayThe metadata returned from the vector search operation.
VectorsvectorsarrayThe vectors returned from the vector search operation.

#Upsert

Insert a vector data into a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_UPSERT
Collection Name (required)collection-namestringThe name of the collection to upsert the data into.
Partition Namepartition-namestringThe name of the partition to upsert the data from. If empty then default partition will be used.
Data (required)dataobjectThe data.
OutputIDTypeDescription
StatusstatusstringUpsert status.

#Batch Upsert

Insert a batch of vector data into a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_BATCH_UPSERT
Collection Name (required)collection-namestringThe name of the collection to upsert the data into.
Partition Namepartition-namestringThe name of the partition to upsert the data from. If empty then default partition will be used.
Array Data (required)array-dataarray[object]The data.
OutputIDTypeDescription
StatusstatusstringBatch upsert status.

#Delete

Delete vector data from a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_DELETE
Collection Name (required)collection-namestringThe name of the collection to delete the data from.
Partition Namepartition-namestringThe name of the partition to delete the data from. If empty then default partition will be used.
FilterfilterstringThe properties filter to be applied to the data with milvus scalar filter, please refer to Filtered-search.
OutputIDTypeDescription
StatusstatusstringDelete status.

#Create Collection

Create a collectio, please refer to Create-Collection

InputIDTypeDescription
Task ID (required)taskstringTASK_CREATE_COLLECTION
Collection Name (required)collection-namestringThe name of the collection to create.
Dimension (required)dimensionintegerThe dimension of the collection.
ID Typeid-typestringThe type of the id.
SchemaschemaobjectThe schema of the collection.
Auto IDauto-idbooleanWhether to auto generate id.
Metric Typemetric-typestringThe metric type of the collection.
Enum values
  • L2
  • IP
  • COSINE
Index Parametersindex-paramsarray[object]The index parameters to be applied to the collection with milvus index parameters, please refer to Create.
ParametersparamsobjectThe parameters to be applied to the collection with milvus parameters, please refer to Parameters.
OutputIDTypeDescription
StatusstatusstringCreate collection status.

#Drop Collection

Drop a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_DROP_COLLECTION
Collection Name (required)collection-namestringThe name of the collection to drop.
OutputIDTypeDescription
StatusstatusstringDrop collection status.

#Create Partition

Create a partition in a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_CREATE_PARTITION
Collection Name (required)collection-namestringThe name of the collection to create the partition in.
Partition Name (required)partition-namestringThe name of the partition to create.
OutputIDTypeDescription
StatusstatusstringCreate partition status.

#Drop Partition

Drop a partition from a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_DROP_PARTITION
Collection Name (required)collection-namestringThe name of the collection to drop the partition from.
Partition Name (required)partition-namestringThe name of the partition to drop.
OutputIDTypeDescription
StatusstatusstringDrop partition status.

#Create Index

Create an index in a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_CREATE_INDEX
Collection Name (required)collection-namestringThe name of the collection to create the index in.
Index Parameters (required)index-paramsobjectThe index parameters to be applied to the collection with milvus index parameters, please refer to Create.
OutputIDTypeDescription
StatusstatusstringCreate index status.

#Drop Index

Drop an index from a collection

InputIDTypeDescription
Task ID (required)taskstringTASK_DROP_INDEX
Collection Name (required)collection-namestringThe name of the collection to drop the index from.
Index Name (required)index-namestringThe name of the index to drop.
OutputIDTypeDescription
StatusstatusstringDrop index status.