The Milvus component is a data component that allows users to build and search vector datasets.
It can carry out the following tasks:
#Release Stage
Alpha
#Configuration
The component definition and tasks are defined in the definition.json and tasks.json files respectively.
#Setup
In order to communicate with Milvus, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's setup
block, or you can create a Connection from
the Integration Settings
page and reference the whole setup
as setup: ${connection.<my-connection-id>}
.
Field | Field ID | Type | Note |
---|
Milvus URL Endpoint (required) | url | string | Fill in your Milvus public URL endpoint with port number, e.g http://3.25.202.142:19530. |
Milvus Username (required) | username | string | Fill in your Milvus username. |
Milvus Password (required) | password | string | Fill in your Milvus password. |
#Supported Tasks
#Vector Search
Perform a vector search on a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_VECTOR_SEARCH |
Collection Name (required) | collection-name | string | The name of the collection to perform vector search on. |
Partition Name | partition-name | string | The name of the partition to vector search the data from. |
Vector (required) | vector | array[number] | An array of dimensions for the vector search. |
Vector Field (required) | vector-field | string | The name of the field to perform vector search on. |
Fields | fields | array[string] | The fields to return in the data. If empty then all fields will be returned. |
Limit (required) | limit | integer | The limit of the data to return. |
Filter | filter | string | The properties filter to be applied to the data with milvus scalar filter, please refer to filter-search. |
Offset | offset | integer | The offset of the data to return. |
Grouping Field | grouping-field | string | The name of the field to group the data by, please refer to Grouping-search. |
Search Parameters | search-params | object | The search parameters to be applied to the data with milvus search parameters, please refer to Search-parameters. |
Output | ID | Type | Description |
---|
Result | result | object | Result of the vector search operation. |
Status | status | string | Vector search status. |
Output Objects in Vector Search
Result
Field | Field ID | Type | Note |
---|
Data | data | array | The points returned from the vector search operation. |
IDs | ids | array | The ids returned from the vector search operation. |
Metadata | metadata | array | The metadata returned from the vector search operation. |
Vectors | vectors | array | The vectors returned from the vector search operation. |
#Upsert
Insert a vector data into a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_UPSERT |
Collection Name (required) | collection-name | string | The name of the collection to upsert the data into. |
Partition Name | partition-name | string | The name of the partition to upsert the data from. If empty then default partition will be used. |
Data (required) | data | object | The data. |
Output | ID | Type | Description |
---|
Status | status | string | Upsert status. |
#Batch Upsert
Insert a batch of vector data into a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_BATCH_UPSERT |
Collection Name (required) | collection-name | string | The name of the collection to upsert the data into. |
Partition Name | partition-name | string | The name of the partition to upsert the data from. If empty then default partition will be used. |
Array Data (required) | array-data | array[object] | The data. |
Output | ID | Type | Description |
---|
Status | status | string | Batch upsert status. |
#Delete
Delete vector data from a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DELETE |
Collection Name (required) | collection-name | string | The name of the collection to delete the data from. |
Partition Name | partition-name | string | The name of the partition to delete the data from. If empty then default partition will be used. |
Filter | filter | string | The properties filter to be applied to the data with milvus scalar filter, please refer to Filtered-search. |
Output | ID | Type | Description |
---|
Status | status | string | Delete status. |
#Create Collection
Create a collectio, please refer to Create-Collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_CREATE_COLLECTION |
Collection Name (required) | collection-name | string | The name of the collection to create. |
Dimension (required) | dimension | integer | The dimension of the collection. |
ID Type | id-type | string | The type of the id. |
Schema | schema | object | The schema of the collection. |
Auto ID | auto-id | boolean | Whether to auto generate id. |
Metric Type | metric-type | string | The metric type of the collection.
Enum values |
Index Parameters | index-params | array[object] | The index parameters to be applied to the collection with milvus index parameters, please refer to Create. |
Parameters | params | object | The parameters to be applied to the collection with milvus parameters, please refer to Parameters. |
Output | ID | Type | Description |
---|
Status | status | string | Create collection status. |
#Drop Collection
Drop a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DROP_COLLECTION |
Collection Name (required) | collection-name | string | The name of the collection to drop. |
Output | ID | Type | Description |
---|
Status | status | string | Drop collection status. |
#Create Partition
Create a partition in a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_CREATE_PARTITION |
Collection Name (required) | collection-name | string | The name of the collection to create the partition in. |
Partition Name (required) | partition-name | string | The name of the partition to create. |
Output | ID | Type | Description |
---|
Status | status | string | Create partition status. |
#Drop Partition
Drop a partition from a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DROP_PARTITION |
Collection Name (required) | collection-name | string | The name of the collection to drop the partition from. |
Partition Name (required) | partition-name | string | The name of the partition to drop. |
Output | ID | Type | Description |
---|
Status | status | string | Drop partition status. |
#Create Index
Create an index in a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_CREATE_INDEX |
Collection Name (required) | collection-name | string | The name of the collection to create the index in. |
Index Parameters (required) | index-params | object | The index parameters to be applied to the collection with milvus index parameters, please refer to Create. |
Output | ID | Type | Description |
---|
Status | status | string | Create index status. |
#Drop Index
Drop an index from a collection
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_DROP_INDEX |
Collection Name (required) | collection-name | string | The name of the collection to drop the index from. |
Index Name (required) | index-name | string | The name of the index to drop. |
Output | ID | Type | Description |
---|
Status | status | string | Drop index status. |