The Instill Model AI Connector enables you to connect the AI models served on the Instill Model Platform for various AI tasks:
- Classification
- Object Detection
- Semantic Segmentation
- Instance Segmentation
- OCR
- Keypoint
- Text Generation
- Text to Image
#Release Stage
Alpha
#Resource Configuration
The up-to-date configuration is maintained here.
Field | Type | Note |
---|---|---|
api_token | string | To access models on Instill Cloud, enter your Instill Cloud API Token. You can find your tokens by visiting your Instill Cloud's Settings > API Tokens page. Leave this field empty to access models on your local Instill Model. |
server_url* | string | Base URL for the Instill Model API. To access models on Instill Cloud, use the base URL https://api.instill.tech . To access models on your local Instill Model, use the base URL http://localhost:9080 . |
#No-code Setup
Comparing to launching a local Instill Model, Instill Cloud is the fastest way to create an Instill Model AI Connector.
👉 You can try out Instill Cloud to use our pre-created Instill Model AI connector for FREE during Open Alpha.
To create an Instill Model AI connector from Instill Cloud, you'll first need to create an Instill Cloud account by visiting console.instill.tech.
-
Visit Model Hub on Instill Cloud to explore available models
-
Go to the AI page of your local Console and click Set up new AI
-
In the Set Up New AI Connector page, fill an ID for your model, this will be the unique identifier of this connector
-
[Optional] Give a short description of your connector in the Description field
-
Click the AI Connector Type ▾ drop-down and choose Instill Model
-
Fill in the following information and click Set up
- Enter API Token. You can find your Instill Cloud API tokens by visiting your Settings > API Tokens page.
- Set the the Server URL https://api.instill.tech to access models on Instill Cloud
- Enter the ID of the model from Model Hub on Instill Cloud that you want to use
-
By default, the connector is disconnected. To connect and use AI connector, go to the connector page and click Connect.
To create an Instill Model AI connector from local Instill Model, follow the same steps, follow the same steps as above, but with the following difference in step 1 and 6:
- Visit local Instill Model from local Console](http://localhost:3000/model-hub) to create and deploy your models
and
-
Fill the following information and click Set up
- Leave the API Key field empty to access models on your local Instill Model
- Set the Server URL to http://localhost:9080 to access models on your local Instill Model.
- Enter the ID of the model served on your local Instill Model that you want to use
#Low-code Setup
For other operations, please refer to the VDP Protobufs.
#Connector Configuration
When using the Instill Model AI connector in your pipeline, certain fields of the input data need to be filled to trigger the pipeline effectively. Here are the expected input and output data fields for the Instill Model AI connector.
#Classification
Input | Type | Description |
---|---|---|
task | string | TASK_CLASSIFICATION |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
image_base64 | string | Image as base64 encoded string |
Output | Type | Description |
---|---|---|
category | string | The predicted category of the input |
score | number | The confidence score of the predicted category of the input |
#Object Detection
Input | Type | Description |
---|---|---|
task | string | TASK_OBJECT_DETECTION |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
image_base64 | string | Image as base64 encoded string |
Output | Type | Description |
---|---|---|
objects | array[object] | A list of detected instance bounding boxes |
#Semantic Segmentation
Input | Type | Description |
---|---|---|
task | string | TASK_SEMANTIC_SEGMENTATION |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
image_base64 | string | Image as base64 encoded string |
Output | Type | Description |
---|---|---|
stuffs | array[object] | A list of RLE binary masks |
#Instance Segmentation
Input | Type | Description |
---|---|---|
task | string | TASK_INSTANCE_SEGMENTATION |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
image_base64 | string | Image as base64 encoded string |
Output | Type | Description |
---|---|---|
objects | array[object] | A list of detected instance bounding boxes |
#OCR
Input | Type | Description |
---|---|---|
task | string | TASK_OCR |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
image_base64 | string | Image as base64 encoded string |
Output | Type | Description |
---|---|---|
objects | array[object] | A list of detected bounding boxes |
#Keypoint
Input | Type | Description |
---|---|---|
task | string | TASK_KEYPOINT |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
image_base64 | string | Image as base64 encoded string |
Output | Type | Description |
---|---|---|
objects | array[object] | A list of keypoint objects, a keypoint object includes all the pre-defined keypoints of a detected object |
#Text Generation
Input | Type | Description |
---|---|---|
task | string | TASK_TEXT_GENERATION |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
prompt | string | Text prompt |
output_len | int | Output length |
bad_words_list | string | Bad words formatted in a string |
stop_words_list | string | Stop words formatted in a string |
top_k | int | Top K |
seed | int | Random noise seed |
Output | Type | Description |
---|---|---|
text | string | Generated text |
#Text to Image
Input | Type | Description |
---|---|---|
task | string | TASK_TEXT_TO_IMAGE |
model_namespace | string | The namespace of the model to be used from Instill Model |
model_id | string | The ID of the model that you want to use from Instill Model |
prompt | string | Text prompt |
cfg_scale | float | How strictly the diffusion process adheres to the prompt text |
steps | int | Number of diffusion steps to run |
samples | int | Number of images to generate |
seed | int | Random noise seed |
Output | Type | Description |
---|---|---|
images | array[image] | A list of generated images as base64 encoded string |