Instill Model | Documentation

The Instill Model AI Connector enables you to connect the AI models served on the Instill Model Platform for various AI tasks:

Classification
Object Detection
Semantic Segmentation
Instance Segmentation
OCR
Keypoint
Text Generation
Text to Image

#Release Stage

Alpha

#Resource Configuration

The up-to-date configuration is maintained here.

Field	Type	Note
api_token	string	To access models on Instill Cloud, enter your Instill Cloud API Token. You can find your tokens by visiting your Instill Cloud's Settings > API Tokens page. Leave this field empty to access models on your local Instill Model.
server_url*	string	Base URL for the Instill Model API. To access models on Instill Cloud, use the base URL `https://api.instill.tech`. To access models on your local Instill Model, use the base URL `http://localhost:9080`.

#No-code Setup

INFO

Comparing to launching a local Instill Model, Instill Cloud is the fastest way to create an Instill Model AI Connector.

👉 You can try out Instill Cloud to use our pre-created Instill Model AI connector for FREE during Open Alpha.

To create an Instill Model AI connector from Instill Cloud, you'll first need to create an Instill Cloud account by visiting console.instill.tech.

Visit Model Hub on Instill Cloud to explore available models
Go to the Resource page and click Add Resource
Select Instill Model
Fill an ID, this will be the unique identifier of this connector
[Optional] Give a short description of your connector in the Description field
Fill in the following information and click Save
- Enter API Token. You can find your Instill Cloud API tokens by visiting your Settings > API Tokens page.
- Set the the Server URL https://api.instill.tech to access models on Instill Cloud

To create an Instill Model AI connector from local Instill Model, follow the same steps, follow the same steps as above, but with the following difference in step 1 and 6:

Visit local Instill Model from local Console](http://localhost:3000/model-hub) to create and deploy your models

and

Fill the following information and click Set up
- Leave the API Key field empty to access models on your local Instill Model
- Set the Server URL to http://localhost:9080 to access models on your local Instill Model.
- Enter the ID of the model served on your local Instill Model that you want to use

#Low-code Setup

cURL(Instill-Cloud)

cURL(Instill-Core)

curl -X POST https://api.instill.tech/v1alpha/users/<user-id>/connector-resources \
--header 'Authorization: Bearer <Instill-Cloud-API-Token>' \
--data '{
  "id": "my-instill-model",
  "connector_definition_name": "connector-definitions/ai-instill-model",
  "configuration": {
    "api_token": "<instill-cloud-api-token>",
    "server_url": "https://api.instill.tech"
  }
}'

For other operations, please refer to the VDP Protobufs.

#Connector Configuration

When using the Instill Model AI connector in your pipeline, certain fields of the input data need to be filled to trigger the pipeline effectively. Here are the expected input and output data fields for the Instill Model AI connector.

#Classification

Input	Type	Description
task	string	`TASK_CLASSIFICATION`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
image_base64	string	Image as base64 encoded string

Output	Type	Description
category	string	The predicted category of the input
score	number	The confidence score of the predicted category of the input

#Object Detection

Input	Type	Description
task	string	`TASK_OBJECT_DETECTION`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
image_base64	string	Image as base64 encoded string

Output	Type	Description
objects	array[object]	A list of detected instance bounding boxes

#Semantic Segmentation

Input	Type	Description
task	string	`TASK_SEMANTIC_SEGMENTATION`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
image_base64	string	Image as base64 encoded string

Output	Type	Description
stuffs	array[object]	A list of RLE binary masks

#Instance Segmentation

Input	Type	Description
task	string	`TASK_INSTANCE_SEGMENTATION`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
image_base64	string	Image as base64 encoded string

Output	Type	Description
objects	array[object]	A list of detected instance bounding boxes

#OCR

Input	Type	Description
task	string	`TASK_OCR`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
image_base64	string	Image as base64 encoded string

Output	Type	Description
objects	array[object]	A list of detected bounding boxes

#Keypoint

Input	Type	Description
task	string	`TASK_KEYPOINT`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
image_base64	string	Image as base64 encoded string

Output	Type	Description
objects	array[object]	A list of keypoint objects, a keypoint object includes all the pre-defined keypoints of a detected object

#Text Generation

Input	Type	Description
task	string	`TASK_TEXT_GENERATION`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
prompt	string	Text prompt
output_len	int	Output length
bad_words_list	string	Bad words formatted in a string
stop_words_list	string	Stop words formatted in a string
top_k	int	Top K
seed	int	Random noise seed

Output	Type	Description
text	string	Generated text

#Text to Image

Input	Type	Description
task	string	`TASK_TEXT_TO_IMAGE`
model_namespace	string	The namespace of the model to be used from Instill Model
model_id	string	The ID of the model that you want to use from Instill Model
prompt	string	Text prompt
cfg_scale	float	How strictly the diffusion process adheres to the prompt text
steps	int	Number of diffusion steps to run
samples	int	Number of images to generate
seed	int	Random noise seed

Output	Type	Description
images	array[image]	A list of generated images as base64 encoded string