Instill Model

The Instill Model component is an AI component that allows users to connect the AI models served on the Instill Model Platform. It can carry out the following tasks:

#Release Stage

Alpha

#Configuration

The component configuration is defined and maintained here.

#Supported Tasks

#Classification

Classify images into predefined categories.

InputIDTypeDescription
Task ID (required)taskstringTASK_CLASSIFICATION
Model Name (required)model-namestringThe Instill Model model to be used.
Image (required)image-base64stringImage base64
OutputIDTypeDescription
CategorycategorystringThe predicted category of the input.
ScorescorenumberThe confidence score of the predicted category of the input.

#Instance Segmentation

Detect, localize and delineate multiple objects in images.

InputIDTypeDescription
Task ID (required)taskstringTASK_INSTANCE_SEGMENTATION
Model Name (required)model-namestringThe Instill Model model to be used.
Image (required)image-base64stringImage base64
OutputIDTypeDescription
Objectsobjectsarray[object]A list of detected instance bounding boxes.

#Keypoint

Detect and localize multiple keypoints of objects in images.

InputIDTypeDescription
Task ID (required)taskstringTASK_KEYPOINT
Model Name (required)model-namestringThe Instill Model model to be used.
Image (required)image-base64stringImage base64
OutputIDTypeDescription
Objectsobjectsarray[object]A list of keypoint objects, a keypoint object includes all the pre-defined keypoints of a detected object.

#Detection

Detect and localize multiple objects in images.

InputIDTypeDescription
Task ID (required)taskstringTASK_DETECTION
Model Name (required)model-namestringThe Instill Model model to be used.
Image (required)image-base64stringImage base64
OutputIDTypeDescription
Objectsobjectsarray[object]A list of detected objects.

#Ocr

Detect and recognize text in images.

InputIDTypeDescription
Task ID (required)taskstringTASK_OCR
Model Name (required)model-namestringThe Instill Model model to be used.
Image (required)image-base64stringImage base64
OutputIDTypeDescription
Objectsobjectsarray[object]A list of detected bounding boxes.

#Semantic Segmentation

Classify image pixels into predefined categories.

InputIDTypeDescription
Task ID (required)taskstringTASK_SEMANTIC_SEGMENTATION
Model Name (required)model-namestringThe Instill Model model to be used.
Image (required)image-base64stringImage base64
OutputIDTypeDescription
Stuffsstuffsarray[object]A list of RLE binary masks.

#Text Generation

Generate texts from input text prompts.

InputIDTypeDescription
Task ID (required)taskstringTASK_TEXT_GENERATION
Model Name (required)model-namestringThe Instill Model model to be used.
Prompt (required)promptstringThe prompt text
System messagesystem-messagestringThe system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. By default, the model’s behavior is using a generic message as "You are a helpful assistant."
Extra Parametersextra-paramsobjectExtra Parameters
Prompt Imagesprompt-imagesarray[string]The prompt images
Chat historychat-historyarray[object]Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : {"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"{.
SeedseedintegerThe seed
TemperaturetemperaturenumberThe temperature for sampling
Top Ktop-kintegerTop k for sampling
Max new tokensmax-new-tokensintegerThe maximum number of tokens for model to generate
OutputIDTypeDescription
TexttextstringText

#Text Generation Chat

Generate texts from input text prompts and chat history.

InputIDTypeDescription
Task ID (required)taskstringTASK_TEXT_GENERATION_CHAT
Model Name (required)model-namestringThe Instill Model model to be used.
Prompt (required)promptstringThe prompt text
System messagesystem-messagestringThe system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. By default, the model’s behavior is using a generic message as "You are a helpful assistant."
Extra Parametersextra-paramsobjectExtra Parameters
Prompt Imagesprompt-imagesarray[string]The prompt images
Chat historychat-historyarray[object]Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : {"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"{.
SeedseedintegerThe seed
TemperaturetemperaturenumberThe temperature for sampling
Top Ktop-kintegerTop k for sampling
Max new tokensmax-new-tokensintegerThe maximum number of tokens for model to generate
OutputIDTypeDescription
TexttextstringText

#Text To Image

Generate images from input text prompts.

InputIDTypeDescription
Task ID (required)taskstringTASK_TEXT_TO_IMAGE
Model Name (required)model-namestringThe Instill Model model to be used.
Prompt (required)promptstringThe prompt text
Extra Parametersextra-paramsobjectExtra Parameters
CFG Scalecfg-scalenumberThe guidance scale, default is 7.5
SamplessamplesintegerThe number of generated samples, default is 1
SeedseedintegerThe seed, default is 0
StepsstepsintegerThe steps, default is 5
OutputIDTypeDescription
Imagesimagesarray[string]Images

#Visual Question Answering

Answer questions based on a prompt and an image.

InputIDTypeDescription
Task ID (required)taskstringTASK_VISUAL_QUESTION_ANSWERING
Model Name (required)model-namestringThe Instill Model model to be used.
Prompt (required)promptstringThe prompt text
System messagesystem-messagestringThe system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. By default, the model’s behavior is using a generic message as "You are a helpful assistant."
Extra Parametersextra-paramsobjectExtra Parameters
Prompt Images (required)prompt-imagesarray[string]The prompt images
Chat historychat-historyarray[object]Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : {"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"{.
SeedseedintegerThe seed
TemperaturetemperaturenumberThe temperature for sampling
Top Ktop-kintegerTop k for sampling
Max new tokensmax-new-tokensintegerThe maximum number of tokens for model to generate
OutputIDTypeDescription
TexttextstringText

#Image To Image

Generate image from input text prompt and image.

InputIDTypeDescription
Task ID (required)taskstringTASK_IMAGE_TO_IMAGE
Model Name (required)model-namestringThe Instill Model model to be used.
Prompt (required)promptstringThe prompt text
Extra Parametersextra-paramsobjectExtra Parameters
Prompt Image (required)image-base64stringThe prompt image
CFG Scalecfg-scalenumberThe guidance scale, default is 7.5
SeedseedintegerThe seed
SamplessamplesintegerThe number of generated samples, default is 1
Top Ktop-kintegerTop k for sampling
OutputIDTypeDescription
Imagesimagesarray[string]Images

Last updated: 6/18/2024, 11:35:38 AM