Model Inference | Documentation

⚗️ Instill Model provides an automated model inference server. You can perform an inference either from the Model Overview page, or via an API Endpoint.

INFO

In the context of Machine Learning (ML) and Artificial Intelligence (AI), the term inference is often compared with training. An inference is where capabilities learnt during model training are used to "infer" a result, typically a prediction about the input data.

#Inference in the Model Overview Page

Navigate to and select your chosen model to bring up its corresponding Model Overview page.

Provide the necessary data or upload files as inputs.
Click the Run button to perform an inference from the model and receive the results in the output window.

TIP

To ensure that the model version you are running is able to serve inference requests, you can check the status of the model version by selecting the Versions tab. To learn more about this please refer to the Model State page.

#Inference via API Endpoint

Once a model version is deployed, it automatically creates a unique API Endpoint for model inference at /users/USER_ID/models/MODEL_ID/versions/VERSION_TAG/trigger. You can find the full cURL snippet for running a particular model in the API tab from its Overview page in the Console.

#Example API Request

Please see the following example to understand how to call the API for an image classification model using cURL or Python.

cURL

Python

export INSTILL_API_TOKEN=********
curl -X POST http://localhost:8080/v1alpha/users/USER_ID/models/MODEL_ID/versions/VERSION_TAG/trigger \
--header "Authorization: $INSTILL_API_TOKEN" \
--data '{
    "taskInputs": [
         {
            "data": {
                "image-url": "https://artifacts.instill.tech/imgs/bear.jpg",
                "type": "image-url",
            }
        },
         {
            "data": {
                "image-url": "https://artifacts.instill.tech/imgs/dog.jpg",
                "type": "image-url",
            }
        }
    ]
}'

This endpoint also allows you to send multiple images in popular formats (.PNG and .JPEG) in a single request. Please see the below examples which show how to call the API using Base64 encoded strings and multipart uploads.

cURL(base64)

cURL(multipart)

export INSTILL_API_TOKEN=********
curl -X POST http://localhost:8080/v1alpha/users/USER_ID/models/MODEL_ID/versions/VERSION_TAG/trigger \
--header "Authorization: $INSTILL_API_TOKEN" \
--data '{
    "taskInputs": [
        {
            "classification": {
                "image-base64": "/9j/4AAQSkZ...iiigD/2Q=="
            }
        },
        {
            "classification": {
                "image-base64": "/9j/4AAQSkZ...iiigD/2Q=="
            }
        }
    ]
}'

The USER_ID, MODEL_ID and VERSION_TAG correspond to the user namespace, the ID of the model and the version tag of the model.

#Connect Models in 💧 Instill VDP

To build pipelines for your AI workflows with models served in ⚗️ Instill Model , you can utilize the AI component for ⚗️ Instill Model within 💧 Instill VDP.