Model Inference | Documentation

⚗️ Instill Model provides an automated model inference server. You can perform an inference either from the Model Overview page, or via an API Endpoint.

INFO

In the context of Machine Learning (ML) and Artificial Intelligence (AI), the term inference is often compared with training. An inference is where capabilities learnt during model training are used to "infer" a result, typically a prediction about the input data.

#Inference in the Model Overview Page

Navigate to and select your chosen model to bring up its corresponding Model Overview page.

Provide the necessary data or upload files as inputs.
Click the Run button to perform an inference from the model and receive the results in the output window.

TIP

To ensure that the model version you are running is able to serve inference requests, you can check the status of the model version by selecting the Versions tab. To learn more about this please refer to the Model State page.

#Inference via API Endpoint

Once a model version is deployed, it automatically creates a unique API Endpoint for model inference at /users/USER_ID/models/MODEL_ID/versions/VERSION_TAG/trigger.

This endpoint allows you to send multiple images in popular formats (.PNG and .JPEG) in a single request. As shown in the examples below, the API is designed to accept batched images:

Sent by remote URL,
Sent as Base64 encoded strings,
Uploaded by multipart.

cURL(url)

cURL(base64)

cURL(multipart)

curl -X POST http://localhost:8080/v1alpha/users/USER_ID/models/MODEL_ID/versions/VERSION_TAG/trigger \
--header 'Authorization: Bearer instill_sk_***' \
--data '{
    "taskInputs": [
        {
            "classification": {
                "imageUrl": "https://artifacts.instill.tech/imgs/dog.jpg"
            }
        },
        {
            "classification": {
                "imageUrl": "https://artifacts.instill.tech/imgs/bear.jpg"
            }
        }
    ]
}'

The USER_ID, MODEL_ID and VERSION_TAG correspond to the user namespace, the ID of the model and the version tag of the model.

#Connect Models in 💧 Instill VDP

To build pipelines for your AI workflows with models served in ⚗️ Instill Model , you can utilize the AI component for ⚗️ Instill Model within 💧 Instill VDP.