Instill Format | Documentation

Instill VDP defines several Instill Formats as data type identifiers. These formats simplify the creation of pipelines by eliminating the complexity of converting unstructured data formats. The supported Instill Formats include primitive data types such as boolean, string, integer, number, and json, as well as unstructured data types like file, document, image, audio, and video, along with array data types.

These Instill Formats enable users to efficiently build pipelines that manage unstructured data in ETL workflows.

#Instill Formats

Instill VDP extends Instill Format from JSON primitive types and MIME types (IANA media types).

#Primitive Data Types

string
number
integer
boolean
json

#Unstructured Data Types

MIME types are defined as <type>/<subtype>, and Instill VDP extends this to categorize unstructured data into five types:

image: e.g., image/jpeg
video: e.g., video/h264
audio: e.g., audio/wav
document: e.g., text/html, application/pdf
file: e.g., application/octet-stream
- Please note that file is a generic type for any file, and the other types (text, image, video, audio, and document) are specialized file types with specific format handling capabilities.

#Auto-Conversion

With Instill Format, users don't need to manually handle format conversion. For instance, if a pipeline accepts a PNG image and a component requires JPEG, they can be directly connected as long as they share the same type image.

Example:

variable:
  image:
    title: A PNG Image
    format: image # User uploads an image in PNG format
component:
  ai-0:
    type: openai
    task: TASK_TEXT_GENERATION
    input:
      prompt: What is in the image?
      prompt-images:
        - ${variable.image} # Component requires input in JPEG format

Auto-conversion not only works within the same type but also supports cross-type conversions. For example, Instill VDP can automatically convert a PDF document into text/markdown format.

Example:

variable:
  pdf:
    title: A PDF Document
    format: document # User uploads a PDF file
component:
  openai-0:
    type: openai
    task: TASK_TEXT_GENERATION
    input:
      prompt: ${variable.pdf} # Component requires text input

Supported Cross-Type Conversions:

document → text

This feature allows users to focus on building business models without worrying about data format conversions, resulting in cleaner, more efficient pipelines.

#Attribute Extraction

Beyond format representation, Instill Format provides attribute extraction capabilities. Supported attribute includes:

Type	Attribute	Description
All Files	`:file-size`	Size of the file in bytes
All Files	`:filename`	Name of the file
All Files	`:content-type`	Content type of the file
All Files	`:data-uri`	Data URI representation of the file
All Files	`:base64`	Base64 representation of the file
Image	`:width`	Width of the image
Image	`:height`	Height of the image
Video	`:duration`	Duration of the video
Video	`:width`	Width of the video
Video	`:height`	Height of the video
Video	`:frame-rate`	Frame rate of the video
Audio	`:duration`	Duration of the audio
Audio	`:sample-rate`	Sample rate of the audio

Example: Accessing the width and height of an image:

variable:
  image:
    title: A PNG Image
    format: image # User uploads an image in PNG format
output:
  bounding-boxes:
    title: Bounding Boxes
    value: ${ai-0.output.bounding-boxes}
  height:
    title: Image Height
    value: ${variable.image:height} # Get the image height
  width:
    title: Image Width
    value: ${variable.image:width} # Get the image width
component:
  ai-0:
    type: instill-model
    task: TASK_DETECTION
    input:
      image: ${variable.image}