Stability AI | Documentation

The Stability AI AI connector enables you to connect the AI models served on the Stability AI Platform for the following tasks:

Text to Image
Image to Image

#Release Stage

Alpha

#Resource Configuration

The up-to-date configuration is maintained here.

Field	Type	Note
api_key*	string	Enter your Stability AI API key. To find your keys, go to your DreamStudio's Account page.

#No-code Setup

PREREQUISITES

To create a Stability AI AI connector, you'll first need to create a Stability AI account by visiting dreamstudio.ai. Access to your API keys can then be managed through DreamStudio's Account page. All new users are granted 25 free credits upon creating an account.

To create a Stability AI AI connector:

Go to the AI page and click Set up new AI
In the Set Up New AI Connector page, fill an ID for your model, this will be the unique identifier of this connector
[Optional] Give a short description of your connector in the Description field
Click the AI Connector Type ▾ drop-down and choose Stability AI
Fill the following information and click Set up
- Fill API Key. Follow the Stability AI documentation to find your API key
By default, the connector is disconnected. To connect and use AI connector, go to the connector page and click Connect.

#Low-code Setup

cURL(Instill-Cloud)

cURL(Instill-Core)

curl -X POST http://localhost:8080/vdp/v1alpha/users/<user-id>/connector-resources \
--header 'Authorization: Bearer <Instill-Cloud-API-Token>' \
--data '{
  "id": "my-stable-diffusion",
  "connector_definition_name": "connector-definitions/ai-stability-ai",
  "configuration": {
    "api_key": "<stability-ai-api-key>"
  }
}'

For other operations, please refer to the VDP Protobufs.

#Connector Configuration

When using the Stability AI AI connector in your pipeline, certain fields of the input data need to be filled to trigger the pipeline effectively. Here are the expected input and output data fields for the Stability AI AI connector.

#Text to Image

Input	Type	Description
task	string	`TASK_TEXT_TO_IMAGE`
engine	string	Stability AI Engine (model) to be used. eg. stable-diffusion-xl-1024-v1-0, stable-diffusion-v1
prompts	array[string]	This has the text prompt, required for all tasks.
cfg_scale	number	How strictly the diffusion process adheres to the prompt text, optional for all tasks.
clip_guidance_preset	string	One from: FAST_BLUE, FAST_GREEN, NONE, SIMPLE, SLOW, SLOWER, SLOWEST, optional for all tasks.
sampler	string	Which sampler to use for the diffusion process, optional for all tasks.
samples	integer	Number of images to generate, optional for all tasks.
seed	integer	Random noise seed, optional for all tasks.
steps	integer	Number of diffusion steps to run, optional for all tasks.
style_preset	string	To guide the image model towards a particular style, optional for all tasks.
weights	array[number]	Weights for each of the prompts in texts, optional for all tasks.
height	integer	Height of the output image, optional.
width	integer	Width of the output image, optional.

Output	Type	Description
images	array[image]	Output images as base64 encoded string.
seeds	array[integer]	Seeds for output images.

#Image to Image

Input	Type	Description
task	string	`TASK_IMAGE_TO_IMAGE`
engine	string	Stability AI Engine (model) to be used. eg. stable-diffusion-xl-1024-v1-0, stable-diffusion-v1
init_image	string	Specify initial image
prompts	array[string]	This has the text prompt
cfg_scale	number	How strictly the diffusion process adheres to the prompt text, optional for all tasks.
clip_guidance_preset	string	One from: FAST_BLUE, FAST_GREEN, NONE, SIMPLE, SLOW, SLOWER, SLOWEST, optional for all tasks.
sampler	string	Which sampler to use for the diffusion process, optional for all tasks.
samples	integer	Number of images to generate, optional for all tasks.
seed	integer	Random noise seed, optional for all tasks.
steps	integer	Number of diffusion steps to run, optional for all tasks.
style_preset	string	To guide the image model towards a particular style, optional for all tasks.
weights	array[number]	Weights for each of the prompts in texts, optional for all tasks.
height	integer	Height of the output image, optional.
width	integer	Width of the output image, optional.
init_image_mode	string	Whether to use image_strength or step_schedule, optional.
image_strength	number	How much influence the init_image has on the diffusion process, optional.

Output	Type	Description
images	array[image]	This will contain the output images as base64 encoded string.
seeds	array[integer]	This will contain the seeds for output images.

#Example

Example input and output data for each task:

TASK_TEXT_TO_IMAGE

TASK_IMAGE_TO_IMAGE

// Input
{
  "prompts": ["A blue cat bakes a birthday cake"],
  "cfg_scale": 7.0,
  "clip_guidance_preset": "SLOWER",
  "samples": 3,
  "style_preset": "cinematic",
  "weights": [0.5, 0.7],
  "height": 512,
  "width": 512
}
// Output
{
  "images": ["<base64-encoded image>"],
  "seeds": [1769541116]
}