The Ollama component is an AI component that allows users to connect the AI models served with the Ollama library. It can carry out the following tasks:
#Release Stage
Alpha
#Configuration
The component definition and tasks are defined in the definition.json and tasks.json files respectively.
#Setup
In order to communicate with Ollama, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's setup block, or you can create a Connection from
the Integration Settings
page and reference the whole setup as setup: ${connection.<my-connection-id>}.
| Field | Field ID | Type | Note |
|---|---|---|---|
| Endpoint (required) | endpoint | string | Fill in your Ollama hosting endpoint. ### WARNING ###: As of 2024-07-26, the Ollama component does not support authentication methods. To prevent unauthorized access to your Ollama serving resources, please implement additional security measures such as IP whitelisting. |
| Model Auto-Pull (required) | auto-pull | boolean | Automatically pull the requested models from the Ollama server if the model is not found in the local cache. |
#Supported Tasks
#Text Generation Chat
Open-source large language models (OSS LLMs) are artificial intelligence models with publicly accessible code and architecture, allowing for free use, modification, and distribution. These models can provide performance comparable to proprietary alternatives. Ollama is a tool that enables you to run and interact with OSS LLMs using limited computational resources. You can install Ollama from: here.
| Input | ID | Type | Description |
|---|---|---|---|
| Task ID (required) | task | string | TASK_TEXT_GENERATION_CHAT |
| Model Name (required) | model | string | The OSS model to be used, check here for list of models available. |
| Prompt (required) | prompt | string | The prompt text. |
| System Message | system-message | string | The system message helps set the behavior of the assistant. For example, you can modify the personality of the assistant or provide specific instructions about how it should behave throughout the conversation. By default, the model’s behavior is set using a generic message as "You are a helpful assistant.". |
| Prompt Images | prompt-images | array[string] | The prompt images. |
| Chat History | chat-history | array[object] | Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : {"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"}. |
| Seed | seed | integer | The seed. |
| Temperature | temperature | number | The temperature for sampling. |
| Top K | top-k | integer | Top k for sampling. |
| Max New Tokens | max-new-tokens | integer | The maximum number of tokens for model to generate. |
Input Objects in Text Generation Chat
Chat History
Incorporate external chat history, specifically previous messages within the conversation. Please note that System Message will be ignored and will not have any effect when this field is populated. Each message should adhere to the format: : {"role": "The message role, i.e. 'system', 'user' or 'assistant'", "content": "message content"}.
| Field | Field ID | Type | Note |
|---|---|---|---|
| Content | content | array | The message content. |
| Role | role | string | The message role, i.e. 'system', 'user' or 'assistant'. |
Content
The message content.
| Field | Field ID | Type | Note |
|---|---|---|---|
| Image URL | image-url | object | The image URL. |
| Text | text | string | The text content. |
| Type | type | string | The type of the content part. Enum values
|
Image URL
The image URL.
| Field | Field ID | Type | Note |
|---|---|---|---|
| URL | url | string | Either a URL of the image or the base64 encoded image data. |
| Output | ID | Type | Description |
|---|---|---|---|
| Text | text | string | Model Output. |
#Local Ollama Instance
To set up an Ollama instance on your local machine, follow the instructions below:
Note: These instructions only work for Instill Core CE
- Follow the tutorial on the official GitHub repository to install Ollama on your machine.
- Follow the instructions in the FAQ section to modify the variable
OLLAMA_HOSTto0.0.0.0, then restart Ollama. - Get the IP address of your machine on the local network.
- On Linux and macOS, open the terminal and type
ifconfig. - On Windows, open the command prompt and type
ipconfig.
- On Linux and macOS, open the terminal and type
- Suppose the IP address is
192.168.178.88, then the Ollama hosting endpoint would be192.168.178.88:11434. - Enjoy fast LLM inference on your local machine and integration with 💧 Instill VDP.
#Text Embeddings
An embedding is a list of floating point numbers that captures semantic information about the text that it represents.
| Input | ID | Type | Description |
|---|---|---|---|
| Task ID (required) | task | string | TASK_TEXT_EMBEDDINGS |
| Model Name (required) | model | string | The OSS model to be used, check here for list of models available. |
| Text (required) | text | string | The text. |
| Output | ID | Type | Description |
|---|---|---|---|
| Embedding | embedding | array[number] | Embedding of the input text. |