Unlocking the Power of Open-Source Large Language Models (LLMs) with Instill Model

#The Importance of Open-Source Large Language Models (LLMs)

Large Language Models (LLMs) are foundational models trained on vast text datasets, encompassing web pages, books, and articles. They are designed to generate text, translate languages, automate tasks, and create various types of content. Proprietary LLMs are typically owned by companies and are only accessible through private APIs. The most prominent proprietary LLMs include OpenAI's ChatGPT, Anthropic's Claude, and Google's PaLM 2 used in Bard chatbot.

However, people have begun to recognize that these proprietary LLMs can impose restrictions on their usage. These LLMs tend to be centralized within a few major tech companies, raising concerns about transparency, bias, robustness, and data privacy. In response to the need for transparency, the LLM community has developed numerous open-source or source-available LLMs. These open-source alternatives compete favorably with very large models in terms of performance. They offer individuals and organizations full control over their data and ensure ethical and legal compliance. With a modest budget and a small amount of data, you can fine-tune small LLMs to achieve impressive results.

#Preparation

Instill Cloud is the recommended platform for accessing these open-source LLMs, offering ease of use and reliability. To get started, follow the quick start guide to set up your Instill Cloud account.

#Access Pre-trained LLMs in Instill Cloud

To access pre-trained AI models hosted in Instill Cloud, log in to the Console and navigate to the Model Hub page. Here, you'll find a collection of pre-trained AI models under the instill-ai account, ready for use in your applications. During the Open Alpha phase, accessing to these models are entirely free.

Each model in the Model Hub is designed to handle specific AI tasks. For detailed information about a specific model, click on its ID and explore the Description and Settings sections.

Below is a list of available LLMs, along with their details and links in Instill Cloud:

The Open LLM Leaderboard ranks Mistral 7B as a strong candidate for practical commercial usage among models below 13B parameters.

#Build an AI pipeline to access the LLMs

In this section, we will build an AI pipeline to compare the outputs of the LLMs served in Instill Cloud.

Step 1: Add the Instill Model connector resource

Instill VDP supports an AI connector to Instill Model where the user can import and serve custom models.

To set up Instill Model connector resource in the Console, follow these steps:

1. Navigate to the Resources page on the left sidebar.
2. Click on + Add Resource, add select Instill Model.
3. Provide a unique ID for the resource. Here we use ins-model. Optionally, you can add a description.
4. Fill in the API Token. To access models in Instill Cloud, enter your Instill Cloud API Token. You can create and find your tokens by visiting the Settings > API Tokens page.
5. Fill in the Server URL with https://api.instill.tech.
6. Click Save.

Now, head to the Pipelines page on the left sidebar. To create your pipeline through the Console, click on + Create Pipeline. This will redirect you to the no-code pipeline builder. The pipeline will be assigned a randomly generated unique ID. You can change it on the right, for example, test-oss-llms. On the canvas, you can drag and drop connectors and operators to construct your pipeline.

Step 2: Build a VDP pipeline using the no-code pipeline builder

The Start component serves as the starting point of a VDP pipeline, where you can define the input data for your applications.

To set up a Start component:

1. Locate the start component on canvas
2. Click Add Field + and select Text. Fill in the Title with Prompt and Key with prompt.
3. Click Save.

This configuration requires one input: a text prompt for all the open-source LLMs.

To configure an Instill Model connector:

1. Click + Create Connector and choose ins-model from existing resource. This will add a OpenAI connector ai_0 to the canvas.
2. Click the ⚙️ icon on the ai_0 to open the configuration panel on the right-hand side.
3. Select TASK_TEXT_GENERATION from the dropdown of the Task field.
4. Set Model ID to choose the model to use, e.g., mpt-7b.
5. Set Model Namespace to instill-ai for the namespace (username) of the model to use.
6. In the Prompt field, specify the text prompt {start.prompt}.
7. Specify a Seed with 0 for a randomization.
8. Click Save.

The Instill Model connector will automatically link to the Start component since we have used {start.prompt} in the Instill Model connector configuration.

With the above steps, we have set up the MPT 7B model. To set up the other two models, LaMA-2 7B and Mistral 7B:

1. Click the ··· icon on the ai_0 and select Duplicate to create another Instill Model Connector, named ai_1, on the canvas.
2. Click the ⚙️ icon on the ai_1 to open configuration right panel.
3. Update Model ID from mpt-7b to llama2-7b.
4. Click Save.

You can repeat the same process to duplicate another component, ai_2, and update the Model ID to mistral-7b to set up Mistral 7B.

The End Component is used at the end of a VDP pipeline to receive the output of the pipeline being triggered.

To set up an End component:

1. Find the end component on canvas.
2. Click Add Field + and fill in the Title with MPT-7B Result, Key with mpt_7b_result and Value with {ai_0.output.text}.
3. Click Add Field + and fill in the Title with Llama2-7B Result, Key with llama2_7b_result and Value with {ai_1.output.text}.
4. Click Add Field + and fill in the Title with Mistral-7B Result, Key with mistral_7b_result and Value with {ai_2.output.text}.
5. Click Save.

You will see that the End component is automatically linked to multiple components: ai_0, ai_1 and ai_2.

Click Save on the top right corner.

#Run the pipeline

🎉 Congratulations! Your pipeline is all set up and ready to go. Now, simply click on the Test button located in the top right corner. You can now put your pipeline to test using real data. For instance, let's input the Prompt as

Click Run.

Test the pipeline with real data

The End component will show the text generation result of each LLM:

Please give it a try, but note that these LLMs models are base models and are not fine-tuned for chat-like dialogue generation.

#Build Your App using the Pipeline

To triggering the pipeline, you need a valid API token. If you don't have one yet, follow these steps:

1. Click on Settings on the left sidebar
2. Navigate to the API Tokens page
3. Click Create Token and give it an ID

VDP automatically generates a dedicated trigger endpoint for each pipeline to process data. To trigger the pipeline, follow these steps:

1. Click the bottom right icon to display the Pipeline toolkit
2. Copy the cURL request in the Trigger Snippet section and pass your API token as a Bearer token <api_token> in the Authorization header
3. Adjust and ingest your data accordingly in the example request

Here are the triggering pipeline example cURL command and example with instill-sdk Python Package:

Where <user-id> represents your username.

You should receive the following response:

#What's Next?

Our journey with Instill Model is just beginning! We have exciting plans on our roadmap, including the continuous deployment of cutting-edge models. Our next target is to provide features for fine-tuning these open-source models on specific tasks with custom data. Stay tuned for more updates!

If you're interested in a quick demo, please feel free to book a meeting with us. Or, please join our Discord to chat online. We'd love to hear from you! 👐