This page curates a list of frequently asked questions from our users, friends, candidates, investors, random people, etc.
Why do you build VDP?
Modern data stack misses unstructured data processing.
It is non-trivial to process unstructured data though. We used to suffer enough in devising our own deep learning models, putting the models in production, running the day-to-day operation, and endlessly building the peripheral MLOps tools to keep the production AI performance consistent. All these happened in-house and were non-scalable.
There must be a better way, and Versatile Data Pipeline (VDP) is the answer.
To prevail AI and to make it accessible to everyone, the point is not merely the algorithms (i.e., the AI models) but the infrastructure tooling to connect the value of the algorithms end-to-end with the modern data stack.
You can find a more detailed narrative in our blog article Why Instill AI exists and Missing piece in modern data stack: unstructured data ETL.
Who are behind VDP?
We are a nimble team formed by members working for years in Computer Vision, Machine Learning, Deep Learning, large-scale database, and cloud-native applications/infrastructure. We have in-depth experience in developing and maintaining sophisticated AI systems.
Before we started to build VDP, we had fought with streaming large volume data (billions of images a day!) to automate Vision tasks using deep learning models, sweating blood to build everything from scratch.
We have learned that model serving for an effective end-to-end data flow concerns not only high throughput and low latency but also cost effectiveness. These criteria are non-trivial to achieve altogether. In the end, we had successfully built a battle-proven AI system in-house and had the system run in production for years.
What we had built can actually be modularised into working components to benefit a broader spectrum of AI tasks and industry sectors. We believe it's time to apply our experiences to make AI more accessible to everyone, especially the data industry.
Is VDP open source?
VDP uses multiple licenses, including Elastic License 2.0 (ELv2) and open-source MIT License. Our mission is to make AI accessible to everyone. The best way to achieve this is to make VDP free to use and source available to everyone, while ensuring we safely create a sustainable business. Please check the VDP License in detail.
Is VDP free?
Yes. VDP is source available so you can self-host it in your basement for free.
How do you make money?
We offer fully managed VDP service on Instill Cloud to users who want to get all the power of VDP without any hassle. It is currently in Open Alpha testing phase - with all features FREE. For more information, see the Pricing page. Rest assured that we will never charge you without your consent.
We are adding new features every day and we need your feedback to help shape the future of the service and build Instill Cloud the best it can be.
What programming language does VDP use or support?
VDP's backend components are Go-based and frontend console is written in Next.js, TypeScript, and TailwindCSS.
Nonetheless, VDP is API-first and cloud-native. You can use cURL or the protobuf auto-generated codes to work with VDP. More developer-friendly multi-language SDKs are on our future roadmap.
What design principles does VDP adopt?
Microservice architecture makes VDP to be extensible, flexible and reusable for scenarios where new components are constantly added in VDP. The overall microservice system can be also very efficient at scaling phase to scale each backend instances based on their individual workload. For example. the
connector-backendmight be way less busy compared with the
IDEALS (Interface segregation, Deployability, Event-driven, Availability over consistency, Loose coupling, and Single responsibility) design principle sets a rigorous framework when there comes to any VDP design questions in our day-to-day development.
API-first approach comes naturally with the adoption of microservice and IDEALS. All backend components in VDP are implemented with API-first design principle so the contract is firmly established for whatever necessary integration task coming in the future.
The twelve-factor methodology provides a down-to-earth guideline for the development and deployment of VDP components.
Building an effective and efficient machine learning product requires implementing the cutting-edge best practices of MLOps. There are many essential components involved in the MLOps cycle, and notably, these components are still evolving.
We believe tooling development in AI and Data industry is marching toward a completely modularisation future, meaning that components in MLOps need to take into account flexibility, extensibility and composability as the first-class citizen as far as the design principle goes.
Why is the company named Instill AI?
We have been attempting to land AI products ever since the surge of deep learning in 2014. On the way, we have learned how challenging this can be. To build effective AI products, educate the market about the AI, and eventually see the AI products helping people in real life. It is fair to say that we, the whole industry, still have a long way to go.
We are enthusiastic about being on the journey, to instill the AI technologies gradually to the industry.
The name is also inspired by a Chinese poem "Delighting in Rain on a Spring Night (春夜喜雨)" by Du Fu (杜甫), particularly for reifying the word "instill":
好雨知時節，當春乃發生。 隨風潛入夜，潤物細無聲。 野徑雲俱黑，江船火獨明。 曉看紅濕處，花重錦官城。
It's pretty hard to have this poem precisely translated in English, but anyway the main idea is to express how delightful when hard working finally pays off.
What is in the mosaic-style logo?
Our visual designer Wen Chen has done a great job to embed Instill AI and the meaning of our effort into the logo.
Which one is Instill AI?
Among all the other Instill AIs you could ever find on the Internet, Instill AI with the domain
instill.tech is the earliest incorporated company in the UK (on 11 June 2020).
If you Google "Instill AI", please always refer to the right one: https://www.instill.tech to find us. 😉