Visual Data Preparation (VDP) is an open-source visual data ETL tool to streamline the end-to-end visual data processing:
- Extract unstructured visual data from pre-built data sources such as cloud/on-prem storage, or IoT devices
- Transform it into analysable structured data by Vision AI models
- Load the transformed data into warehouses, applications, or other destinations
- 🚀 The fastest way to build end-to-end visual data pipelines
- ⚡️ High-performing backends implemented in Go
- 🖱️ One-click import & deploy models
- 📦 Standardise structured outputs for computer vision tasks
- 🔌 Pre-built ETL data connectors for extensive data access
- 🪢 Perform in
SYNCfor real-time inference and
ASYNCfor on-demand workload
- 🧁 Scalable API-first microservice design for great developer experience
- 🤠 Build for every Vision AI and Data practitioner with no-/low-code interfaces
We've made it easy to get started with VDP either on a local machine or on our demo playground.
Make sure you have the prerequisites set up:
- macOS or Linux - VDP works on macOS or Linux, but does not support Windows at the moment.
- Docker and Docker Compose - VDP uses Docker Compose to run all services at local.
Get started locally by running:
git clone https://github.com/instill-ai/vdp.git && cd vdpmake all
Once the services are up, the Console UI is ready to go at http://localhost:3000.
⚠︎ Downloading the Triton server image will take a while, but it should be just a one-time effort.
To shut down all running services:
Just visit the demo website to try out VDP.
#Start building with VDP
If this is your first time setting up VDP, access the Console (http://localhost:3000) and you should see the onboarding page. Please enter your email and you are all set!
Jump right in
📔 Dive into the core concepts
👐 We love contribution to VDP in any forms:
- Please refer to the guideline for local development
- Please open a topic in the repository Discussions for any feature requests
- Please open issues for bug report in the repository
- Please refer to the VDP project board to track progress.
Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use latest release.
VDP is still in Alpha. If you encounter any issues, we'd love to help. Join us on Discord to share with and get help from an active and friendly community!
👋 Pick a role in #welcome and say hi in #introduce-yourself channel
👋 Ask VDP questions in our #vdp channel
👋 Share what you've been working on with VDP in #showcase channel