Getting started

Visual Data Preparation (VDP) is an open-source visual data ETL tool to streamline the end-to-end visual data processing:

  • Extract unstructured visual data from pre-built data sources such as cloud/on-prem storage, or IoT devices
  • Transform it into analysable structured data by Vision AI models
  • Load the transformed data into warehouses, applications, or other destinations

#Highlights

  • 🚀 The fastest way to build end-to-end visual data pipelines
  • ⚡️ High-performing backends implemented in Go
  • 🖱️ One-click import & deploy models
  • 📦 Standardise structured outputs for computer vision tasks
  • 🔌 Pre-built ETL data connectors for extensive data access
  • 🪢 Perform in SYNC for real-time inference and ASYNC for on-demand workload
  • 🧁 Scalable API-first microservice design for great developer experience
  • 🤠 Build for every Vision AI and Data practitioner with no-/low-code interfaces

#Quick start

We've made it easy to get started with VDP either on a local machine or on our demo playground.

#Local run

Make sure you have the prerequisites set up:

  • macOS or Linux - VDP works on macOS or Linux, but does not support Windows at the moment.
  • Docker and Docker Compose - VDP uses Docker Compose to run all services at local.

Get started locally by running:


git clone https://github.com/instill-ai/vdp.git && cd vdp
make all

Once the services are up, the Console UI is ready to go at http://localhost:3000.

⚠︎ Downloading the Triton server image will take a while, but it should be just a one-time effort.

To shut down all running services:


make down

#Demo playground

Just visit the demo website to try out VDP.

#Start building with VDP

If this is your first time setting up VDP, access the Console (http://localhost:3000) and you should see the onboarding page. Please enter your email and you are all set!

Onboarding page of the VDP Console

Jump right in

⚡️ Build your first SYNC pipeline with no-/low-code

⚡️ Build your first ASYNC pipeline with no-/low-code

#Learn VDP

📔 Dive into the core concepts

📔 Check out the FAQ and tutorials

📔 Explore the Source Connectors and Destinations Connectors to find the connectors you want

📔 Explore the APIs of Protobufs or OpenAPI (after make all or make doc)

#Contribution

👐 We love contribution to VDP in any forms:

WARNING

Code in the main branch tracks under-development progress towards the next release and may not work as expected. If you are looking for a stable alpha version, please use latest release.

#Community support

VDP is still in Alpha. If you encounter any issues, we'd love to help. Join us on Discord to share with and get help from an active and friendly community!

👋 Pick a role in #welcome and say hi in #introduce-yourself channel

👋 Ask VDP questions in our #vdp channel

👋 Share what you've been working on with VDP in #showcase channel

#Learn more

🌟 GitHub

📚 Blog

📖 Newsletter

🚀 Product Website

Last updated: 12/2/2022, 11:24:55 AM