Set up local development environment

VDP is built with the microservice architecture. To develop each service independently, we assign profiles to each service in the docker-compose-dev.yml file in VDP. This allows us to selectively enabling services for various usages, e.g., debugging or development.

INFO

Services are associated with profiles through the profiles attribute, which takes an array of profile names. A service will be started only if one of its profile names is activated. A service without profiles will always be started.

VDP assigns five different profile names:

  • console - start all the dependent services for console
  • mgmt - start all the dependent services for mgmt-backend
  • connector - start all the dependent services for connector-backend
  • model - start all the dependent services for model-backend
  • pipeline - start all the dependent services for pipeline-backend
  • all - start all services

Use one of the profile name to develop the corresponding service:


# In VDP project
make build PROFILE=<valid-profile-name>
make dev PROFILE=<valid-profile-name>

The following guideline shows detailed guideline about how to develop the connector-backend.

#Start dependent VDP services for connector-backend

On the local machine, clone vdp repository in your workspace, move to the repository folder, and launch all dependent microservices:


# Clone VDP project
cd <your-workspace>
git clone https://github.com/instill-ai/vdp.git && cd vdp
# Use profile `connector` to launch all dependent services for `connector-backend`
make build PROFILE=connector
make dev PROFILE=connector

#Run dev connector-backend

Clone connector-backend repository in your workspace and move to the repository folder:


cd <your-workspace>
git clone https://github.com/instill-ai/connector-backend.git
cd connector-backend

Build & run the dev image


make build
make dev

Now, you have the Go project set up in the container, in which you can compile and run the binaries together with the integration test in each container shell.

Start the connector-backend server


docker exec -it connector-backend /bin/bash
go run ./cmd/migration
go run ./cmd/init
go run ./cmd/main

Start the Temporal worker


docker exec -it connector-backend /bin/bash
go run ./cmd/worker

Run the integration test During development, you can run the integration test to make sure the updated connector-backend works as intended:


docker exec -it connector-backend /bin/bash
make integration-test

#Stop the dev container


make stop

Last updated: 8/19/2022, 1:07:06 AM