BigQuery

The BigQuery Data Connector allows users to insert data to BigQuery tables.

#Configuration

Latest configuration is maintained here

There are four things needed to configure a BigQuery connector: Project ID, Dataset ID, Table Name and JSON Key (for auth). To get these details, visit https://console.cloud.google.com/bigquery and navigate to your project and then dataset.

#No-code Setup

To create a BigQuery data connector effortlessly, follow these steps:

  1. Go to the Data page and click Set up new data connector
  2. Click the Data connector type ▾ drop-down and choose BigQuery
  3. Fill in the required fields
  4. Optionally, give a short description in the Description field

Now go to the Data page, the corresponding BigQuery data connector should be connected.

#Low-code Setup

For advanced users who prefer a more hands-on approach, you can create a BigQuery data connector using the following cURL commands:

  1. Create a BigQuery data connector:
cURL
Copy

curl --location http://localhost:8080/vdp/v1alpha/users/<user-id>/connector-resources
--header 'Authorization: Bearer <Your-API-Token>' \
--data '{
"id": "bigquery",
"connector_definition_name": "connector-definitions/data-bigquery",
"description": "BigQuery connector",
"configuration": {
"json_key": "<JSON key file contents here>"
"project_id": "<project-id>",
"dataset_id": "<dataset-id>",
"table_name": "<table-name>"
}
}'

For other operations, please refer to the VDP Protobufs.

#Supported Tasks

The BigQuery connector supports these tasks: Insert

  1. Insert - use this to insert a new record to BigQuery. It accepts an object of key-value pair which is in accordance with your table schema. For example, if your table schema contains the following fields - id INT, name STRING. The object should look something like:

{
"id": 1,
"name": "Object Name"
}

Ref: https://cloud.google.com/bigquery/docs/introduction

Last updated: 10/31/2023, 10:43:34 AM