BigQuery | Documentation

The BigQuery Data Connector allows users to insert data to BigQuery tables.

#Configuration

Latest configuration is maintained here

There are four things needed to configure a BigQuery connector: Project ID, Dataset ID, Table Name and JSON Key (for auth). To get these details, visit https://console.cloud.google.com/bigquery and navigate to your project and then dataset.

#No-code Setup

To create a BigQuery data connector effortlessly, follow these steps:

Go to the Data page and click Set up new data connector
Click the Data connector type ▾ drop-down and choose BigQuery
Fill in the required fields
Optionally, give a short description in the Description field

Now go to the Data page, the corresponding BigQuery data connector should be connected.

#Low-code Setup

For advanced users who prefer a more hands-on approach, you can create a BigQuery data connector using the following cURL commands:

Create a BigQuery data connector:

cURL

curl  --location http://localhost:8080/vdp/v1alpha/users/<user-id>/connectors
      --header 'Authorization: Bearer <Your-API-Token>' \
      --data '{
        "id": "bigquery",
        "connector_definition_name": "connector-definitions/data-bigquery",
        "description": "BigQuery connector",
        "configuration": {
          "json_key": "<JSON key file contents here>"
          "project_id": "<project-id>",
          "dataset_id": "<dataset-id>",
          "table_name": "<table-name>"
        }
      }'

For other operations, please refer to the VDP Protobufs.

#Supported Tasks

The BigQuery connector supports these tasks: Insert

Insert - use this to insert a new record to BigQuery. It accepts an object of key-value pair which is in accordance with your table schema. For example, if your table schema contains the following fields - id INT, name STRING. The object should look something like:

{
  "id": 1,
  "name": "Object Name"
}

Ref: https://cloud.google.com/bigquery/docs/introduction