The BigQuery component is a data component that allows users to insert data to BigQuery tables. It can carry out the following tasks:
#Release Stage
Alpha
#Configuration
The component definition and tasks are defined in the definition.json and tasks.json files respectively.
#Setup
In order to communicate with Google, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's setup block, or you can create a Connection from
the Integration Settings
page and reference the whole setup as setup: ${connection.<my-connection-id>}.
| Field | Field ID | Type | Note |
|---|---|---|---|
| JSON Key File Contents (required) | json-key | string | Contents of the JSON key file with access to the bucket. |
| BigQuery Project ID (required) | project-id | string | Fill in your BigQuery Project ID. |
| BigQuery Dataset ID (required) | dataset-id | string | Fill in your BigQuery Dataset ID. |
| BigQuery Table Name (required) | table-name | string | Fill in your BigQuery Table Name. |
#Supported Tasks
#Insert
Insert data to BigQuery.
| Input | ID | Type | Description |
|---|---|---|---|
| Task ID (required) | task | string | TASK_INSERT |
| Data | data | object | The data to be inserted to BigQuery. |
| Output | ID | Type | Description |
|---|---|---|---|
| Status | status | string | Status of the upload operation. |
#Read
Read data from BigQuery.
| Input | ID | Type | Description |
|---|---|---|---|
| Task ID (required) | task | string | TASK_READ |
| Filtering | filtering | string | The filter to be applied to the data with SQL syntax, which starts with WHERE clause. |
| Output | ID | Type | Description |
|---|---|---|---|
| Data | data | array[object] | The data to be read from BigQuery. |