Google Cloud Storage (GCS)

The Google Cloud Storage (GCS) Data Connector allows users to upload data to Google's Cloud Storage.

#Configuration

Latest configuration is maintained here

There are two things needed to configure a GCS connector: Bucket name and JSON Key (for auth). To get these details, visit https://console.cloud.google.com/storage/browser and navigate to your bucket.

#No-code Setup

To create a GCS data connector effortlessly, follow these steps:

Go to the Data page and click Set up new data connector
Click the Data connector type ▾ drop-down and choose Google Cloud Storage
Fill in the required fields
Optionally, give a short description in the Description field

Now go to the Data page, the corresponding GCS data connector should be connected.

#Low-code Setup

For advanced users who prefer a more hands-on approach, you can create a GCS data connector using the following cURL commands:

Create a GCS data connector:

cURL

curl  --location http://localhost:8080/vdp/v1alpha/users/<user-id>/connector-resources
      --header 'Authorization: Bearer <Your-API-Token>' \
      --data '{
        "id": "gcs-connector",
        "connector_definition_name": "connector-definitions/data-gcs",
        "description": "GCS connector",
        "configuration": {
          "json_key": "<JSON key file contents here>"
          "bucket_name": "<bucket name>"
        }
      }'

For other operations, please refer to the VDP Protobufs.

#Supported Tasks

The GCS connector supports these tasks: Upload.

Upload - use this to upload the contents to the given bucket. The input should look something like -

{
  "object_name": "<name of the object/file>",
  "data": "<data to be saved>"
}

Ref: https://console.cloud.google.com/storage