The Google Cloud Storage component is a data component that allows users to upload data to Google's Cloud Storage.
It can carry out the following tasks:
#Release Stage
Alpha
#Configuration
The component definition and tasks are defined in the definition.json and tasks.json files respectively.
#Setup
In order to communicate with Google, the following connection details need to be
provided. You may specify them directly in a pipeline recipe as key-value pairs
within the component's setup
block, or you can create a Connection from
the Integration Settings
page and reference the whole setup
as setup: ${connection.<my-connection-id>}
.
Field | Field ID | Type | Note |
---|
JSON Key File Contents (required) | json-key | string | Contents of the JSON key file with access to the bucket. |
#Supported Tasks
#Upload
Upload data to Google Cloud Storage.
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_UPLOAD |
Bucket Name (required) | bucket-name | string | Name of the bucket to be used for object storage |
Object Name (required) | object-name | string | The name of the object to be created |
Data (required) | data | string | The data to be saved in the object |
Output | ID | Type | Description |
---|
Authenticated URL (optional) | authenticated-url | string | Only users granted permission can access the object with this link |
Gsutil URI (optional) | gsutil-uri | string | File path to this resource in Cloud Storage |
Public Access (optional) | public-access | boolean | Whether the object is publicly accessible |
Public URL (optional) | public-url | string | Anyone with this link can access the object on the public Internet |
Upload Status | status | string | Status of the upload operation |
#Read Objects
Read objects from Google Cloud Storage.
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_READ_OBJECTS |
Bucket Name (required) | bucket-name | string | Name of the bucket to be used for object storage |
Delimiter | delimiter | string | The delimiter to use when listing objects |
Prefix | prefix | string | The prefix to use when listing objects |
Versions | versions | boolean | Whether to list all versions of an object |
Start Offset | start-offset | string | The offset to start listing objects from |
End Offset | end-offset | string | The offset to stop listing objects at |
Include Trailing Delimiter | include-trailing-delimiter | boolean | Whether to include the delimiter in the object name |
Match Glob | match-glob | string | Whether to match objects using a glob pattern |
Include Folders As Prefixes | include-folders-as-prefixes | boolean | Whether to include folders as prefixes |
Output | ID | Type | Description |
---|
Text Objects (optional) | text-objects | array[object] | The text objects in the bucket |
Image Objects (optional) | image-objects | array[object] | The image objects in the bucket |
Document Objects (optional) | document-objects | array[object] | The document objects in the bucket |
Audio Objects (optional) | audio-objects | array[object] | The audio objects in the bucket |
Video Objects (optional) | video-objects | array[object] | The video objects in the bucket |
Output Objects in Read Objects
Text Objects
Field | Field ID | Type | Note |
---|
Attributes | attributes | object | The attributes of the object |
Data | data | string | The data of the object |
Attributes
Field | Field ID | Type | Note |
---|
Content Disposition | content-disposition | string | The content disposition of the object |
Content Encoding | content-encoding | string | The content encoding of the object |
Content Language | content-language | string | The content language of the object |
Content Type | content-type | string | The content type of the object |
Media Link | media-link | string | The media link of the object |
Metadata | metadata | object | The object metadata |
Object Name | name | string | The name of the object |
Owner | owner | string | The owner of the object |
Size | size | integer | The size of the object |
Storage Class | storage-class | string | The storage class of the object |
Image Objects
Field | Field ID | Type | Note |
---|
Attributes | attributes | object | The attributes of the object |
Data | data | string | The data of the object |
Attributes
Field | Field ID | Type | Note |
---|
Content Disposition | content-disposition | string | The content disposition of the object |
Content Encoding | content-encoding | string | The content encoding of the object |
Content Language | content-language | string | The content language of the object |
Content Type | content-type | string | The content type of the object |
Media Link | media-link | string | The media link of the object |
Metadata | metadata | object | The object metadata |
Object Name | name | string | The name of the object |
Owner | owner | string | The owner of the object |
Size | size | integer | The size of the object |
Storage Class | storage-class | string | The storage class of the object |
Document Objects
Field | Field ID | Type | Note |
---|
Attributes | attributes | object | The attributes of the object |
Data | data | string | The data of the object |
Attributes
Field | Field ID | Type | Note |
---|
Content Disposition | content-disposition | string | The content disposition of the object |
Content Encoding | content-encoding | string | The content encoding of the object |
Content Language | content-language | string | The content language of the object |
Content Type | content-type | string | The content type of the object |
Media Link | media-link | string | The media link of the object |
Metadata | metadata | object | The object metadata |
Object Name | name | string | The name of the object |
Owner | owner | string | The owner of the object |
Size | size | integer | The size of the object |
Storage Class | storage-class | string | The storage class of the object |
Audio Objects
Field | Field ID | Type | Note |
---|
Attributes | attributes | object | The attributes of the object |
Data | data | string | The data of the object |
Attributes
Field | Field ID | Type | Note |
---|
Content Disposition | content-disposition | string | The content disposition of the object |
Content Encoding | content-encoding | string | The content encoding of the object |
Content Language | content-language | string | The content language of the object |
Content Type | content-type | string | The content type of the object |
Media Link | media-link | string | The media link of the object |
Metadata | metadata | object | The object metadata |
Object Name | name | string | The name of the object |
Owner | owner | string | The owner of the object |
Size | size | integer | The size of the object |
Storage Class | storage-class | string | The storage class of the object |
Video Objects
Field | Field ID | Type | Note |
---|
Attributes | attributes | object | The attributes of the object |
Data | data | string | The data of the object |
Attributes
Field | Field ID | Type | Note |
---|
Content Disposition | content-disposition | string | The content disposition of the object |
Content Encoding | content-encoding | string | The content encoding of the object |
Content Language | content-language | string | The content language of the object |
Content Type | content-type | string | The content type of the object |
Media Link | media-link | string | The media link of the object |
Metadata | metadata | object | The object metadata |
Object Name | name | string | The name of the object |
Owner | owner | string | The owner of the object |
Size | size | integer | The size of the object |
Storage Class | storage-class | string | The storage class of the object |
#Create Bucket
Read objects from Google Cloud Storage.
Input | ID | Type | Description |
---|
Task ID (required) | task | string | TASK_CREATE_BUCKET |
Bucket Name (required) | bucket-name | string | Name of the bucket to be used for object storage |
Project ID (required) | project-id | string | The project ID to create the bucket in |
Location | location | string | The location to create the bucket in |
Output | ID | Type | Description |
---|
Result (optional) | result | string | The result of the operation |
Bucket Name (optional) | name | string | The name of the bucket |
Location (optional) | location | string | The location of the bucket |