Google Cloud Storage

The Google Cloud Storage component is a data component that allows users to upload data to Google's Cloud Storage. It can carry out the following tasks:

#Release Stage

Alpha

#Configuration

The component definition and tasks are defined in the definition.json and tasks.json files respectively.

#Setup

In order to communicate with Google, the following connection details need to be provided. You may specify them directly in a pipeline recipe as key-value pairs within the component's setup block, or you can create a Connection from the Integration Settings page and reference the whole setup as setup: ${connection.<my-connection-id>}.

FieldField IDTypeNote
JSON Key File Contents (required)json-keystringContents of the JSON key file with access to the bucket.

#Supported Tasks

#Upload

Upload data to Google Cloud Storage.

InputIDTypeDescription
Task ID (required)taskstringTASK_UPLOAD
Bucket Name (required)bucket-namestringName of the bucket to be used for object storage
Object Name (required)object-namestringThe name of the object to be created
Data (required)datastringThe data to be saved in the object
OutputIDTypeDescription
Authenticated URL (optional)authenticated-urlstringOnly users granted permission can access the object with this link
Gsutil URI (optional)gsutil-uristringFile path to this resource in Cloud Storage
Public Access (optional)public-accessbooleanWhether the object is publicly accessible
Public URL (optional)public-urlstringAnyone with this link can access the object on the public Internet
Upload StatusstatusstringStatus of the upload operation

#Read Objects

Read objects from Google Cloud Storage.

InputIDTypeDescription
Task ID (required)taskstringTASK_READ_OBJECTS
Bucket Name (required)bucket-namestringName of the bucket to be used for object storage
DelimiterdelimiterstringThe delimiter to use when listing objects
PrefixprefixstringThe prefix to use when listing objects
VersionsversionsbooleanWhether to list all versions of an object
Start Offsetstart-offsetstringThe offset to start listing objects from
End Offsetend-offsetstringThe offset to stop listing objects at
Include Trailing Delimiterinclude-trailing-delimiterbooleanWhether to include the delimiter in the object name
Match Globmatch-globstringWhether to match objects using a glob pattern
Include Folders As Prefixesinclude-folders-as-prefixesbooleanWhether to include folders as prefixes
OutputIDTypeDescription
Text Objects (optional)text-objectsarray[object]The text objects in the bucket
Image Objects (optional)image-objectsarray[object]The image objects in the bucket
Document Objects (optional)document-objectsarray[object]The document objects in the bucket
Audio Objects (optional)audio-objectsarray[object]The audio objects in the bucket
Video Objects (optional)video-objectsarray[object]The video objects in the bucket
Output Objects in Read Objects

Text Objects

FieldField IDTypeNote
AttributesattributesobjectThe attributes of the object
DatadatastringThe data of the object

Attributes

FieldField IDTypeNote
Content Dispositioncontent-dispositionstringThe content disposition of the object
Content Encodingcontent-encodingstringThe content encoding of the object
Content Languagecontent-languagestringThe content language of the object
Content Typecontent-typestringThe content type of the object
Media Linkmedia-linkstringThe media link of the object
MetadatametadataobjectThe object metadata
Object NamenamestringThe name of the object
OwnerownerstringThe owner of the object
SizesizeintegerThe size of the object
Storage Classstorage-classstringThe storage class of the object

Image Objects

FieldField IDTypeNote
AttributesattributesobjectThe attributes of the object
DatadatastringThe data of the object

Attributes

FieldField IDTypeNote
Content Dispositioncontent-dispositionstringThe content disposition of the object
Content Encodingcontent-encodingstringThe content encoding of the object
Content Languagecontent-languagestringThe content language of the object
Content Typecontent-typestringThe content type of the object
Media Linkmedia-linkstringThe media link of the object
MetadatametadataobjectThe object metadata
Object NamenamestringThe name of the object
OwnerownerstringThe owner of the object
SizesizeintegerThe size of the object
Storage Classstorage-classstringThe storage class of the object

Document Objects

FieldField IDTypeNote
AttributesattributesobjectThe attributes of the object
DatadatastringThe data of the object

Attributes

FieldField IDTypeNote
Content Dispositioncontent-dispositionstringThe content disposition of the object
Content Encodingcontent-encodingstringThe content encoding of the object
Content Languagecontent-languagestringThe content language of the object
Content Typecontent-typestringThe content type of the object
Media Linkmedia-linkstringThe media link of the object
MetadatametadataobjectThe object metadata
Object NamenamestringThe name of the object
OwnerownerstringThe owner of the object
SizesizeintegerThe size of the object
Storage Classstorage-classstringThe storage class of the object

Audio Objects

FieldField IDTypeNote
AttributesattributesobjectThe attributes of the object
DatadatastringThe data of the object

Attributes

FieldField IDTypeNote
Content Dispositioncontent-dispositionstringThe content disposition of the object
Content Encodingcontent-encodingstringThe content encoding of the object
Content Languagecontent-languagestringThe content language of the object
Content Typecontent-typestringThe content type of the object
Media Linkmedia-linkstringThe media link of the object
MetadatametadataobjectThe object metadata
Object NamenamestringThe name of the object
OwnerownerstringThe owner of the object
SizesizeintegerThe size of the object
Storage Classstorage-classstringThe storage class of the object

Video Objects

FieldField IDTypeNote
AttributesattributesobjectThe attributes of the object
DatadatastringThe data of the object

Attributes

FieldField IDTypeNote
Content Dispositioncontent-dispositionstringThe content disposition of the object
Content Encodingcontent-encodingstringThe content encoding of the object
Content Languagecontent-languagestringThe content language of the object
Content Typecontent-typestringThe content type of the object
Media Linkmedia-linkstringThe media link of the object
MetadatametadataobjectThe object metadata
Object NamenamestringThe name of the object
OwnerownerstringThe owner of the object
SizesizeintegerThe size of the object
Storage Classstorage-classstringThe storage class of the object

#Create Bucket

Read objects from Google Cloud Storage.

InputIDTypeDescription
Task ID (required)taskstringTASK_CREATE_BUCKET
Bucket Name (required)bucket-namestringName of the bucket to be used for object storage
Project ID (required)project-idstringThe project ID to create the bucket in
LocationlocationstringThe location to create the bucket in
OutputIDTypeDescription
Result (optional)resultstringThe result of the operation
Bucket Name (optional)namestringThe name of the bucket
Location (optional)locationstringThe location of the bucket