Skip to content

chalmagr/java-storage

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Google Cloud Storage Client for Java

Java idiomatic client for Cloud Storage.

Maven Stability

Quickstart

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>com.google.cloud</groupId>
      <artifactId>libraries-bom</artifactId>
      <version>26.44.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage</artifactId>
  </dependency>
  <dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage-control</artifactId>
  </dependency>
</dependencies>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-storage</artifactId>
  <version>2.42.0</version>
</dependency>
<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-storage-control</artifactId>
  <version>2.42.0</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:2.42.0')

implementation 'com.google.cloud:google-cloud-storage'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-storage:2.42.0'

If you are using SBT, add this to your dependencies:

libraryDependencies += "com.google.cloud" % "google-cloud-storage" % "2.42.0"

Authentication

See the Authentication section in the base directory's README.

Authorization

The client application making API calls must be granted authorization scopes required for the desired Cloud Storage APIs, and the authenticated principal must have the IAM role(s) required to access GCP resources using the Cloud Storage API calls.

Getting Started

Prerequisites

You will need a Google Cloud Platform Console project with the Cloud Storage API enabled. You will need to enable billing to use Google Cloud Storage. Follow these instructions to get your project set up. You will also need to set up the local development environment by installing the Google Cloud Command Line Interface and running the following commands in command line: gcloud auth login and gcloud config set project [YOUR PROJECT ID].

Installation and setup

You'll need to obtain the google-cloud-storage library. See the Quickstart section to add google-cloud-storage as a dependency in your code.

About Cloud Storage

Cloud Storage is a durable and highly available object storage service. Google Cloud Storage is almost infinitely scalable and guarantees consistency: when a write succeeds, the latest copy of the object will be returned to any GET, globally.

See the Cloud Storage client library docs to learn how to use this Cloud Storage Client Library.

About Storage Control

The Storage Control API lets you perform metadata-specific, control plane, and long-running operations.

The Storage Control API creates one space to perform metadata-specific, control plane, and long-running operations apart from the Storage API. Separating these operations from the Storage API improves API standardization and lets you run faster releases.

If you are using Maven with BOM, add this to your pom.xml file:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.google.cloud</groupId>
            <artifactId>libraries-bom</artifactId>
            <version>26.44.0</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

<dependencies>
    <dependency>
        <groupId>com.google.cloud</groupId>
        <artifactId>google-cloud-storage-control</artifactId>
    </dependency>
</dependencies>

If you are using Maven without the BOM, add this to your dependencies:

<dependency>
    <groupId>com.google.cloud</groupId>
    <artifactId>google-cloud-storage-control</artifactId>
    <version>2.41.0</version>
</dependency>

If you are using Gradle 5.x or later, add this to your dependencies:

implementation platform('com.google.cloud:libraries-bom:26.44.0')
implementation 'com.google.cloud:google-cloud-storage-control'

If you are using Gradle without BOM, add this to your dependencies:

implementation 'com.google.cloud:google-cloud-storage-control:2.41.0'

Creating an authorized service object

To make authenticated requests to Google Cloud Storage, you must create a service object with credentials. You can then make API calls by calling methods on the Storage service object. The simplest way to authenticate is to use Application Default Credentials. These credentials are automatically inferred from your environment, so you only need the following code to create your service object:

import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;

Storage storage = StorageOptions.getDefaultInstance().getService();

For other authentication options, see the Authentication page in Google Cloud Java.

Storing data

Stored objects are called "blobs" in google-cloud and are organized into containers called "buckets". Blob, a subclass of BlobInfo, adds a layer of service-related functionality over BlobInfo. Similarly, Bucket adds a layer of service-related functionality over BucketInfo. In this code snippet, we will create a new bucket and upload a blob to that bucket.

Add the following imports at the top of your file:

import static java.nio.charset.StandardCharsets.UTF_8;

import com.google.cloud.storage.Blob;
import com.google.cloud.storage.Bucket;
import com.google.cloud.storage.BucketInfo;

Then add the following code to create a bucket and upload a simple blob.

Important: Bucket names have to be globally unique (among all users of Cloud Storage). If you choose a bucket name that already exists, you'll get a helpful error message telling you to choose another name. In the code below, replace "my_unique_bucket" with a unique bucket name. See more about naming rules here.

// Create a bucket
String bucketName = "my_unique_bucket"; // Change this to something unique
Bucket bucket = storage.create(BucketInfo.of(bucketName));

// Upload a blob to the newly created bucket
BlobId blobId = BlobId.of(bucketName, "my_blob_name");
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).setContentType("text/plain").build();
Blob blob = storage.create(blobInfo, "a simple blob".getBytes(UTF_8));

A complete example for creating a blob can be found at UploadObject.java.

At this point, you will be able to see your newly created bucket and blob on the Google Developers Console.

Retrieving data

Now that we have content uploaded to the server, we can see how to read data from the server. Add the following line to your program to get back the blob we uploaded.

BlobId blobId = BlobId.of(bucketName, "my_blob_name");
byte[] content = storage.readAllBytes(blobId);
String contentString = new String(content, UTF_8);

A complete example for accessing blobs can be found at DownloadObject.java.

Updating data

Another thing we may want to do is update a blob. The following snippet shows how to update a Storage blob if it exists.

BlobId blobId = BlobId.of(bucketName, "my_blob_name");
Blob blob = storage.get(blobId);
if (blob != null) {
  byte[] prevContent = blob.getContent();
  System.out.println(new String(prevContent, UTF_8));
  WritableByteChannel channel = blob.writer();
  channel.write(ByteBuffer.wrap("Updated content".getBytes(UTF_8)));
  channel.close();
}

Listing buckets and contents of buckets

Suppose that you've added more buckets and blobs, and now you want to see the names of your buckets and the contents of each one. Add the following code to list all your buckets and all the blobs inside each bucket.

// List all your buckets
System.out.println("My buckets:");
for (Bucket bucket : storage.list().iterateAll()) {
  System.out.println(bucket);

  // List all blobs in the bucket
  System.out.println("Blobs in the bucket:");
  for (Blob blob : bucket.list().iterateAll()) {
    System.out.println(blob);
  }
}

Complete source code

See ListObjects.java for a complete example.

Example Applications

  • Bookshelf - An App Engine application that manages a virtual bookshelf.
    • This app uses google-cloud to interface with Cloud Datastore and Cloud Storage. It also uses Cloud SQL, another Google Cloud Platform service.
  • Flexible Environment/Storage example - An app that uploads files to a public Cloud Storage bucket on the App Engine Flexible Environment runtime.

Samples

Samples are in the samples/ directory.

Sample Source Code Try it
Configure Retries source code Open in Cloud Shell
Generate Signed Post Policy V4 source code Open in Cloud Shell
Get Service Account source code Open in Cloud Shell
Quickstart Grpc Dp Sample source code Open in Cloud Shell
Quickstart Grpc Sample source code Open in Cloud Shell
Quickstart Sample source code Open in Cloud Shell
Quickstart Storage Control Sample source code Open in Cloud Shell
Add Bucket Default Owner source code Open in Cloud Shell
Add Bucket Iam Conditional Binding source code Open in Cloud Shell
Add Bucket Iam Member source code Open in Cloud Shell
Add Bucket Label source code Open in Cloud Shell
Add Bucket Owner source code Open in Cloud Shell
Change Default Storage Class source code Open in Cloud Shell
Configure Bucket Cors source code Open in Cloud Shell
Create Bucket source code Open in Cloud Shell
Create Bucket Dual Region source code Open in Cloud Shell
Create Bucket Pub Sub Notification source code Open in Cloud Shell
Create Bucket With Object Retention source code Open in Cloud Shell
Create Bucket With Storage Class And Location source code Open in Cloud Shell
Create Bucket With Turbo Replication source code Open in Cloud Shell
Delete Bucket source code Open in Cloud Shell
Delete Bucket Pub Sub Notification source code Open in Cloud Shell
Disable Bucket Versioning source code Open in Cloud Shell
Disable Default Event Based Hold source code Open in Cloud Shell
Disable Lifecycle Management source code Open in Cloud Shell
Disable Requester Pays source code Open in Cloud Shell
Disable Uniform Bucket Level Access source code Open in Cloud Shell
Enable Bucket Versioning source code Open in Cloud Shell
Enable Default Event Based Hold source code Open in Cloud Shell
Enable Lifecycle Management source code Open in Cloud Shell
Enable Requester Pays source code Open in Cloud Shell
Enable Uniform Bucket Level Access source code Open in Cloud Shell
Get Bucket Autoclass source code Open in Cloud Shell
Get Bucket Metadata source code Open in Cloud Shell
Get Bucket Rpo source code Open in Cloud Shell
Get Default Event Based Hold source code Open in Cloud Shell
Get Public Access Prevention source code Open in Cloud Shell
Get Requester Pays Status source code Open in Cloud Shell
Get Retention Policy source code Open in Cloud Shell
Get Uniform Bucket Level Access source code Open in Cloud Shell
List Bucket Iam Members source code Open in Cloud Shell
List Buckets source code Open in Cloud Shell
List Pub Sub Notifications source code Open in Cloud Shell
Lock Retention Policy source code Open in Cloud Shell
Make Bucket Public source code Open in Cloud Shell
Print Bucket Acl source code Open in Cloud Shell
Print Bucket Acl Filter By User source code Open in Cloud Shell
Print Pub Sub Notification source code Open in Cloud Shell
Remove Bucket Cors source code Open in Cloud Shell
Remove Bucket Default Kms Key source code Open in Cloud Shell
Remove Bucket Default Owner source code Open in Cloud Shell
Remove Bucket Iam Conditional Binding source code Open in Cloud Shell
Remove Bucket Iam Member source code Open in Cloud Shell
Remove Bucket Label source code Open in Cloud Shell
Remove Bucket Owner source code Open in Cloud Shell
Remove Retention Policy source code Open in Cloud Shell
Set Async Turbo Rpo source code Open in Cloud Shell
Set Bucket Autoclass source code Open in Cloud Shell
Set Bucket Default Kms Key source code Open in Cloud Shell
Set Bucket Website Info source code Open in Cloud Shell
Set Client Endpoint source code Open in Cloud Shell
Set Default Rpo source code Open in Cloud Shell
Set Public Access Prevention Enforced source code Open in Cloud Shell
Set Public Access Prevention Inherited source code Open in Cloud Shell
Set Retention Policy source code Open in Cloud Shell
Create Folder source code Open in Cloud Shell
Create Hierarchical Namespace Bucket source code Open in Cloud Shell
Delete Folder source code Open in Cloud Shell
Get Folder source code Open in Cloud Shell
List Folders source code Open in Cloud Shell
Rename Folder source code Open in Cloud Shell
Activate Hmac Key source code Open in Cloud Shell
Create Hmac Key source code Open in Cloud Shell
Deactivate Hmac Key source code Open in Cloud Shell
Delete Hmac Key source code Open in Cloud Shell
Get Hmac Key source code Open in Cloud Shell
List Hmac Keys source code Open in Cloud Shell
Create Managed Folder source code Open in Cloud Shell
Delete Managed Folder source code Open in Cloud Shell
Get Managed Folder source code Open in Cloud Shell
List Managed Folders source code Open in Cloud Shell
Add File Owner source code Open in Cloud Shell
Batch Set Object Metadata source code Open in Cloud Shell
Change Object Csek To Kms source code Open in Cloud Shell
Change Object Storage Class source code Open in Cloud Shell
Compose Object source code Open in Cloud Shell
Copy Object source code Open in Cloud Shell
Copy Old Version Of Object source code Open in Cloud Shell
Delete Object source code Open in Cloud Shell
Delete Old Version Of Object source code Open in Cloud Shell
Download Byte Range source code Open in Cloud Shell
Download Encrypted Object source code Open in Cloud Shell
Download Object source code Open in Cloud Shell
Download Object Into Memory source code Open in Cloud Shell
Download Public Object source code Open in Cloud Shell
Download Requester Pays Object source code Open in Cloud Shell
Generate Encryption Key source code Open in Cloud Shell
Generate V4 Get Object Signed Url source code Open in Cloud Shell
Generate V4 Put Object Signed Url source code Open in Cloud Shell
Get Object Metadata source code Open in Cloud Shell
List Objects source code Open in Cloud Shell
List Objects With Old Versions source code Open in Cloud Shell
List Objects With Prefix source code Open in Cloud Shell
Make Object Public source code Open in Cloud Shell
Move Object source code Open in Cloud Shell
Print File Acl source code Open in Cloud Shell
Print File Acl For User source code Open in Cloud Shell
Release Event Based Hold source code Open in Cloud Shell
Release Temporary Hold source code Open in Cloud Shell
Remove File Owner source code Open in Cloud Shell
Rotate Object Encryption Key source code Open in Cloud Shell
Set Event Based Hold source code Open in Cloud Shell
Set Object Metadata source code Open in Cloud Shell
Set Object Retention Policy source code Open in Cloud Shell
Set Temporary Hold source code Open in Cloud Shell
Stream Object Download source code Open in Cloud Shell
Stream Object Upload source code Open in Cloud Shell
Upload Encrypted Object source code Open in Cloud Shell
Upload Kms Encrypted Object source code Open in Cloud Shell
Upload Object source code Open in Cloud Shell
Upload Object From Memory source code Open in Cloud Shell
Allow Divide And Conquer Download source code Open in Cloud Shell
Allow Parallel Composite Upload source code Open in Cloud Shell
Download Bucket source code Open in Cloud Shell
Download Many source code Open in Cloud Shell
Upload Directory source code Open in Cloud Shell
Upload Many source code Open in Cloud Shell

Troubleshooting

To get help, follow the instructions in the shared Troubleshooting document.

Transport

Cloud Storage uses HTTP/JSON for the transport layer.

Supported Java Versions

Java 8 or above is required for using this client.

Google's Java client libraries, Google Cloud Client Libraries and Google Cloud API Libraries, follow the Oracle Java SE support roadmap (see the Oracle Java SE Product Releases section).

For new development

In general, new feature development occurs with support for the lowest Java LTS version covered by Oracle's Premier Support (which typically lasts 5 years from initial General Availability). If the minimum required JVM for a given library is changed, it is accompanied by a semver major release.

Java 11 and (in September 2021) Java 17 are the best choices for new development.

Keeping production systems current

Google tests its client libraries with all current LTS versions covered by Oracle's Extended Support (which typically lasts 8 years from initial General Availability).

Legacy support

Google's client libraries support legacy versions of Java runtimes with long term stable libraries that don't receive feature updates on a best efforts basis as it may not be possible to backport all patches.

Google provides updates on a best efforts basis to apps that continue to use Java 7, though apps might need to upgrade to current versions of the library that supports their JVM.

Where to find specific information

The latest versions and the supported Java versions are identified on the individual GitHub repository github.com/GoogleAPIs/java-SERVICENAME and on google-cloud-java.

Versioning

This library follows Semantic Versioning, but does update Storage interface to introduce new methods which can break your implementations if you implement this interface for testing purposes.

Contributing

Contributions to this library are always welcome and highly encouraged.

See CONTRIBUTING for more information how to get started.

Please note that this project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms. See Code of Conduct for more information.

License

Apache 2.0 - See LICENSE for more information.

CI Status

Java Version Status
Java 8 Kokoro CI
Java 8 OSX Kokoro CI
Java 8 Windows Kokoro CI
Java 11 Kokoro CI

Java is a registered trademark of Oracle and/or its affiliates.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 99.9%
  • Other 0.1%