Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Google Sheets Integration #142

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Google Sheets Integration #142

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions src/main/java/com/impactupgrade/nucleus/client/GoogleSheetsClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
package com.impactupgrade.nucleus.client;

import com.google.api.client.googleapis.auth.oauth2.GoogleCredential;
import com.google.api.client.googleapis.javanet.GoogleNetHttpTransport;
import com.google.api.client.json.gson.GsonFactory;
import com.google.api.services.sheets.v4.Sheets;
import com.google.api.services.sheets.v4.SheetsScopes;
import com.google.api.services.sheets.v4.model.ValueRange;
import com.impactupgrade.nucleus.environment.Environment;
import com.impactupgrade.nucleus.environment.EnvironmentConfig;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.json.JSONObject;

import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.security.GeneralSecurityException;
import java.util.List;

public class GoogleSheetsClient {

private static final Logger log = LogManager.getLogger(GoogleSheetsClient.class);

private final Sheets sheets;

public GoogleSheetsClient(EnvironmentConfig.GoogleSheets googleSheetsConfig) throws GeneralSecurityException, IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you detail for us how all these details are initially found? Do we need an Oauth connection process in Portal? Does it require a Google app?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requires connected app in Google Cloud and service account with granted access to the given spreadsheet.
I will add these details as a comment in the code.

// Build service account credential
// Configuration parameters represent Google's JSON file with the credential information
JSONObject jsonObject = new JSONObject();
jsonObject.put("type", "service_account");
jsonObject.put("project_id", googleSheetsConfig.projectId);
jsonObject.put("private_key_id", googleSheetsConfig.privateKeyId);
jsonObject.put("private_key", googleSheetsConfig.secretKey);
jsonObject.put("client_email", googleSheetsConfig.clientEmail);
jsonObject.put("client_id", googleSheetsConfig.clientId);
jsonObject.put("auth_uri", googleSheetsConfig.authUri);
jsonObject.put("token_uri", googleSheetsConfig.tokenServerUrl);
jsonObject.put("auth_provider_x509_cert_url", googleSheetsConfig.authProviderCertUrl);
jsonObject.put("client_x509_cert_url", googleSheetsConfig.clientCertUrl);
jsonObject.put("client_x509_cert_url", googleSheetsConfig.clientCertUrl);
jsonObject.put("universe_domain", "googleapis.com");

// For this scenario a service account is needed, which is an account that belongs to the connected application instead of
// to an individual end user.
// Google APIs is called on behalf of the service account, so users
// aren't directly involved (there is no redirect to the login screen and user is not prompted to grant permissions to connected app)
GoogleCredential googleCredentials = GoogleCredential
.fromStream(new ByteArrayInputStream(jsonObject.toString().getBytes()))
.createScoped(List.of(SheetsScopes.SPREADSHEETS));

sheets = new Sheets.Builder(
GoogleNetHttpTransport.newTrustedTransport(),
GsonFactory.getDefaultInstance(), googleCredentials)
.setApplicationName(googleSheetsConfig.applicationName)
.build();
}

public void append(String sheetId, List<Object> cellValues) throws IOException {
ValueRange valueRange = new ValueRange().setValues(List.of(cellValues));
sheets.spreadsheets().values()
// appending 1 row starting from 1st column
.append(sheetId, "A1", valueRange)
.setValueInputOption("RAW")
.execute();
}

public static void main(String[] args) throws GeneralSecurityException, IOException {

Environment env = new Environment() {
@Override
public EnvironmentConfig getConfig() {
EnvironmentConfig envConfig = new EnvironmentConfig();
envConfig.googleSheets.projectId = "gentle-complex-390515";
envConfig.googleSheets.privateKeyId = "804553f7d5f18d087b62459f90e5a1b14ed509c9";
envConfig.googleSheets.secretKey = "-----BEGIN PRIVATE KEY-----\nMIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDdUbanTn8rq7mG\nlPY42U8lrY0rUVCYy6YJjeYtl8Sv/RKXIlqkT+7YU2d0n0E7xP0NWo5fQ8PGJ3Px\n2zB23CepBREPjctc8TtRvDAguXkicu+hIiamAIT6mnmd0DLPpQyJogd1BUgVyJF4\n6HQtnmyBPUsm9Eu/FwZ3CDvkPV3zY6lBDg9Zwnt07uS5zWpTJbHSTfin3ZKWVGt4\nQu5n+gU0pvAFUWDd16MgBNfuSjXRMZGzHI0bdXHT40Um0Oinmhiab65N/xXtLZfU\nMGSiMgUc3BQXccchhMhiLcuBXATSTyE+B6MiJzunDSaCHG0cHCHHdPuH4WAsUZhs\n4jfg/hfZAgMBAAECggEACP0fX8KwqA+osJZevLbYv9VBaUu7bAVDaFpuyZXDK4mq\nBmDjEQ7dCsSybDpmnhyVUYRGyYg5TJRAIYfPO1icNMFrrLfL1WnHyL1NsBqQWK2V\n3XPDYZUeUYZSH657zdKshG+EAYT2JUpY3DIGu+6WBha89WdRJ0DyZoW7Vv0GEpM7\nj/3hPUTUXoQZdXRLKlhCX/dVJduo2QIJjIIu8bzCLmlqTCa8cqYAx3UwJulfHXoY\nz8Ug2TYzRTykr02JPNyuNuk0R4UA6M4SZSdYpMhDBpb70OGE+sBCndKEjeouOS2S\nQcHkLR0O1IMzp2sz7IxL1j5DQeb0SZCr2CboTJ8j7QKBgQDvFQrkzos44foW8tNn\nGnm78VnGv+T16bcSORpvHxUlm2pfCdIlVdHgKa1Dvm4pIs3LlSLnN6Mg550Ni0Mo\nOPdy2KLaG3GU74ytqOoFdbi+JifaI2/lRVvLrQ06rnUI6JLjY5fWJF6zdl3XyxsN\nf2+echzxDv85t/SCoXzPRO2NzQKBgQDs+uU1chU5WxqzOiYTB3v4jM2FTC6xJNjs\nzWf6kVHyQ7/sqDRG3ginYx8vMPB16e1aExzo3mgpfwyXKjxPQ+qa1QT0QEdlvi1p\nGfa7fdlN/7hAN7tl0DYIKzItk0hNXWJewyvuE2y8dK3ZS3gF+YMMMJ5e2hkrYbu1\nUlUu8tKGPQKBgQDOgy2awDIP21orsmoa2Aqo5eu3OpAqPkvtCLglngKlLl6uYwxL\nRZr49ub76iS7kZ2TqWmxsSROSuIlDdLfjn1njWr13Ni6XkT0yEAEoVAHp2urCAsi\nTkvhXcRcmM7s9//RPHit91J5z9d1i7H9ccNXaJhJPLwG/jfNEnJ9krtjTQKBgQDU\nMjipabTdfdljoO7U3T/BqHqjIDsy/ZaMO8UeVZ91+fpR86+TwV8oWxZiUEUQoF2a\n6UBauEO23H+un/AO3falm5brCt+jl+3bjZcj/aVmNVOLlRvlJ9Ip8Fvm+VmlhLf/\nuG2OqbAU87lzuCMJ3ojcknBM6Kfe84175/RErMOb1QKBgQCn3rN0cG+wrjtt/zmh\nz9FsAcNa9uz7hxYGjPaOd79whEeNYCI4bfDgZwDl68rdl+i6t6t8i3RRJ/A3THGo\noe52rawtJTq9ZumH9Zk0AzjvGzH7NK6oLRgmXTegGEiaZZXLJy4mjn5y6vJxHGO6\nev22ZUlhqKsvCu7t8ZF38n59kw==\n-----END PRIVATE KEY-----\n";
envConfig.googleSheets.clientEmail = "[email protected]";
envConfig.googleSheets.clientId = "108979308765806490060";
envConfig.googleSheets.authUri = "https://accounts.google.com/o/oauth2/auth";
envConfig.googleSheets.tokenServerUrl = "https://oauth2.googleapis.com/token";
envConfig.googleSheets.authProviderCertUrl = "https://www.googleapis.com/oauth2/v1/certs";
envConfig.googleSheets.clientCertUrl = "https://www.googleapis.com/robot/v1/metadata/x509/service-accoint1%40gentle-complex-390515.iam.gserviceaccount.com";
return envConfig;
}
};


GoogleSheetsClient googleSheetsClient = new GoogleSheetsClient(env.getConfig().googleSheets);
googleSheetsClient.append(
"13kEKngPh9_ksp3lC7GhiYDTGY36h8IIaNY8mIIuBvMk",
List.of("value1", "value2", "value3", "value4"));
}

}
12 changes: 12 additions & 0 deletions src/main/java/com/impactupgrade/nucleus/environment/EnvironmentConfig.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,6 +343,18 @@ public static class MBT extends Platform {
}

public MBT mbt = new MBT();

public static class GoogleSheets extends Platform {
public String clientEmail;
public String privateKeyId;
public String projectId;
public String authUri;
public String authProviderCertUrl;
public String clientCertUrl;
public String applicationName;
}

public GoogleSheets googleSheets = new GoogleSheets();

public MetadataKeys metadataKeys = new MetadataKeys();

Expand Down
8 changes: 5 additions & 3 deletions src/main/java/com/impactupgrade/nucleus/service/segment/XeroAccountingPlatformService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
import com.google.api.client.http.BasicAuthentication;
import com.google.api.client.http.GenericUrl;
import com.google.api.client.http.javanet.NetHttpTransport;
import com.google.api.client.json.jackson2.JacksonFactory;
import com.google.api.client.json.gson.GsonFactory;
import com.google.common.base.Strings;
import com.impactupgrade.nucleus.dao.HibernateDao;
import com.impactupgrade.nucleus.entity.Organization;
Expand Down Expand Up @@ -195,7 +195,9 @@ protected Optional<Contact> getContact(String accountNumber) throws Exception {
// Boolean includeArchived,
null,
// Boolean summaryOnly
true
true,
// String searchTerm
null
);
return contacts.getContacts().stream().findFirst();
}
Expand Down Expand Up @@ -343,7 +345,7 @@ protected String getAccessToken() throws Exception {
log.info("token expired; jwt={} now={}; refreshing...", jwt.getExpiresAt().getTime(), now);

try {
TokenResponse tokenResponse = new RefreshTokenRequest(new NetHttpTransport(), new JacksonFactory(),
TokenResponse tokenResponse = new RefreshTokenRequest(new NetHttpTransport(), GsonFactory.getDefaultInstance(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VSydor Are we positive this won't break anything? There are caveats to how Jackson vs. Gson handle default mappings. IIRC, Jackson defaults to camelCase while Gson defaults to snake_case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, good point - will double check this part.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@brmeyer xero-java is using an older version of com.google.api-client:google-api-client lib (same lib required for google sheets integration)
I think the best we can do is to upgrade the version of xero java (use the newest one) that relies on the most recent version of google-api-client.
Updated nucleus-parent to use most recent version of xero-java:
impactupgrade/nucleus-parent#8

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VSydor Forgive me for not remembering this: since nucleus-parent is updated, do we still need this switch to Gson in Xero?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Short answer is: yes - we do

Long one:
Both Google Sheets lib (google-api-services-sheets) and Xero lib (xero-java) depend on (google-oauth-client)
Sheets is using (1.34.1) and Xero uses (1.23.0)

We either need to update xero lib version (will also use newer version of google-auth), or just update the code to use Sheets version of google auth (Gson instead of Jackson)

new GenericUrl(tokenServerUrl), refreshToken)
.setClientAuthentication(new BasicAuthentication(this.clientId, this.clientSecret))
.execute();
Expand Down
Loading