From 46563ff9810672b2ca011ce25d46f68e4764d3ae Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Tue, 2 Oct 2018 14:35:21 +0200 Subject: [PATCH 01/11] Add steps for deployment to Heroku --- README.md | 119 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 117 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 780cbcf..d6b85ad 100644 --- a/README.md +++ b/README.md @@ -39,7 +39,7 @@ The application requires: - `CORP_DOMAIN` is your corporate domain (i.e.: mycompany.com) used to identify Salesforce users without corporate email - `COOKIE_SECRET` is a secret used to sign the session cookie - `ADMIN_TOKEN` is a secret used to edit/delete Org information such as name or description - - `ENCRYPTION_KEY` is a hex string representing 32 random bytes, used to encrypt/decrypt the Oauth refresh tokens (AES 256) + - `ENCRYPTION_KEY` is 32 random bytes (**Please Note:** MUST be hex encoded), used to encrypt/decrypt the Oauth refresh tokens (AES 256) 3. Install Node.js dependencies through Yarn, with `yarn install` 4. Run the server with `node server.js`, confirm you see the `App listening on port 3000` message in the console 5. Load `http://localhost:3000/setup`, confirm you see the `Successfully setup DB` message in the console @@ -61,7 +61,122 @@ When ready for production deployment: - `SAML_ENTRY_POINT` - `SAML_ISSUER` - `SAML_CALLBACK` - - `SAML_CERT` + - `SAML_CERT` (maps to the 'cert' configuration parameter: *"Identity Provider's public PEM-encoded X.509 signing certificate using the cert confguration key. The "BEGIN CERTIFICATE" and "END CERTIFICATE" lines should be stripped out and the certificate should be provided on a single line."*) + +### Deployment to Heroku +Below are complete steps to deploy the application on Heroku. If you do not have a Heroku account head over to https://signup.heroku.com to create an account first. Once you have an account ensure you have the Heroku CLI (command line interface) installed (see https://devcenter.heroku.com/articles/heroku-cli). The below steps walks through setting up Org Monitor with a Developer Org but the steps applies equally well to a Production org. + +#### Clone Git Repo and Create App +```bash +# clone repo +$ git clone git@github.com:forcedotcom/OrgMonitor.git +$ cd OrgMonitor + +# create app (also sets git remote) +$ heroku apps:create --region eu + +# get appname from git remote +$ APP_NAME=`git remote get-url heroku | cut -d'/' -f4 | cut -d'.' -f1` +$ echo $APP_NAME +funky-medina-23982 +``` + +*Please Note:* Below I simply use funky-medina-23982 to refer to the app on Heroku i.e. what the APP_NAME variable contains now. + +#### Create Connected App in Salesforce +1. Open Salesforce Setup +2. Search for "App Manager" +3. Click "New Connected App" and fill in + - Connected App Name + - API Name + - Contact Email +4. Check "Enable OAuth Settings" +5. Set "Callback URL" to https://funky-medina-23982.herokuapp.com/callback (replace with actual app name) +6. Select the following OAuth Scopes: + - `Access and manage your data (api)` + - `Perform requests on your behalf at any time (refresh_token, offline_access)` +7. Save to close + +#### Gather Info (to replace below) +1. Reopen the Connected app and note down the "Consumer Key" and "Consumer Secret" +2. Hex encode 32 characters of random characters (http://www.convertstring.com/EncodeDecode/HexEncode) +3. Create yourself a password for `ADMIN_TOKEN` +4. Create yourself a password for `COOKIE_SECRET` + +#### Configure Heroku app, push source and open +```bash +# create addons and set config +heroku addons:create mongolab:sandbox +heroku addons:create heroku-postgresql:hobby-dev +heroku config:set CLIENT_ID=foo +heroku config:set CLIENT_SECRET=bar +heroku config:set REDIRECT_URI=https://$APP_NAME.herokuapp.com/callback +heroku config:set CORP_DOMAIN=lekkimworld.com +heroku config:set COOKIE_SECRET=baz +heroku config:set ADMIN_TOKEN=gaz +heroku config:set ENCRYPTION_KEY=3242384142324532343230334337313636384446313944453334394630334436 +heroku config:set NODE_ENV=development + +# push app source to Heroku +git push heroku master + +# start worker dyno +heroku ps:scale -a $APP_NAME worker=1:free + +# load /setup to configure app +curl https://$APP_NAME.herokuapp.com/setup + +# restart app +heroku restart -a $APP_NAME + +# open app in browser +open https://$APP_NAME.herokuapp.com +``` + +#### Test it out! +Now is a good time to ensure you can open the app in the browser. From here either follow the next section on how to configure SAML for authentication or skip it to move to adding an org to OrgMonitor. + +#### My Domain and SAML +1. In Salesforce Setup enable My Domain and deploy to all users (if not enabled). Note down the custom domain you've chosen. Below I use `demoitout.my.salesforce.com` +2. Search for "Identity Provider" in Setup and ensure Identity Provider is enabled +3. Search for "Single Sign-On Setings" in Setup and open +4. Ensure "SAML Enabled" is checked +5. Open the Connected App you created earlier +6. Check "Enable SAML" +7. Fill in + - Entity Id: "funky-medina-23982" (use actual app name) + - ACS URL: https://funky-medina-23982.herokuapp.com/login/callback +8. Save to close + +```bash +# configure app to use SAML for login +heroku config:set SAML_CALLBACK=https://$APP_NAME.herokuapp.com/login/callback +heroku config:set SAML_ISSUER=$APP_NAME +heroku config:set SAML_CERT=MIIErDCCA...96TOK7Ph +heroku config:set SAML_ENTRY_POINT=https://demoitout.my.salesforce.com/idp/endpoint/HttpRedirect + +# set NODE_ENV to production to require authentication +heroku config:set -a $APP_NAME NODE_ENV=production + +# open app to ensure it requires to to authenticate +open https://$APP_NAME.herokuapp.com +``` + +#### Create Salesforce user and Profile +1. Open Salesforce Setup +2. Clone the "Standard User" Profile and call it "Org Monitor" (or what ever you wish) and remove all rights, CRUD access etc. Now check the following permissions: + - `API Enabled` + - `View All Users` + - `View Health Check` + - `View Setup and Configuration` +3. Ensure the Profile allows access to the Connected App you created +4. Save the Profile +5. Create a new user assigning the Profile you just created. Remove all rights. Login as the created user using the reset-link as normal. + +#### Add org to OrgMonitor +1. Open the app to /add/prod to add a Production / Developer org (using https://login.salesforce.com for login) or /add/sandbox to add a Sandbox org (using https://test.salesforce.com for login) +2. Perform OAuth authorization +3. Ensure org shows up in OrgMonitor with data ## License From d117b3ec497c6ecde81f791730a85f2bcc85d4be Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Tue, 2 Oct 2018 14:35:47 +0200 Subject: [PATCH 02/11] Remove unused import and make Syncing.. into Syncing... --- routes/index.js | 3 +-- worker.js | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/routes/index.js b/routes/index.js index df31879..ca79e46 100644 --- a/routes/index.js +++ b/routes/index.js @@ -8,7 +8,6 @@ const express = require('express') const router = express.Router() const jsforce = require('jsforce') -// const _ = require('lodash') const compare = require('secure-compare') const Org = require('../lib/org.js') const Crypto = require('../lib/crypto.js') @@ -78,7 +77,7 @@ router.get('/callback', async (req, res) => { instanceUrl: conn.instanceUrl, loginUrl: conn.loginUrl, refreshToken: Crypto.encrypt(conn.refreshToken), - healthCheckScore: 'Syncing..' + healthCheckScore: 'Syncing...' } // Store credentials in DB diff --git a/worker.js b/worker.js index 01b7423..cd25b31 100644 --- a/worker.js +++ b/worker.js @@ -10,7 +10,7 @@ const Org = require('./lib/org.js') agenda.define('refreshOrg', async (job, done) => { const jobData = job.attrs.data - console.log(`[${jobData.orgId}] Syncing..`) + console.log(`[${jobData.orgId}] Syncing...`) let org = await Org.get(jobData.orgId) let data = await org.fetchRemoteData() await Org.saveData(data) From 525d702368516d91a371c3a2c20544227b748630 Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Tue, 2 Oct 2018 14:49:43 +0200 Subject: [PATCH 03/11] README clarifications --- README.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index d6b85ad..fbcfd6c 100644 --- a/README.md +++ b/README.md @@ -93,13 +93,13 @@ funky-medina-23982 4. Check "Enable OAuth Settings" 5. Set "Callback URL" to https://funky-medina-23982.herokuapp.com/callback (replace with actual app name) 6. Select the following OAuth Scopes: - - `Access and manage your data (api)` - - `Perform requests on your behalf at any time (refresh_token, offline_access)` + - Access and manage your data (api) + - Perform requests on your behalf at any time (refresh_token, offline_access) 7. Save to close #### Gather Info (to replace below) -1. Reopen the Connected app and note down the "Consumer Key" and "Consumer Secret" -2. Hex encode 32 characters of random characters (http://www.convertstring.com/EncodeDecode/HexEncode) +1. Reopen the Connected app and note down the "Consumer Key" (`CLIENT_ID`) and "Consumer Secret" (`CLIENT_SECRET`)) +2. Hex encode 32 characters of random characters (http://www.convertstring.com/EncodeDecode/HexEncode) (`ENCRYPTION_KEY`) 3. Create yourself a password for `ADMIN_TOKEN` 4. Create yourself a password for `COOKIE_SECRET` @@ -137,7 +137,7 @@ open https://$APP_NAME.herokuapp.com Now is a good time to ensure you can open the app in the browser. From here either follow the next section on how to configure SAML for authentication or skip it to move to adding an org to OrgMonitor. #### My Domain and SAML -1. In Salesforce Setup enable My Domain and deploy to all users (if not enabled). Note down the custom domain you've chosen. Below I use `demoitout.my.salesforce.com` +1. In Salesforce Setup enable My Domain and deploy to all users (if not enabled). Note down the custom domain you've chosen. Below I use `demoitout.my.salesforce.com` (see when I set SAML_ENTRY_POINT) 2. Search for "Identity Provider" in Setup and ensure Identity Provider is enabled 3. Search for "Single Sign-On Setings" in Setup and open 4. Ensure "SAML Enabled" is checked From 5ba9b3b9862ea729188178be68a6f5cece4b2710 Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Wed, 3 Oct 2018 14:18:32 +0200 Subject: [PATCH 04/11] Ignore my steps --- .gitignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitignore b/.gitignore index 5ce57aa..ecb06cd 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,4 @@ node_modules npm-debug.log .DS_Store .env +my_steps.txt From add05d578bfc4582726d4b14d17a5aee70ddede6 Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Thu, 3 Jan 2019 11:08:46 +0100 Subject: [PATCH 05/11] Add question for custom objects using filter --- lib/org.js | 2 +- lib/questions.js | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/org.js b/lib/org.js index cc18078..8e2aa33 100644 --- a/lib/org.js +++ b/lib/org.js @@ -66,7 +66,7 @@ class Org { queryMore(data, result) }) } else { - result.records = data // to preserve other metadata info + result.records = question.filter ? data.filter(question.filter) : data // to preserve other metadata info resolve(result) } } diff --git a/lib/questions.js b/lib/questions.js index 612755b..e38d475 100644 --- a/lib/questions.js +++ b/lib/questions.js @@ -47,7 +47,8 @@ let questions = [ { icon: 'fa-code', name: 'Apex classes', q: 'select count() from apexclass' }, { icon: 'fa-code', name: 'Apex triggers', q: 'select count() from apextrigger' }, { engine: 'tooling', icon: 'fa-code', name: 'Remote Site Settings', q: 'select SiteName,EndpointUrl,ProtocolMismatch from remoteproxy where IsActive=true' }, - { engine: 'tooling', icon: 'fa-code', name: 'sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true' } + { engine: 'tooling', icon: 'fa-code', name: 'sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true' }, + { engine: 'tooling', icon: 'fa-code', name: 'sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true', filter: (record) => record.QualifiedApiName.substr(-3) === '__c' } ] module.exports = questions From 9e657a9c781377df5dfaba8f86ff17d32d58fbbc Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Thu, 3 Jan 2019 11:10:47 +0100 Subject: [PATCH 06/11] Update name of question --- lib/questions.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/questions.js b/lib/questions.js index e38d475..6c4b1d3 100644 --- a/lib/questions.js +++ b/lib/questions.js @@ -48,7 +48,7 @@ let questions = [ { icon: 'fa-code', name: 'Apex triggers', q: 'select count() from apextrigger' }, { engine: 'tooling', icon: 'fa-code', name: 'Remote Site Settings', q: 'select SiteName,EndpointUrl,ProtocolMismatch from remoteproxy where IsActive=true' }, { engine: 'tooling', icon: 'fa-code', name: 'sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true' }, - { engine: 'tooling', icon: 'fa-code', name: 'sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true', filter: (record) => record.QualifiedApiName.substr(-3) === '__c' } + { engine: 'tooling', icon: 'fa-code', name: 'Custom sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true', filter: (record) => record.QualifiedApiName.substr(-3) === '__c' } ] module.exports = questions From 64c45cb22311362dffed083eaf8cfe19b7f59092 Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Thu, 3 Jan 2019 11:35:52 +0100 Subject: [PATCH 07/11] Moved where filter was applied --- lib/org.js | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/lib/org.js b/lib/org.js index 8e2aa33..fb92439 100644 --- a/lib/org.js +++ b/lib/org.js @@ -66,7 +66,7 @@ class Org { queryMore(data, result) }) } else { - result.records = question.filter ? data.filter(question.filter) : data // to preserve other metadata info + result.records = data // to preserve other metadata info resolve(result) } } @@ -89,6 +89,13 @@ class Org { try { answer.a = await query(question) + console.log(`Looking for filter for question with name '${question.name}`) + if (answer.a && answer.a.records && question.filter) { + console.log(`Found filter for question with name '${question.name} - invoking it on the records (count: ${answer.a.records.length})`) + let records = answer.a.records.filter(question.filter) + console.log(`Invoking filter for question with name '${question.name} reduced records from ${answer.a.records.length} to ${records.length}`) + answer.a.records = records + } } catch (e) { console.error(`[${this.orgId}] Non-blocking error while querying data:`, { query: question.q, error: JSON.stringify(e) }) answer.a = null From f511032d7e8f82634e33608e783190302ca1d9a4 Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Thu, 3 Jan 2019 11:45:02 +0100 Subject: [PATCH 08/11] Swap params --- lib/questions.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/questions.js b/lib/questions.js index 6c4b1d3..b8aaa1a 100644 --- a/lib/questions.js +++ b/lib/questions.js @@ -48,7 +48,7 @@ let questions = [ { icon: 'fa-code', name: 'Apex triggers', q: 'select count() from apextrigger' }, { engine: 'tooling', icon: 'fa-code', name: 'Remote Site Settings', q: 'select SiteName,EndpointUrl,ProtocolMismatch from remoteproxy where IsActive=true' }, { engine: 'tooling', icon: 'fa-code', name: 'sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true' }, - { engine: 'tooling', icon: 'fa-code', name: 'Custom sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true', filter: (record) => record.QualifiedApiName.substr(-3) === '__c' } + { engine: 'tooling', icon: 'fa-code', name: 'Custom sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsFlsEnabled=true and IsCustomizable=true', filter: (record) => record.QualifiedApiName.substr(-3) === '__c' } ] module.exports = questions From 07048d2b34abf58b0dac25570acb7e130abeb925 Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Thu, 3 Jan 2019 11:46:32 +0100 Subject: [PATCH 09/11] Update size --- lib/org.js | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/org.js b/lib/org.js index fb92439..293a9a5 100644 --- a/lib/org.js +++ b/lib/org.js @@ -95,6 +95,7 @@ class Org { let records = answer.a.records.filter(question.filter) console.log(`Invoking filter for question with name '${question.name} reduced records from ${answer.a.records.length} to ${records.length}`) answer.a.records = records + answer.a.size = records.length } } catch (e) { console.error(`[${this.orgId}] Non-blocking error while querying data:`, { query: question.q, error: JSON.stringify(e) }) From 049e53173d84ddc110490b5c490d8d70c7749d28 Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Thu, 3 Jan 2019 11:48:34 +0100 Subject: [PATCH 10/11] Update totalSize as well --- lib/org.js | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/org.js b/lib/org.js index 293a9a5..6b4202a 100644 --- a/lib/org.js +++ b/lib/org.js @@ -96,6 +96,7 @@ class Org { console.log(`Invoking filter for question with name '${question.name} reduced records from ${answer.a.records.length} to ${records.length}`) answer.a.records = records answer.a.size = records.length + answer.a.totalSize = records.length } } catch (e) { console.error(`[${this.orgId}] Non-blocking error while querying data:`, { query: question.q, error: JSON.stringify(e) }) From b24c1ab9cd4e582df3f5097817790b674becdeec Mon Sep 17 00:00:00 2001 From: Mikkel Flindt Heisterberg Date: Thu, 3 Jan 2019 11:53:34 +0100 Subject: [PATCH 11/11] Add question for custom objects with FILTER_NAMESPACE from env --- lib/questions.js | 3 +++ 1 file changed, 3 insertions(+) diff --git a/lib/questions.js b/lib/questions.js index b8aaa1a..f59d905 100644 --- a/lib/questions.js +++ b/lib/questions.js @@ -50,5 +50,8 @@ let questions = [ { engine: 'tooling', icon: 'fa-code', name: 'sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsCustomizable=true and IsFlsEnabled=true' }, { engine: 'tooling', icon: 'fa-code', name: 'Custom sObjects', q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsFlsEnabled=true and IsCustomizable=true', filter: (record) => record.QualifiedApiName.substr(-3) === '__c' } ] +if (process.env.FILTER_NAMESPACE) { + questions.push({ engine: 'tooling', icon: 'fa-code', name: `Custom sObjects with ${process.env.FILTER_NAMESPACE} namespace`, q: 'select MasterLabel,QualifiedApiName,NewUrl from entitydefinition where IsFlsEnabled=true and IsCustomizable=true', filter: (record) => record.QualifiedApiName.substr(-3) === '__c' && record.QualifiedApiName.indexOf(process.env.FILTER_NAMESPACE) === 0 }) +} module.exports = questions