Skip to content

Latest commit

 

History

History
executable file
·
466 lines (285 loc) · 21.8 KB

README.md

File metadata and controls

executable file
·
466 lines (285 loc) · 21.8 KB
Raw

Here is the quickest way to get started with a macOS laptop (new or old) to work as a developer of several popular stacks and in several clouds."

TL;DR: Run!

This approach is faster and repeatable than manually clicking through everything. It's less error-prone because we've worked out the dependency clashes for you. We created the script to automatically take care of workarounds to known issues.

To avoid repeating myself, all you need is follow my step-by-step instructions at
https://wilsonmar.github.io/mac-setup

Why?

Most tutorials ask you to manually type or copy and paste strings from web pages (often with missing steps), which can take time, and be error-prone. And most webinar demos seem to brag rather than teach skills. To be more helpful, this repo talks with code by including manual documentation in the "configuration as code" movement for minimizing risk and ensuring consistency.

Roadmap

Scripts in this repo gives you a way to install, configure, start, and clean-up after a large set of programs runs code for several languages in various "stacks":

  • MEAN (MongoDB, Express, Angular, NodeJs) with the MeanJs sample app

  • MERN (MongoDB, Express, React/Redux, NodeJs) for "Universal" apps, including WebPack

  • PERN (PostgreSQL, Express, React, Node-postgres) with utilities PGAdmin, Postgresapp, react-bootstrap, nodemon, (Dockerized)

  • JAM (Jekyll, APIs, Markup) with a sample Github.io website

  • MAMP (Macintosh, Apache/Nginx, MySQL, PHP) for WordPress websites

  • Elastic (ELK) stack (Elasticsearch, Logstash, Kibana, etc.)

  • Serverless on Amazon Lambda, Azure Functions, Google Actions, Iron.io, etc.

  • CNCF (Cloud Native Computing Foundation) stack of Kubernetes, Prometheus monitoring, OpenTracing, JeagerTracing, FluentD log collector, and other projects

  • DevSecOps "stack" of Git, Nexus/Artifactory, Jenkins, MVN, Vagrant, Docker, etc.

  • Cloud management tools Terraform, AWS Cloud Formation, etc.

This repo enables you to get the above up and running in a matter of minutes. Being able to get started quickly means that you can get working with the application rather than the ceremonies of installation.

This repo brings to Mac users the "frequent, small, and reversible changes" for Agile.

In this course, well-known DevOps practitioners Ernest Mueller and James Wickett provide an overview of the DevOps movement, focusing on the core value of CAMS (culture, automation, measurement, and sharing)

Limited Memory

If you're now thinking "a Mac can't run every one of these programs" you're correct. A Mac has 16 GB - 32 GB of RAM.

This repo isn't designed to run every service, but to enable you to use one at a time.

The advantage gained by this script is fast change. This script enables you to switch quickly among sets of programs to quickly evaluate the technical aspects of each stack actually running -- not just conceptually -- but really working together at the same time.

Scripts here are modular. Its default setting is to not install everything. It installs only what you tell it to by adding a keyword in the control file.

Keywords to trigger install are specified in category variables:

  • MAC_TOOLS Homebrew, mas, Ansible, 1Password, PowerShell, etc.

  • DATA_TOOLS MongoDB, postgresql, mysql, mariadb, graphql?

  • EDITORS Atom, Code, Eclipse, Emacs, IntelliJ, Macvim, STS, Sublime, Textmate, vim

  • BROWSERS chrome, firefox, brave, phantomjs

  • GIT_CLIENTS git, cola, github, gitkraken, smartgit, sourcetree, tower, magit, gitup

  • GIT_TOOLS hooks, tig, lfs, diff-so-fancy, grip, p4merge, git-flow, signing, hub

  • JAVA_TOOLS Maven, Ant, Gradle, TestNG, Cucumber, Junit4, Junit5, Yarn, dbunit, Mockito, JMeter, GCViewer, JProfiler, etc.

  • PYTHON_TOOLS Virtualenv, jupyter, anaconda, ipython, numpy, scipy, matplotlib, pytest, robotframework, etc.

  • NODE_TOOLS Bower, gulp, gulp-cli, npm-check, jscs, less, jshint, eslint, webpack, etc.

  • LOCALHOSTS Apache (httpd, apachectl), iron

  • TEST_TOOLS selenium, sikulix, golum, dbunit?

  • CLOUD_TOOLS aws, gcp, azure, cf, heroku, docker, vagrant, terraform, serverless

  • MON_TOOLS (for monitoring) WireShark, Prometheus, others

  • VIZ_TOOLS (for visualization) Grafana, others (Prometheus, Kibana, Graphite)

  • COLAB_TOOLS (for collaboration) google-hangouts, hipchat, joinme, keybase, microsoft-lync, skype, slack, teamviewer, whatsapp, sococo, zoom

  • MEDIA_TOOLS Camtasia, Kindle (others: Snagit, etc.)

Links for individual apps above take you to technical descriptions about that technology.

The categories are run in dependency sequence. MAC_TOOLS are installed to provide underlying utilities, then DATA_TOOLS provides databases, then servers are installed, etc.

Run types

"Genius bars" providing support to laptop users make use of this to quickly ready a new laptop for developers joining their organization. This helps developers skip wasted days installing (and doing it differently than colleagues).

This repo brings DevSecOps-style "immutable architecture" to MacOS laptops. Immutability means replacing the whole machine instance instead of upgrading or repairing faulty components.

But you don't have to start from scratch.

This script also performs updates and uninstall too. Although you may use Apple's Time Machine app to backup everything to a USB drive or AirPort Time Capsule, you may want a way to keep up with the latest changes in apps updated to the latest version. Remember the "openssl" update scare?

This script upgrades all programs it knows about if you run the script with the RUNTYPE set to "upgrade". Use this script to install and configure what you need at the moment.

Change the RUNTYPE to "remove" and it clears up disk space. (But be careful that overuse can fragment your disk space)

Servers to work offline

This bash script enables you to work offline because it installs several servers. You manage allocation of port numbers in one place:

   ELASTIC_PORT="9200"    # DATA_TOOLS from default 9200
   GRAFANA_PORT="8089"    # VIZ_TOOLS from default 8080
   JEKYLL_PORT="4000"     # LOCAOHOSTS from default 4000
   JENKINS_PORT="8088"    # LOCALHOSTS from default 8080
   KIBANA_PORT="5601"     # DATA_TOOLS default 5601
   MYSQL_PORT="3060"      # DATA_TOOLS default 3060
   MEANJS_PORT="3000"     # NODE_TOOLS from default 3000
   MINIKUBE_PORT="8083"   # LOCAOHOSTS from default 8080
   NEO4J_PORT="7474"      # DATA_TOOL default 7474 HTTPS: 7473
   NGINX_PORT="8086"      # LOCALHOSTS from default 8080
   PACT_PORT="6666"       # TEST_TOOLS from default 6666
   POSTGRESQL_PORT="5432" # DATA_TOOLS default 5432
   PROMETHEUS_PORT="9090" # MON_TOOLS default 9090
   REDIS_PORT="6379"      # DATA_TOOLS default 6379
   SONAR_PORT="9000"      # DATA_TOOLS default 9000
   TOMCAT_PORT="8087"     # LOCALHOSTS from default 8080

Docker instances use the same ports.

The above list is from the secrets.sh file in your $HOME folder, which you edit to specify which port numbers and keywords to specify apps you want installed.

The file's name is suffixed with ".sh" because it is a runnable script that establishes memory variables for a Setup script to reference.

Homebrew

Most of the apps installed make use of installation code defined in the Homebrew repository online. There is a file (of Ruby code) for each brew install formula at:

https://github.com/Homebrew/homebrew-core/blob/master/Formula/httpd.rb

PROTIP: Before downloads a brew formula, we recommend that you look at its Ruby code to verify what really occurs and especially where files come from.

brew edit wget

In fact, we recommend that you install a binary repository proxy that supply you vetted files from a trusted server instead of retrieving whatever is the latest on the public Homebrew server.

Homebrew currently has over 4,500 formulas.

To install and configure programs which don't have brew installation formulas, various commands such as curl, sed, cut, etc. are used in the script.

Yes, you can just run brew yourself, one at a time. But logic in the script goes beyond what Homebrew does, and configures the component just installed:

  • Undo a brew error (such as needing an unset)
  • Install dependent components where necessary
  • Display the version number installed (to a log)
  • Add alias and paths in .bash_profile (if needed)
  • Perform configuration (such as adding a missing file needed for mariadb to start)
  • Edit configuration settings (such as changing default port within Nginx within config.conf file)
  • Upgrade and uninstall if that is available
  • Run a demo using the component to ensure that what has been installed actually works.

Make this work for you

The section below explains to someone relatively new to Mac machines the steps to automate installation of additional MacOS application programs. Along the way, we explore basic skills to use a command-line Terminal and common commands.

  1. Get Mac laptop hardware, boot-up

  2. Obtain the Mac's Launch bar by positioning your mouse at the bottom edge of the screen until it appears.

  3. If you don't see an icon for the Terminal program, click the magnifying glass icon always at the upper-right corner and type in Term until "Terminal app" is highlighted, then press Enter to accept it.

  4. Click menu Shell then click New Window for a Terminal session.

    PROTIP: More experienced people hover the mouse over New Window and click on one of the options.

    The Terminal program is called a "Bash" shell, which is a contraction of the term "Bourne-agan shell", which is a play on words.

    Version with Grep

  5. Test if you have Bash v4 installed by typing this:

    bash --version | grep 'bash'

    Bash 4.0 was released in 2009, but Apple still ships version 3.x, which first released in 2007.

    You have a recent version of Bash if you see:

    GNU bash, version 4.4.19(1)-release (x86_64-apple-darwin17.3.0)

    PROTIP: The attribute to obtain the version can vary among different commands. "--version" or "-v" or "version" may be used.

    Hold the Shift key to press the | (called pipe) key at the upper-right of the keyboard.

    The grep 'bash' is needed to filter out lines that do not contain the word "bash" in the response such as:

    GNU bash, version 4.4.19(1)-release (x86_64-apple-darwin17.3.0)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

    If you have bash v3 that comes with MacOS, this blog describes what is improved by version 4.

    mac-bash4.sh initialization

  6. Switch to back to this web page by holding down the command key and pressing Tab repeatedly until it rests on the browser icon.

  7. Triple-click on the script line below to highlight it for copying:

    sh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-bash-up.sh)"

  8. Press Command+C to copy it to your invisible Clipboard.

  9. Switch to the Terminal by holding down command and pressing Tab repeatedly until it rests on the Termial icon.

  10. At the Terminal, click on a Terminal window and paste in the command by holding down command then V. It doesn't matter what folder you're on at this point.

  11. Press Enter to run the command, which upgrades Bash to version 4 and copies a file to your Home folder.

    The script first makes use of the Ruby program to install Homebrew which, in turn, installs Bash v4 using the brew command to download and configure packages.

  12. After it runs, verify the version again as described above to ensure it's version 4.

    Home folder

  13. The default location the Terminal command opens to by default is your "Home" folder, which you can reach anytime by:

    cd

  14. The "~" (tilde character) prompt represents the $HOME folder, which is equivalent to a path that contains your user account, such as (if you were me):

    /Users/wilsonmar

  15. You can also use this variable to reach your account's Home folder:

    cd $HOME

    In other words these commands all achieve the same result:

    cd = cd ~ = cd $HOME

    secrets.sh at Home

    It's wise to avoid storing secrets in GitHub or other public repository. Files stored in your user $HOME holder (outside a Git-managed folder) have no chance to be uploaded from the Git repository. The script references secrets there.

    And if the script doesn't see a secrets file in your $HOME folder, it copies one there from the repo's sample file.

    NOTE the secrets.sh is a clear-text file.

    Encrypting and Decrypting secrets

    Optionally, you may store secrets and configurations in an encrypted file after some initial configuration.

    Run script ./secrets.edit.sh to decrypt the contents of secrets.sh for the mac-setup-all.sh script to use.

    Run script ./secrets.lock.sh to encrypt the contents of secrets.sh.

    Utilities "blackbox" or "git-secret" can be used to handle

    Text edit secrets.sh

  16. Use a text editor to edit the secrets.sh file using a text editor that comes pre-loaded on every Mac:

    textedit ~/secrets.sh

    The tilde character specifies that the file is in your Home folder.

    Top of file Shebang

    Looking in the file, consider the first line in the secrets.sh file:

    #!/bin/bash

    That is the "Bourne-compliant" path for the Bash v3.2 shell installed by default on MacOS up to High Sierra. BTW, other Linux flavors may alternately use this for portability:

    #!/usr/bin/env

    BTW, unlike Windows, which determines the program to open files based on the suffix (or extension) of the file name, Linux shell programs such as Bash reference the "shebang" on the first line inside the file.

  17. Open another Terminal window.

  18. View the above files to see that they are binary executable files, such as:

    textedit /usr/bin/bash

  19. Exit the file.

  20. Press the command key with the back-tick (`) at the upper-left of the keyboard to switch among textedit windows.

    Version 4 Shebang

    If you instead see this on the first line:

    #!/usr/local/bin/bash

    that is the Bash program associated with Bash v4.

    Version 4 is needed for "associative arrays" needed in the script.

    This is why we needed to first upgrade Bash before running other scripts.

    App keywords

    The initial secrets.sh file does not have keywords which specify additional apps to install.

  21. Scroll down or press command+F to type an app keyword to find its category.

    Edit port numbers

  22. Scroll to the list of ports (listed above).

  23. May sure that none of the ports are the same (conflicts).

  24. Save the file and exit the text editor.

    Setup all

  25. Now copy, switch, click and paste in a Terminal window to run this command:

    sh -c "$(curl -fsSL https://raw.githubusercontent.com/wilsonmar/mac-setup/master/mac-setup-all.sh)"

    The script referenced in the command obtains more files needed by cloning from a public GitHub repository ( https://github.com/wilsonmar/mac-setup) to a folder under your home folder named "mac-setup".

  26. Wait for the script to finish.

    On a 4mbps network the run takes less than 5 minutes for a minimal install. PROTIP: A faster network or a proxy Nexus server providing installers within the firewall would speed things up and ensure that vetted installers are used.

    When the script ends it pops up a log file in the TextEdit program that comes with MacOS.

  27. Switch to the TextEdit window by clicking it.

  28. Scroll to review the log file. Press command+F to input text to search.

  29. Close the log file by clicking the red button.

  30. Switch to a Finder window to your account's Home folder and delete log files.

    mac-setup files

    The folder contains these files and folders:

    • Files within folder "hooks" are used by Git (if marked for install.)
    • File "mac-bash-profile.txt" contains starter entries to insert in ~/.bash_profile that is executed before MacOS opens a Terminal session.

    Subsequent runs

    To update what is installed on your Mac, re-run the mac-setup.zsh bash script.

  31. cd into your Home folder to find the secrets.sh file.

  32. Edit the file, then run again locally:

    chmod +x mac-setup-all.sh
./mac-setup-all.sh

    The chmod (pronounced "che-mod") changes the permissions for executing the file.

    Now let's look at the Bash coding techniques used in the scripts mentioned above, at: https://wilsonmar.github.io/bash-coding

Mas Mac apps

The brew formula "mas" manages Apple Store apps, but it only manages apps that have already been paid for. mas does not install apps new to your Apple Store account.

Apps on Apple's App Store for Mac need to be installed manually. Popular apps include:

.pkg and .dmg

.pkg and .dmg files can be downloaded to install apps.

Cloud Sync

Apps for syncing to cloud providers are installed mostly for manual use:

Dropbox, OneDrive, Google Drive, Amazon Drive

Additional apps

Wait, there's more

Lists of Mac programs:

Others like this

Here are other scripts to install on Mac: