Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial docs for installing on GKE #172

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Conversation

wajihyassine
Copy link
Member

@wajihyassine wajihyassine commented Oct 3, 2024

Description of the change

Updates the documentation for deploying OSDFIR Infrastructure on GKE.

Applicable issues

Additional information

Checklist

  • Chart version bumped in Chart.yaml according to semver. This is not necessary when the changes only affect README.md files.
  • Newly added variables are documented in the values.yaml
  • Title of the pull request is descriptive

@wajihyassine wajihyassine self-assigned this Oct 4, 2024
@wajihyassine wajihyassine marked this pull request as ready for review October 4, 2024 21:30
@wajihyassine
Copy link
Member Author

wajihyassine commented Oct 4, 2024

@hacktobeer @jkppr @tomchop @aarontp -> would you guys mind giving this a try and letting know if this was easy to follow/no failed steps along the way? Should only take ~10-15mins but if it takes longer also would be good to know

Copy link
Collaborator

@hacktobeer hacktobeer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL, I was not able to continue past a permission error. See comments.

export ZONE="us-central1-f" # The zone where you want to create the cluster
export CLUSTER="osdfir-cluster" # The name you choose for your K8s cluster
export NAMESPACE="default" # Your K8s namespace (can be left as 'default')
export KSA_NAME="turbinia" # Your Turbinia K8s service account (defaults to 'turbinia' if not set)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was unclear to me if this had to be created or not...

plugin for kubectl:

```bash
gcloud components install gke-gcloud-auth-plugin
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ERROR: (gcloud.components.install)
You cannot perform this action because the Google Cloud CLI component manager
is disabled for this installation. You can run the following command
to achieve the same result for this installation:

sudo apt-get install google-cloud-cli-gke-gcloud-auth-plugin

```bash
git clone https://github.com/log2timeline/dftimewolf.git && cd dftimewolf
pip install poetry
poetry install && poetry shell
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

had an issue on my machine where $HOME/.local/bin was not added to my PATH so poetry was not found.

First, create a disk to process using a name such as `test-disk`:

```bash
gcloud compute disks create test-disk --zone $ZONE
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This created an empty 500GB disk...maybe create a 10GB disk with the default debian12 image?

Then on a third terminal, run the dfTimewolf recipe:

```bash
dftimewolf gcp_turbinia_ts $PROJECT_ID --disk_names test-disk
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to add $ZONE at the end iof the command.
dftimewolf gcp_turbinia_ts $PROJECT_ID --disk_names test-disk $ZONE

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Command (and all Turbinia tasks failed) with below error:

("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/[email protected]/token?scopes=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform%2Chttps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute%2Chttps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute.readonly%2Chttps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.full_control%2Chttps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.read_only%2Chttps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdevstorage.read_write from the Google Compute Engine metadata service. Status: 404 Response:\nb'Unable to generate access token; IAM returned 404 Not Found: Not found; Gaia id not found for email [email protected]\\n'", <google_auth_httplib2._Response object at 0x7e88f6045de0>). Something is wrong with your Application Default Credentials. Try running: $ gcloud auth application-default login]
```

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like some service account/permission issue.....

I tried everything in a fresh/clean/reset gcp cloudshell (so a new vm)


## Step 3: Create the Turbinia GCP Service Account

To process virtual machine disks in Google Cloud Platform (GCP) with Turbinia,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to create the "turbinia" service account first as it does not exist!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create Installing OSDFIR Infrastructure on GKE Docs
2 participants