From b036170e8cc552d76673c1cbc39e252be2910016 Mon Sep 17 00:00:00 2001 From: stuxf <70670632+stuxf@users.noreply.github.com> Date: Thu, 10 Aug 2023 02:15:27 +0000 Subject: [PATCH 1/3] add docs --- docs/index.rst | 6 ++ docs/quickstart.rst | 158 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 164 insertions(+) create mode 100644 docs/quickstart.rst diff --git a/docs/index.rst b/docs/index.rst index e84a492..0497bc4 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -99,6 +99,12 @@ the ``docker`` image; you can use ``python`` or any other image with a working Introduction +.. toctree:: + :maxdepth: 1 + :caption: Quickstart Guide + + quickstart + .. toctree:: :maxdepth: 1 :caption: Contents diff --git a/docs/quickstart.rst b/docs/quickstart.rst new file mode 100644 index 0000000..3c9f975 --- /dev/null +++ b/docs/quickstart.rst @@ -0,0 +1,158 @@ +Quickstart Guide +================ + +This guide will help you get started with using rcds, and deploying to both kubernetes and rctf backends. + +This guide assumes that you have already set up a working rctf instance. It also assumes that you will be using GKE (Google Kubernetes Engine) as your kubernetes backend, and cloudflare as your DNS provider. + +Using at least python version 3.9, clone this repository, and run the following command to install rcds onto your system: + +.. code-block:: bash + + git clone https://github.com/redpwn/rcds + cd rcds + pip3 install . + +Furthermore, also make sure to install the following tools onto your system. + +- **kubectl**: Kubernetes command-line tool for interacting with clusters: `kubectl Installation Guide `_ + +- **helm**: Package manager for Kubernetes: `helm Installation Guide `_ + +- **gcloud**: Command-line interface for Google Cloud Platform (GCP): `gcloud Installation Guide `_ + +- **docker-engine**: Used for building docker containers on your system: `docker Installation Guide `_ + +First, create a folder or repository where you will be storing your challenges to deploy. + +The directory structure should look like something like this: + +.. code-block:: bash + + . + ├── category1 + │ ├── challenge1 + │ │ ├── Dockerfile + │ │ ├── challenge1.file + │ │ └── challenge.yml + │ ├── challenge2 + │ │ ├── Dockerfile + │ │ ├── challenge2.file + │ │ └── challenge.yml + │ └── challenge3 + │ ├── Dockerfile + │ ├── challenge3.file + │ └── challenge3.yml + │ + ├── category2 + │ ├── chalenge1... + │ + └── rcds.yaml + + +If you want an example, check out AmateursCTF's challenge repository here: `AmateursCTF Challenge Repository `_ + +Inside the folder, create a file called ``rcds.yaml``. This file will contain the configuration for your deployment. An example of this config can be found at `Sample Configs <./config-samples#gke-and-rctf-on-gitlab-ci>`_ + +Configure the ``rcds.yaml`` file to match your deployment. More information about the configuration options can be found at `Configuration <./project>`_ + +Additionally, create a ``.env`` file. It should look like the following: + +.. code-block:: env + + RCDS_RCTF_URL=https://ctf.example.com + RCDS_RCTF_TOKEN=999omGulJ8OUxy+NNMmfV4VbErhHf5HTxRU07FKFdDYQmEGworLsxl2G6Hdl6BgrkYvhfAZoR0IEdE0XXlurGB1szIjdIk1whr3iSP2ZIdAC7chSDlk9SL/iN68J + +You can obtain the token by creating an admin account (`Instructions here `_), copying the token from the admin page, and then url decoding it. + +From google cloud, create a standard kubernetes cluster. Make sure that it is NOT an autopilot cluster, which google cloud will try to default to. If doing this through the UI, there should be an option on the top right to switch to a standard cluster. + +Enable Dataplane V2 if it is not already enabled by default. Additionally, under the network settings tab, create a new tag called ``open-nodeports``. We will be using this later to configure the firewall. + +Once the kubernetes cluster is created, click the connect button on the top bar of the cluster page and connect to the cluster using the command provided. This will set up your local kubectl configuration to connect to the cluster. + +Additionally, we will need to create a container registry to store our docker images. To do this, go to the container registry page, and create a new registry. Make sure to select the same region as your kubernetes cluster. + +Once the registry is created, we will need to configure docker to be able to push to this registry. To do this, right click on the registry, and click setup instructions on google cloud. This will give you a command to run to configure docker to push to this registry. Run this command on your local machine. + +Make sure to configure your ``rcds.yaml`` file to match the name of your container registry. + +Once that's done, go to the VPC Network tab of google cloud, and assign a static IP address to one of the nodes in your cluster. This will be the IP address that your challenges will be hosted on, so configure DNS to point to this IP address. + +Additionally, configure firewall rules in the VPC Network tab to allow traffic for the allow-nodeports rule that we created earlier. + +Once done, install the following helm charts onto your cluster: + +- **Traefik**: In order to install Traefik, follow the guide provided in the Traefik documentation on using Helm charts: `Install Traefik using Helm `_ + +- **Cert-Manager**: For installing Cert-Manager, ensure that you install CRDs using the second option: `Install Cert-Manager with CRDs using Helm `_ + +Once done, go to the VPC Network IP addresses tab, and convert the traefik IP address to a static IP address. This will be the IP address that your web challenges will be hosted on, so add a wildcard DNS entry to point to this IP address. For example, add an A record pointing at ``*.example.com``. + +Finally, we're going to configure the automatic TLS certificate generation. To do this, fill out the following template and name it ``certs.yml``: + +.. code-block:: yaml + + apiVersion: v1 + kind: Secret + metadata: + name: cloudflare-token + type: Opaque + stringData: + api-token: "API_TOKEN_GOES_HERE" + --- + apiVersion: cert-manager.io/v1 + kind: Issuer + metadata: + name: letsencrypt-issuer + spec: + acme: + email: "EMAIL@GOES-HERE" + server: https://acme-v02.api.letsencrypt.org/directory + privateKeySecretRef: + name: letsencrypt-issuer-key + solvers: + - dns01: + cloudflare: + apiTokenSecretRef: + name: cloudflare-token + key: api-token + --- + apiVersion: cert-manager.io/v1 + kind: Certificate + metadata: + name: wildcard-domain + spec: + secretName: wildcard-domain + issuerRef: + name: letsencrypt-issuer + kind: Issuer + group: cert-manager.io + commonName: "*.DOMAIN.GOES.HERE" + dnsNames: + - "DOMAIN.GOES.HERE" + - "*.DOMAIN.GOES.HERE" + --- + apiVersion: traefik.containo.us/v1alpha1 + kind: TLSStore + metadata: + name: default + spec: + certificates: + - secretName: wildcard-domain + defaultCertificate: + secretName: wildcard-domain + +You'll need to create a cloudflare API key with permissions to Edit zone DNS. Once you've replaced all the values inside ``certs.yml`` (email, domain, api token), run the following command to create the resources: + +.. code-block:: bash + + kubectl apply -f certs.yml + +You should now be good to go! + +To deploy your challenges, run the following command to load your environment variables and deploy your challenges: + +.. code-block:: bash + + env $(cat .env) rcds deploy From 07e404e2b3413ed7df94e4dcbebfae84f6d2214b Mon Sep 17 00:00:00 2001 From: stuxf <70670632+stuxf@users.noreply.github.com> Date: Tue, 22 Aug 2023 00:22:06 -0700 Subject: [PATCH 2/3] Update quickstart.rst --- docs/quickstart.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/quickstart.rst b/docs/quickstart.rst index 3c9f975..d685277 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -63,11 +63,11 @@ Additionally, create a ``.env`` file. It should look like the following: RCDS_RCTF_URL=https://ctf.example.com RCDS_RCTF_TOKEN=999omGulJ8OUxy+NNMmfV4VbErhHf5HTxRU07FKFdDYQmEGworLsxl2G6Hdl6BgrkYvhfAZoR0IEdE0XXlurGB1szIjdIk1whr3iSP2ZIdAC7chSDlk9SL/iN68J -You can obtain the token by creating an admin account (`Instructions here `_), copying the token from the admin page, and then url decoding it. +You can obtain the token by creating an admin account (`Instructions here `_). Once you have created the admin account, click on the profile page, and grab the token from the "Copy Link" button on the top right. Make sure to URL decode the token before putting it in the ``.env`` file. -From google cloud, create a standard kubernetes cluster. Make sure that it is NOT an autopilot cluster, which google cloud will try to default to. If doing this through the UI, there should be an option on the top right to switch to a standard cluster. +From google cloud, create a standard kubernetes cluster. Make sure that it is NOT an autopilot cluster, which google cloud will try to default to. Autopilot clusters in google cloud have limited permissions, and you will almost certainly need more permissions, so selecting the standard cluster is a must. If doing this through the UI, there should be an option on the top right to switch to a standard cluster. -Enable Dataplane V2 if it is not already enabled by default. Additionally, under the network settings tab, create a new tag called ``open-nodeports``. We will be using this later to configure the firewall. +Enable Dataplane V2 by clicking the "Dataplane V2" checkbox under clusters -> networking if it is not already enabled by default. Additionally, under the network settings tab, create a new tag called ``open-nodeports``. We will be using this later to configure the firewall. Once the kubernetes cluster is created, click the connect button on the top bar of the cluster page and connect to the cluster using the command provided. This will set up your local kubectl configuration to connect to the cluster. @@ -75,7 +75,13 @@ Additionally, we will need to create a container registry to store our docker im Once the registry is created, we will need to configure docker to be able to push to this registry. To do this, right click on the registry, and click setup instructions on google cloud. This will give you a command to run to configure docker to push to this registry. Run this command on your local machine. -Make sure to configure your ``rcds.yaml`` file to match the name of your container registry. +Make sure to configure your ``rcds.yaml`` file to match the name of your container registry you created. Your docker image prefix should look something like this: ``us-central1-docker.pkg.dev/amateursctf/ctf-docker-test``. Typically it will be in the format ``REGION-NAME-docker.pkg.dev/PROJECT-NAME/REPOSITORY-NAME``. You can find it by clicking on your project from the Artifact Registry page, and clicking on the copy icon from the top left hand side of the page. + +.. code-block:: yaml + + docker: + image: + prefix: us-central1-docker.pkg.dev/amateursctf/ctf-docker-test Once that's done, go to the VPC Network tab of google cloud, and assign a static IP address to one of the nodes in your cluster. This will be the IP address that your challenges will be hosted on, so configure DNS to point to this IP address. @@ -143,7 +149,7 @@ Finally, we're going to configure the automatic TLS certificate generation. To d defaultCertificate: secretName: wildcard-domain -You'll need to create a cloudflare API key with permissions to Edit zone DNS. Once you've replaced all the values inside ``certs.yml`` (email, domain, api token), run the following command to create the resources: +You'll need to create a cloudflare API key with permissions to Edit zone DNS. For more information on how to create an API key, check out the `Cloudflare documentation `_.Once you've replaced all the values inside ``certs.yml`` (email, domain, api token), run the following command to create the resources: .. code-block:: bash From ceb876459fd5f8960d46f6968ed70d9632c9e52f Mon Sep 17 00:00:00 2001 From: stuxf <70670632+stuxf@users.noreply.github.com> Date: Fri, 22 Dec 2023 07:04:36 +0000 Subject: [PATCH 3/3] add some fixes --- docs/quickstart.rst | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/quickstart.rst b/docs/quickstart.rst index d685277..e421a04 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -65,13 +65,13 @@ Additionally, create a ``.env`` file. It should look like the following: You can obtain the token by creating an admin account (`Instructions here `_). Once you have created the admin account, click on the profile page, and grab the token from the "Copy Link" button on the top right. Make sure to URL decode the token before putting it in the ``.env`` file. -From google cloud, create a standard kubernetes cluster. Make sure that it is NOT an autopilot cluster, which google cloud will try to default to. Autopilot clusters in google cloud have limited permissions, and you will almost certainly need more permissions, so selecting the standard cluster is a must. If doing this through the UI, there should be an option on the top right to switch to a standard cluster. +From google cloud, create a standard kubernetes cluster. Make sure that it is NOT an autopilot cluster, which google cloud will try to default to. Autopilot clusters in google cloud have limited permissions, and you will almost certainly need more permissions, so selecting the standard cluster is a must. If doing this through the UI, there should be an option on the top right to switch to a standard cluster. Additionally, while creating the cluster, set the boot disk size of each individual node (Node Pools -> default-pool -> node) to 50gb instead of 100gb. -Enable Dataplane V2 by clicking the "Dataplane V2" checkbox under clusters -> networking if it is not already enabled by default. Additionally, under the network settings tab, create a new tag called ``open-nodeports``. We will be using this later to configure the firewall. +Enable Dataplane V2 by clicking the "Dataplane V2" checkbox under clusters -> networking if it is not already enabled by default. Additionally, under the network settings tab (Node Pools -> default-pool -> Networking), create a new tag called ``open-nodeports``. We will be using this later to configure the firewall. Once the kubernetes cluster is created, click the connect button on the top bar of the cluster page and connect to the cluster using the command provided. This will set up your local kubectl configuration to connect to the cluster. -Additionally, we will need to create a container registry to store our docker images. To do this, go to the container registry page, and create a new registry. Make sure to select the same region as your kubernetes cluster. +Additionally, we will need to create a container registry to store our docker images. To do this, go to the artifact registry (note that container is being deprecated, so we use artifact instead) page, and create a new registry. Make sure to select the same region as your kubernetes cluster. Once the registry is created, we will need to configure docker to be able to push to this registry. To do this, right click on the registry, and click setup instructions on google cloud. This will give you a command to run to configure docker to push to this registry. Run this command on your local machine. @@ -85,7 +85,7 @@ Make sure to configure your ``rcds.yaml`` file to match the name of your contain Once that's done, go to the VPC Network tab of google cloud, and assign a static IP address to one of the nodes in your cluster. This will be the IP address that your challenges will be hosted on, so configure DNS to point to this IP address. -Additionally, configure firewall rules in the VPC Network tab to allow traffic for the allow-nodeports rule that we created earlier. +Additionally, configure firewall rules in the VPC Network tab to allow all traffic for the allow-nodeports rule that we created earlier. Once done, install the following helm charts onto your cluster: @@ -93,7 +93,7 @@ Once done, install the following helm charts onto your cluster: - **Cert-Manager**: For installing Cert-Manager, ensure that you install CRDs using the second option: `Install Cert-Manager with CRDs using Helm `_ -Once done, go to the VPC Network IP addresses tab, and convert the traefik IP address to a static IP address. This will be the IP address that your web challenges will be hosted on, so add a wildcard DNS entry to point to this IP address. For example, add an A record pointing at ``*.example.com``. +Once done, go to the VPC Network IP addresses tab, and convert the traefik IP address to a static IP address. It should be marked there as traefik, and if not can be found by using kubectl. This will be the IP address that your web challenges will be hosted on, so add a wildcard DNS entry to point to this IP address. For example, add an A record pointing at ``*.example.com``. Finally, we're going to configure the automatic TLS certificate generation. To do this, fill out the following template and name it ``certs.yml``: @@ -116,13 +116,13 @@ Finally, we're going to configure the automatic TLS certificate generation. To d email: "EMAIL@GOES-HERE" server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: - name: letsencrypt-issuer-key + name: letsencrypt-issuer-key solvers: - - dns01: - cloudflare: - apiTokenSecretRef: - name: cloudflare-token - key: api-token + - dns01: + cloudflare: + apiTokenSecretRef: + name: cloudflare-token + key: api-token --- apiVersion: cert-manager.io/v1 kind: Certificate