Skip to content

Commit

Permalink
fixes minor improvements
Browse files Browse the repository at this point in the history
  • Loading branch information
anik3tra0 committed Jan 30, 2024
1 parent 272d8e0 commit 19deaca
Show file tree
Hide file tree
Showing 5 changed files with 71 additions and 40 deletions.
10 changes: 5 additions & 5 deletions efs_mount.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,21 +45,21 @@ To mount an EFS file system on an EC2 instance, the instance must have an NFS cl

1. **Create a Mount Point**:
```bash
sudo mkdir -p $HOME/efs/mnt
sudo mkdir -p /efs/mnt
```

2. **Mount the EFS File System**:
- You can find the DNS name of your EFS file system in the AWS Management Console (under the EFS section).
- Mount the EFS using:
```bash
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 {EFS-DNS-Name}:/ $HOME/efs/mnt
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 {EFS-DNS-Name}:/ /efs/mnt
```

3. **Automatic Mount on Reboot** (Optional):
- Edit `/etc/fstab` to add an entry for the EFS to automatically mount it on system reboots.
- Add the following line:
```
{EFS-DNS-Name}:/ $HOME/efs/mnt nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 0 0
{EFS-DNS-Name}:/ /efs/mnt nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 0 0
```

### Step 5: Verify the Mount
Expand All @@ -72,13 +72,13 @@ To mount an EFS file system on an EC2 instance, the instance must have an NFS cl
### Step 6: Create a directory for vmagent configs

```bash
mkdir -p $HOME/efs/mnt/vmagent-cfgs
mkdir -p /efs/mnt/vmagent-cfgs
```

### Step 7: Mount the [vmagent](vmagent.yaml) config

```bash
scp ./vmagent.yaml username@ec2_host_ip:~/efs/mnt/vmagent-cfgs
scp ./vmagent.yaml username@ec2_host_ip:/efs/mnt/vmagent-cfgs
```

### Additional Considerations
Expand Down
8 changes: 4 additions & 4 deletions efs_unmount.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,18 +6,18 @@ Unmounting an Amazon Elastic File System (EFS) from a host, such as an EC2 insta
```

```bash
sudo lsof $HOME/efs/mnt
sudo lsof /efs/mnt
```

Replace `$HOME/efs/mnt` with the actual mount point of your EFS file system. If this command outputs any processes, you should stop them before proceeding.
Replace `/efs/mnt` with the actual mount point of your EFS file system. If this command outputs any processes, you should stop them before proceeding.

2. **Unmount the File System**: Use the `umount` command to unmount the EFS file system.

```bash
sudo umount $HOME/efs/mnt
sudo umount /efs/mnt
```

Again, replace `$HOME/efs/mnt` with your EFS mount point. If the file system is busy (perhaps due to some processes still using it), you might see an error. In that case, ensure all processes using the file system are stopped.
Again, replace `/efs/mnt` with your EFS mount point. If the file system is busy (perhaps due to some processes still using it), you might see an error. In that case, ensure all processes using the file system are stopped.

3. **Verify the File System is Unmounted**: You can verify that the file system has been unmounted successfully by using the `df` command.

Expand Down
65 changes: 37 additions & 28 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ data "aws_vpc" "aws_vpc" {
id = var.vpc_id
}

resource "aws_security_group" "vmagent_sg" {
resource "aws_security_group" "last9_vmagent_sg" {
vpc_id = data.aws_vpc.aws_vpc.id

egress {
Expand All @@ -17,20 +17,22 @@ resource "aws_security_group" "vmagent_sg" {
}

tags = {
Name = "vmagent-security-group"
Name = "last9-vmagent-security-group"
last9_enabled = true
}
}

resource "aws_efs_file_system" "vmagent_efs" {
resource "aws_efs_file_system" "last9_vmagent_efs" {
creation_token = "vmagent-efs"

tags = {
Name = "VMagent EFS"
Name = "VMagent EFS"
last9_enabled = true
}

}

resource "aws_security_group" "vmagent_efs_sg" {
resource "aws_security_group" "last9_vmagent_efs_sg" {
name = "vmagent-efs-security-group"
description = "Security group for EFS"
vpc_id = data.aws_vpc.aws_vpc.id
Expand All @@ -50,28 +52,30 @@ resource "aws_security_group" "vmagent_efs_sg" {
}

tags = {
Name = "EFS Security Group"
Name = "EFS Security Group"
last9_enabled = true
}
}

resource "aws_efs_mount_target" "vmagent_efs_mt" {
resource "aws_efs_mount_target" "last9_vmagent_efs_mt" {
for_each = toset(var.subnet_ids)
file_system_id = aws_efs_file_system.vmagent_efs.id
file_system_id = aws_efs_file_system.last9_vmagent_efs.id
subnet_id = each.value
security_groups = [aws_security_group.vmagent_efs_sg.id]
security_groups = [aws_security_group.last9_vmagent_efs_sg.id]
}

resource "aws_security_group_rule" "ecs_to_efs" {
type = "egress"
from_port = 2049
to_port = 2049
protocol = "tcp"
security_group_id = aws_security_group.vmagent_sg.id
source_security_group_id = aws_security_group.vmagent_efs_sg.id
security_group_id = aws_security_group.last9_vmagent_sg.id
source_security_group_id = aws_security_group.last9_vmagent_efs_sg.id
}

resource "aws_cloudwatch_log_group" "vmagent_ecs_log_group" {
name = "/ecs/${var.environment}/vmagent"
resource "aws_cloudwatch_log_group" "last9_vmagent_ecs_log_group" {
name = "/ecs/${var.environment}/vmagent"
retention_in_days = 5
}

resource "aws_iam_role" "ecs_execution_role" {
Expand Down Expand Up @@ -133,7 +137,7 @@ resource "aws_iam_policy" "efs_access_policy" {
"elasticfilesystem:DescribeAccessPoints",
"elasticfilesystem:DescribeMountTargets"
],
Resource = aws_efs_file_system.vmagent_efs.arn
Resource = aws_efs_file_system.last9_vmagent_efs.arn
}
]
})
Expand All @@ -144,27 +148,28 @@ resource "aws_iam_role_policy_attachment" "ecs_execution_role_efs_attachment" {
policy_arn = aws_iam_policy.efs_access_policy.arn
}

resource "aws_ecs_service" "vmagent-service" {
name = "vmagent-ecs-service"
resource "aws_ecs_service" "last9-vmagent-service" {
name = "last9-vmagent-ecs-service"
cluster = var.ecs_cluster_id
task_definition = aws_ecs_task_definition.vmagent_task_definition.arn
task_definition = aws_ecs_task_definition.last9_vmagent_task_definition.arn
launch_type = "FARGATE"
desired_count = 1

network_configuration {
subnets = var.subnet_ids
security_groups = [aws_security_group.vmagent_sg.id]
security_groups = [aws_security_group.last9_vmagent_sg.id]
assign_public_ip = true # Check this and configure as required
}

tags = {
Name = "VMAgent"
Environment = var.environment
Name = "last9-vmagent"
Environment = var.environment
last9_enabled = true
}
}

resource "aws_ecs_task_definition" "vmagent_task_definition" {
family = "vmagent"
resource "aws_ecs_task_definition" "last9_vmagent_task_definition" {
family = "last9-vmagent"
network_mode = "awsvpc"
requires_compatibilities = ["FARGATE"]
cpu = "2048" # adjust based on your application requirements
Expand All @@ -173,9 +178,9 @@ resource "aws_ecs_task_definition" "vmagent_task_definition" {
execution_role_arn = aws_iam_role.ecs_execution_role.arn

volume {
name = "vmagent-volume"
name = "last9-vmagent-volume"
efs_volume_configuration {
file_system_id = aws_efs_file_system.vmagent_efs.id
file_system_id = aws_efs_file_system.last9_vmagent_efs.id
root_directory = "/"
transit_encryption = "ENABLED"
transit_encryption_port = 2049
Expand All @@ -184,7 +189,7 @@ resource "aws_ecs_task_definition" "vmagent_task_definition" {

container_definitions = jsonencode([
{
name = "vmagent-scraper"
name = "last9-vmagent-scraper"
image = "victoriametrics/vmagent:latest"
essential = true

Expand All @@ -206,7 +211,7 @@ resource "aws_ecs_task_definition" "vmagent_task_definition" {
logConfiguration = {
logDriver = "awslogs",
options = {
"awslogs-group" = aws_cloudwatch_log_group.vmagent_ecs_log_group.name
"awslogs-group" = aws_cloudwatch_log_group.last9_vmagent_ecs_log_group.name
"awslogs-region" = var.aws_region,
"awslogs-stream-prefix" = "-vmagent",
}
Expand All @@ -221,13 +226,12 @@ resource "aws_ecs_task_definition" "vmagent_task_definition" {

mountPoints = [
{
sourceVolume = "vmagent-volume",
sourceVolume = "last9-vmagent-volume",
containerPath = var.container_mount_path,
readOnly = false,
}
]


environment = [
{
name = "ENVIRONMENT",
Expand All @@ -236,4 +240,9 @@ resource "aws_ecs_task_definition" "vmagent_task_definition" {
]
}
])

tags = {
last9_enabled = true
Environment = var.environment
}
}
2 changes: 1 addition & 1 deletion outputs.tf
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
output "ecs_service_name" {
description = "Name of the ECS Service"
value = aws_ecs_service.vmagent-service.name
value = aws_ecs_service.last9-vmagent-service.name
}

# Add other outputs as necessary
26 changes: 24 additions & 2 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@

## Overview

This Terraform module is designed to deploy a Docker container as an AWS ECS Fargate task with an attached Amazon EFS filesystem. It allows for customized configurations, including setting up environment variables and remote write credentials for Levitate.
This Terraform module is designed to deploy a Docker container as an AWS ECS Fargate task with an attached Amazon EFS
filesystem. It allows for customized configurations, including setting up environment variables and remote write
credentials for VictoriaMetrics (or similar tools).

## Prerequisites

Expand Down Expand Up @@ -57,10 +59,30 @@ Replace the values with your specific configuration details.

## Notes

- Ensure that the provided AWS credentials have the necessary permissions for creating and managing ECS and EFS resources.
- Ensure that the provided AWS credentials have the necessary permissions for creating and managing ECS and EFS
resources.
- Review and adjust security group rules as per your organization's security policies.
- Validate your Terraform configurations with `terraform plan` before applying changes with `terraform apply`.

### How to mount/unmount configuration files to the created EFS

- Refer [efs_mount.md](efs_mount.md)
- Refer [efs_unmount.md](efs_unmount.md)

## Disclaimer

Before proceeding with state management in Terraform, it is essential to adhere to your organization's established best
practices and guidelines for managing Terraform state. If your organization has specific state management practices in
place, please follow them diligently.

However, in cases where your organization does not have established Terraform state management practices, or if you are
working on a one-time operation, you may consider checking in the state files directly into your Git repository, rather
than implementing a dedicated remote state backend like Amazon S3 or other storage solutions.

Please exercise caution and consider the security and access control implications of storing state files in Git. This
approach may not be suitable for all scenarios, and it is crucial to assess the potential risks and benefits before
making a decision.

Ultimately, the choice of state management strategy should align with your project's specific requirements and your
organization's policies. If in doubt or if security and compliance concerns arise, it is advisable to consult with your
organization's infrastructure or security team for guidance on the best approach to Terraform state management.

0 comments on commit 19deaca

Please sign in to comment.