Skip to content

Commit

Permalink
add pre-commit hooks, golangci-lint config
Browse files Browse the repository at this point in the history
  • Loading branch information
mluffman committed Dec 21, 2023
1 parent abb9b66 commit 9fb986d
Show file tree
Hide file tree
Showing 22 changed files with 123 additions and 42 deletions.
51 changes: 51 additions & 0 deletions .golangci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
run:
timeout: 5m
skip-dirs:
- test_data
- mocks

issues:
exclude-rules:
- path: _test\.go
linters:
- errcheck
- dupl
- gocritic

linters:
disable-all: true
enable:
- asciicheck
- bidichk
- bodyclose
- dogsled
- dupl
- exportloopref
- gci
- gocritic
- gofmt
- gosimple
- govet
- ineffassign
- misspell
- nilerr
- prealloc
- revive
- staticcheck
- tenv
- thelper
- unconvert
- unused
- wastedassign
- whitespace
- errcheck
- exportloopref

linters-settings:
govet:
enable-all: true
gci:
sections:
- standard # Captures all standard packages if they do not match another section.
- default # Contains all imports that could not be matched to another section type.
- prefix(github.com/thought-machine/please)
15 changes: 15 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v2.3.0
hooks:
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
- repo: https://github.com/golangci/golangci-lint
rev: v1.55.2
hooks:
- id: golangci-lint
- repo: https://github.com/thought-machine/spot-interruption-exporter
rev: master
hooks:
- id: gci-imports
6 changes: 6 additions & 0 deletions .pre-commit-hooks.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
- id: gci-imports
name: GCI Imports
description: Runs GCI (https://github.com/daixiang0/gci) on all files
entry: gci write .
language: golang
types: [go]
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Publishes a prometheus metric `interruption_events_total` that increments by 1 whenever a spot instance has been preempted.

This is a very helpful metric, as it
This is a very helpful metric, as it

- helps correlate workload issues with spot interruption times

Expand All @@ -18,9 +18,9 @@ A single deployment of the infrastructure and app is intended to serve all Kuber

## How it works

Spot preemption events are emitted as an audit log that contain the compute instance ID. These audit logs are forwarded to a pubsub topic via GCP Log Sink. The app then subscribes to this topic and handles the interruption event.
Spot preemption events are emitted as an audit log that contain the compute instance ID. These audit logs are forwarded to a pubsub topic via GCP Log Sink. The app then subscribes to this topic and handles the interruption event.

The audit log for instance preemption does not contain information about the Kubernetes cluster the instance may or may not have been associated with. Since the node is already deleted by the time the preemption event is received, the compute API cannot be queried for more information.
The audit log for instance preemption does not contain information about the Kubernetes cluster the instance may or may not have been associated with. Since the node is already deleted by the time the preemption event is received, the compute API cannot be queried for more information.

To work around this, the app keeps a mapping of compute instance ID to Kubernetes cluster. It can then use this when processing preemption events to publish the correct `kubernetes_cluster` label on the metric.

Expand Down Expand Up @@ -77,7 +77,7 @@ $ gcloud pubsub topics publish sie-interruption-topic --project <project> --mess
}'
```

You can send a test instance creation message via
You can send a test instance creation message via
```bash
$ gcloud pubsub topics publish sie-creation-topic --project <project> --message '{
"protoPayload": {
Expand Down
3 changes: 2 additions & 1 deletion config.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
package main

import (
"gopkg.in/yaml.v3"
"os"

"gopkg.in/yaml.v3"
)

type PrometheusConfig struct {
Expand Down
5 changes: 2 additions & 3 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,20 @@ module github.com/thought-machine/spot-interruption-exporter
go 1.20

require (
cloud.google.com/go/compute v1.20.1
cloud.google.com/go/pubsub v1.33.0
github.com/googleapis/google-cloudevents-go v0.7.0
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/prometheus/client_golang v1.16.0
github.com/stretchr/testify v1.8.3
go.uber.org/mock v0.3.0
go.uber.org/zap v1.26.0
google.golang.org/api v0.126.0
google.golang.org/protobuf v1.31.0
gopkg.in/yaml.v3 v3.0.1
)

require (
cloud.google.com/go v0.110.4 // indirect
cloud.google.com/go/compute v1.20.1 // indirect
cloud.google.com/go/compute/metadata v0.2.3 // indirect
cloud.google.com/go/iam v1.1.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
Expand All @@ -43,7 +43,6 @@ require (
golang.org/x/sync v0.2.0 // indirect
golang.org/x/sys v0.8.0 // indirect
golang.org/x/text v0.9.0 // indirect
google.golang.org/api v0.126.0 // indirect
google.golang.org/appengine v1.6.7 // indirect
google.golang.org/genproto v0.0.0-20230706204954-ccb25ca9f130 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20230629202037-9506855d4529 // indirect
Expand Down
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,6 @@ go.opencensus.io v0.24.0 h1:y73uSU6J157QMP2kn2r30vwW1A2W2WFwSCGnAVxeaD0=
go.opencensus.io v0.24.0/go.mod h1:vNK8G9p7aAivkbmorf4v+7Hgx+Zs0yY+0fOtgBfjQKo=
go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI=
go.uber.org/goleak v1.2.0 h1:xqgm/S+aQvhWFTtR0XK3Jvg7z8kGV8P4X14IzwN3Eqk=
go.uber.org/mock v0.3.0 h1:3mUxI1No2/60yUYax92Pt8eNOEecx2D3lcXZh2NEZJo=
go.uber.org/mock v0.3.0/go.mod h1:a6FSlNadKUHUa9IP5Vyt1zh4fC7uAwxMutEAscFbkZc=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.26.0 h1:sI7k6L95XOKS281NhVKOFCUNIvv9e0w4BF8N3u+tCRo=
Expand Down
1 change: 0 additions & 1 deletion infra/gcp/event-forwarder/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,3 @@ resource "google_pubsub_subscription" "subscription" {
message_retention_duration = "600s"
retain_acked_messages = false
}

4 changes: 2 additions & 2 deletions internal/cache/cache_test.go
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
package cache

import (
"github.com/stretchr/testify/suite"
"testing"
"time"

"github.com/stretchr/testify/suite"
)

type CacheTestSuite struct {
suite.Suite
sut Cache
}

func TestCacheTestSuite(t *testing.T) {
Expand Down
7 changes: 4 additions & 3 deletions internal/compute/compute.go
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
package compute

import (
compute "cloud.google.com/go/compute/apiv1"
"cloud.google.com/go/compute/apiv1/computepb"
"context"
"errors"
"fmt"
"strings"

compute "cloud.google.com/go/compute/apiv1"
"cloud.google.com/go/compute/apiv1/computepb"
"go.uber.org/zap"
"google.golang.org/api/iterator"
"strings"
)

var (
Expand Down
3 changes: 2 additions & 1 deletion internal/compute/compute_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,10 @@ package compute

import (
"context"
"testing"

"github.com/stretchr/testify/suite"
"go.uber.org/zap"
"testing"
)

type ComputeTestSuite struct {
Expand Down
3 changes: 2 additions & 1 deletion internal/events/pubsub.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
package events

import (
gcppubsub "cloud.google.com/go/pubsub"
"context"

gcppubsub "cloud.google.com/go/pubsub"
"go.uber.org/zap"
)

Expand Down
8 changes: 4 additions & 4 deletions internal/handlers/handlers.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,17 @@
package handlers

import (
"strings"
"sync"
"time"

gcppubsub "cloud.google.com/go/pubsub"
"github.com/googleapis/google-cloudevents-go/cloud/auditdata"
"github.com/thought-machine/spot-interruption-exporter/internal/cache"
"github.com/thought-machine/spot-interruption-exporter/internal/compute"
"github.com/thought-machine/spot-interruption-exporter/internal/metrics"
"go.uber.org/zap"
"google.golang.org/protobuf/encoding/protojson"
"strings"
"sync"
"time"
)

type instanceInterruptionEvent struct {
Expand Down Expand Up @@ -105,5 +106,4 @@ func messageToInstanceCreationEvent(m *gcppubsub.Message) (instanceCreationEvent
ResourceID: entry.ProtoPayload.ResourceName,
ClusterName: clusterName,
}, nil

}
6 changes: 3 additions & 3 deletions internal/handlers/handlers_test.go
Original file line number Diff line number Diff line change
@@ -1,14 +1,15 @@
package handlers

import (
"sync"
"testing"

gcppubsub "cloud.google.com/go/pubsub"
"github.com/stretchr/testify/suite"
"github.com/thought-machine/spot-interruption-exporter/internal/cache"
"github.com/thought-machine/spot-interruption-exporter/internal/handlers/test_data"
"github.com/thought-machine/spot-interruption-exporter/internal/metrics/mocks"
"go.uber.org/zap"
"sync"
"testing"
)

type HandlersTestSuite struct {
Expand Down Expand Up @@ -89,7 +90,6 @@ func (suite *HandlersTestSuite) TestMessageToInstanceInterruptionEvent() {
}

func (suite *HandlersTestSuite) TestMessageToInstanceCreationEvent() {

event, err := messageToInstanceCreationEvent(mockCreationMessage)
suite.NoError(err)
suite.Equal("projects/123456789/zones/europe-west1-c/instances/fake-resource", event.ResourceID)
Expand Down
2 changes: 1 addition & 1 deletion internal/handlers/test_data/creation-event.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@
]
}
}
}
}
2 changes: 1 addition & 1 deletion internal/handlers/test_data/interruption-event.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
"methodName": "compute.instances.preempted",
"resourceName": "projects/mock-project/zones/europe-west1-c/instances/mock-instance-spot-3706-5b909138-nr65"
}
}
}
5 changes: 3 additions & 2 deletions internal/metrics/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,13 @@ package metrics

import (
"fmt"
"log"
"net/http"

"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
"github.com/prometheus/client_golang/prometheus/promhttp"
"go.uber.org/zap"
"log"
"net/http"
)

var (
Expand Down
2 changes: 1 addition & 1 deletion kustomize/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ pubsub:
instance_interruption_subscription_name: sie-interruption-subscription
prometheus:
port: 8090
path: /metrics
path: /metrics
2 changes: 1 addition & 1 deletion kustomize/example-overlay/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ pubsub:
instance_interruption_subscription_name: sie-interruption-subscription
prometheus:
port: 1111
path: /bespoke-metrics-path
path: /bespoke-metrics-path
2 changes: 1 addition & 1 deletion kustomize/example-overlay/kustomization.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,4 @@ patches:
- path: sa-patch.yaml

resources:
- [email protected]:thought-machine/spot-interruption-exporter.git//kustomize
- [email protected]:thought-machine/spot-interruption-exporter.git//kustomize
2 changes: 1 addition & 1 deletion kustomize/service.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ spec:
ports:
- name: metrics
port: 8090
targetPort: metrics
targetPort: metrics
Loading

0 comments on commit 9fb986d

Please sign in to comment.