From 8573cb4ddc0092eb8012c43b877e11b913da1613 Mon Sep 17 00:00:00 2001 From: Adam Gordon Bell Date: Mon, 13 May 2024 16:22:51 -0400 Subject: [PATCH] Article Updates (#836) * fix github links to earthly org * WIP * repo change * WIP * WIP * updating GRPC proxy article * updating v6 * WIP * lint --- blog/_data/navigation.yml | 2 +- blog/_posts/2021-12-20-golang-http.md | 20 +++++----- blog/_posts/2022-01-11-golang-command-line.md | 22 +++++------ blog/_posts/2022-02-03-golang-sqlite.md | 27 ++++++++------ blog/_posts/2022-02-16-golang-grpc-example.md | 14 +++---- blog/_posts/2022-03-17-golang-grpc-gateway.md | 37 ++++++++++++++----- blog/_posts/2022-03-18-buf-protobuf.md | 8 ++-- blog/_posts/2022-05-04-aws-lambda-golang.md | 2 +- .../_posts/2022-06-13-aws-lambda-api-proxy.md | 8 ++-- blog/_posts/2022-07-20-terraform-lambda.md | 2 +- blog/_posts/2022-08-02-terraform-route53.md | 2 +- 11 files changed, 83 insertions(+), 61 deletions(-) diff --git a/blog/_data/navigation.yml b/blog/_data/navigation.yml index 7041bf79a..b3d27584c 100644 --- a/blog/_data/navigation.yml +++ b/blog/_data/navigation.yml @@ -138,7 +138,7 @@ activity-tracker: url: /golang-sqlite/ - title: "Part 4: gRPC" url: /golang-grpc-example/ - - title: "Part 4: gRPC Gateway" + - title: "Part 5: gRPC Gateway" url: /golang-grpc-gateway/ - title: "Side Quest: Protobuf's" url: /buf-protobuf/ diff --git a/blog/_posts/2021-12-20-golang-http.md b/blog/_posts/2021-12-20-golang-http.md index 6bbfb3fc8..4f8c6a8f5 100644 --- a/blog/_posts/2021-12-20-golang-http.md +++ b/blog/_posts/2021-12-20-golang-http.md @@ -12,7 +12,7 @@ internal-links: - golang http excerpt: | Learn how to build a JSON HTTP server using Golang in this tutorial. Discover the basics of creating a Golang web service, handling HTTP requests, and working with JSON data. -last_modified_at: 2023-09-19 +last_modified_at: 2024-04-07 --- **This article explains Golang JSON services. Earthly simplifies the build and test processes for Go web services. [Learn more](https://cloud.earthly.dev/login).** @@ -190,7 +190,7 @@ test:
Test my containerized service
-What I've built up so far is on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/WebServer), but it doesn't do much. For my activity tracker to be useful, it will need to understand and store activities. +What I've built up so far is on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/WebServer), but it doesn't do much. For my activity tracker to be useful, it will need to understand and store activities. So let's move on to my activity data structures. @@ -228,7 +228,7 @@ Doing that, I can write an insert function like this: ~~~{.go captionb="internal/server/activity.go"} func (c *Activities) Insert(activity Activity) uint64 { - activity.ID = uint64(len(c.activities)) + activity.ID = uint64(len(c.activities)) + 1 c.activities = append(c.activities, activity) return activity.ID } @@ -238,10 +238,10 @@ And retrieve is super simple as well: ~~~{.go captionb="internal/server/activity.go"} func (c *Activities) Retrieve(id uint64) (Activity, error) { - if id >= uint64(len(c.activities)) { + if id > uint64(len(c.activities)) { return Activity{}, ErrIDNotFound } - return c.activities[id], nil + return c.activities[id-1], nil } ~~~ @@ -360,7 +360,7 @@ We now have half our API working! ~~~{.bash caption=">_"} > curl -X POST localhost:8080 -d \ '{"activity": {"description": "christmas eve class", - time:"2021-12-24T12:42:31Z"}}' + "time":"2021-12-24T12:42:31Z"}}' ~~~ ~~~{.merge-code} @@ -428,7 +428,7 @@ curl -X POST localhost:8080 -d \ class","id":15} ~~~ -the whole thing, including some edge cases I left out is on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/ActivityLog). +the whole thing, including some edge cases I left out is on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/ActivityLog). If fact, I can now update my shell script `test.sh` to exercise these endpoints. @@ -467,12 +467,12 @@ And now, since I wrote that `Earthfile` that starts up the service and runs `tes
{% picture content-wide-nocrop {{site.pimages}}{{page.slug}}/1010.png --alt {{ GitHub Actions }} %} -
Passing End to End tests on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/ActivityLog)
+
Passing End to End tests on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/ActivityLog)
## That's a Wrap -There we go. I have a working service that I've put up on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/ActivityLog) with an active CI process. It doesn't persist its data, it doesn't allow me to access my activity log in any other way than by id, and it doesn't have a UI, but I'm starting to get a feel for how web services are built in Golang, which was the point. +There we go. I have a working service that I've put up on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/ActivityLog) with an active CI process. It doesn't persist its data, it doesn't allow me to access my activity log in any other way than by id, and it doesn't have a UI, but I'm starting to get a feel for how web services are built in Golang, which was the point. As an activity tracker, what I have so far is pretty weak. But as a learning lesson, I've found it valuable. @@ -484,7 +484,7 @@ Now I just have to start being active! Maybe building a command line client for ### Linting -In the first version of this example I used `Id` everywhere instead of `ID`, which is incorrect capitalization (per `go lint` and [Alex](/blog/authors/Alex/)). To prevent further style issues like this as I continue building this application I'm linting my code going forward using [`golangci-lint`](https://golangci-lint.run/) which with the [right configuration](https://github.com/adamgordonbell/cloudservices/blob/v1-http-server/ActivityLog/.golangci.yml) calls several go linters, including `go lint`. +In the first version of this example I used `Id` everywhere instead of `ID`, which is incorrect capitalization (per `go lint` and [Alex](/blog/authors/Alex/)). To prevent further style issues like this as I continue building this application I'm linting my code going forward using [`golangci-lint`](https://golangci-lint.run/) which with the [right configuration](https://github.com/earthly/cloud-services-example/blob/v1-http-server/ActivityLog/.golangci.yml) calls several go linters, including `go lint`. ### Errors diff --git a/blog/_posts/2022-01-11-golang-command-line.md b/blog/_posts/2022-01-11-golang-command-line.md index 5bb92b929..92c23bf57 100644 --- a/blog/_posts/2022-01-11-golang-command-line.md +++ b/blog/_posts/2022-01-11-golang-command-line.md @@ -13,7 +13,7 @@ topic: cli funnel: 2 excerpt: | Learn how to build a command-line JSON client in Golang to interact with a REST service for storing workout activities. The article covers topics such as parsing command-line flags, making HTTP requests, handling errors, and testing the client. -last_modified_at: 2023-09-19 +last_modified_at: 2024-04-07 --- **This article is about Golang activity tracking. Earthly can streamline your build process. [Check it out](https://cloud.earthly.dev/login).** @@ -40,7 +40,7 @@ The existing backend doesn't support `list` yet, so we will skip that one for no First, I create a new folder for my client: ~~~{.bash caption=">_"} -$ go mod init github.com/adamgordonbell/cloudservices/activityclient +$ go mod init github.com/earthly/cloud-services-example/activityclient ~~~ ## Command Line Flags @@ -334,7 +334,7 @@ package client import ( ... - api "github.com/adamgordonbell/cloudservices/activity-log" + api "github.com/earthly/cloud-services-example/activity-log" ) ~~~ @@ -489,7 +489,7 @@ Then I just need to `json.Unmarshall` my activity document: return document.Activity, nil ~~~ -And with that, I have a [working](https://github.com/adamgordonbell/cloudservices/tree/v2-cli/activity-client), though basic, client. So I'm going to add some light testing and then call it a day. +And with that, I have a [working](https://github.com/earthly/cloud-services-example/tree/v2-cli/activity-client), though basic, client. So I'm going to add some light testing and then call it a day. ## Testing the Happy Path @@ -512,7 +512,7 @@ Assuming the backend service is up, and the client is built, this will test that ## Continuous Integration -I can quickly hook this happy path up to CI by extending my previous [Earthfile](https://github.com/adamgordonbell/cloudservices/blob/v2-cli/Earthfile). +I can quickly hook this happy path up to CI by extending my previous [Earthfile](https://github.com/earthly/cloud-services-example/blob/v2-cli/Earthfile). I'll create a test target for my activity client (`ac-test`), and copy in client binary and the test script: @@ -526,7 +526,7 @@ test: Then I'll start-up the docker container for the service (using its GitHub path) and run `test.sh`: ~~~{.dockerfile captionb="Earthfile"} - WITH DOCKER --load agbell/cloudservices/activityserver=github.com/adamgordonbell/cloudservices/ActivityLog+docker + WITH DOCKER --load agbell/cloudservices/activityserver=github.com/earthly/cloud-services-example/ActivityLog+docker RUN docker run -d -p 8080:8080 agbell/cloudservices/activityserver && \ ./test.sh END @@ -565,7 +565,7 @@ func (c *Activities) Insert(activity api.Activity) int { My initial attempts to import the JSON service types into the CLI client were a failure. Problems encountered included: -* **Problem:** module `module github.com/adamgordonbell/cloudservices/activitylog` was in a folder called `ActivityLog`. This caused inconsistency caused problems when importing. +* **Problem:** module `module github.com/earthly/cloud-services-example/activitylog` was in a folder called `ActivityLog`. This caused inconsistency caused problems when importing. **Solution** I renamed all packages to be kebab-cased. `ActivityLog` is now `activity-log`. Problem solved! * **Problem:** Backend using uint64 and frontend using int leading to `cannot use id (type int) as type uint64 in field value` everywhere. @@ -575,20 +575,20 @@ My initial attempts to import the JSON service types into the CLI client were a **Solution** use `replace` in `go.mod` to use local version of `activity-log` in `activity-client`. ~~~ -module github.com/adamgordonbell/cloudservices/activity-client +module github.com/earthly/cloud-services-example/activity-client go 1.17 -require github.com/adamgordonbell/cloudservices/activity-log v0.0.0 +require github.com/earthly/cloud-services-example/activity-log v0.0.0 -replace github.com/adamgordonbell/cloudservices/activity-log => ../activity-log +replace github.com/earthly/cloud-services-example/activity-log => ../activity-log ~~~ ### What's Next -So now I've learned the basics of building a command-line tool that calls a JSON web-service in GoLang. It went pretty smoothly, and the amount of code I had to write was [pretty minimal](https://github.com/adamgordonbell/cloudservices/tree/v2-cli/activity-client). +So now I've learned the basics of building a command-line tool that calls a JSON web-service in GoLang. It went pretty smoothly, and the amount of code I had to write was [pretty minimal](https://github.com/earthly/cloud-services-example/tree/v2-cli/activity-client). There are two things I want to add to the activity tracker next. First, since all that calls to backend service are in this client, I want to move to GRPC. Second, I need some persistence - right now, the service holds everything in memory. I can't have a power outage erasing all of my hard work. diff --git a/blog/_posts/2022-02-03-golang-sqlite.md b/blog/_posts/2022-02-03-golang-sqlite.md index 53ba479ab..55184a26f 100644 --- a/blog/_posts/2022-02-03-golang-sqlite.md +++ b/blog/_posts/2022-02-03-golang-sqlite.md @@ -22,7 +22,7 @@ Welcome back. I'm an experienced developer, learning Golang by building an activ **If you're curious about the basics of storing persistent data into a SQL database using Golang, this tutorial will be helpful for you.** I'm going to be using `sqlite3`, but I'll add lots of headings, so you can skip ahead if `sqlite` is not your thing. -My plan is to add SQLite persistence to [the backend service](https://github.com/adamgordonbell/cloudservices) so that my workouts aren't lost if the service goes down. And once I have that, I'll add the `--list` command to my command line client and add an end point for it. it's the type of feature that is simple to do with a SQL backend. +My plan is to add SQLite persistence to [the backend service](https://github.com/earthly/cloud-services-example) so that my workouts aren't lost if the service goes down. And once I have that, I'll add the `--list` command to my command line client and add an end point for it. it's the type of feature that is simple to do with a SQL backend. ## Install SQLite @@ -195,8 +195,6 @@ go get github.com/mattn/go-sqlite3 Finally, let's jump into the Golang code. -{% include_html cta/bottom-cta.html %} - ## Golang SQL Repository Previously `server.Activities` contained a slice of `api.Activity`. Now I'm going to update it to contain a pointer to a `sql.DB`. This will be my database handle. It will be how I store and retrieve the activity records. Here is a diff: @@ -240,7 +238,7 @@ import ( "log" "sync" - api "github.com/adamgordonbell/cloudservices/activity-log" + api "github.com/earthly/cloud-services-example/activity-log" + _ "github.com/mattn/go-sqlite3" ) ~~~ @@ -285,7 +283,7 @@ func NewActivities() (*Activities, error) { } ~~~ -
Initialize the [database](https://github.com/adamgordonbell/cloudservices/blob/v3-sqlite/activity-log/internal/server/activity.go)
+
Initialize the [database](https://github.com/earthly/cloud-services-example/blob/v3-sqlite/activity-log/internal/server/activity.go)
## Golang Insert Into Database @@ -365,8 +363,10 @@ Thankfully, I can use [`sql.QueryRow`](https://github.com/golang/go/blob/2580d0e My usage looks like this: ~~~{.go captionb="internal/server/activity.go"} - row := c.db.QueryRow("SELECT id, time, description FROM activities WHERE id=?", id) - +row := c.db.QueryRow(` + SELECT id, time, description + FROM activities + WHERE id=?`, id) ~~~ To convert the database values into my struct `api.Activity` I used `row.Scan` ( or `row.Scan` for multiple rows). It copies columns from the row into the value pointed at by each of its arguments. @@ -378,12 +378,16 @@ func (c *Activities) Retrieve(id int) (api.Activity, error) { log.Printf("Getting %d", id) // Query DB row based on ID - row := c.db.QueryRow("SELECT id, time, description FROM activities WHERE id=?", id) +row := c.db.QueryRow(` + SELECT id, time, description + FROM activities + WHERE id=?`, id) // Parse row into Activity struct activity := api.Activity{} var err error - if err = row.Scan(&activity.ID, &activity.Time, &activity.Description); err == sql.ErrNoRows { + if err = row.Scan(&activity.ID, &activity.Time, &activity.Description); + err == sql.ErrNoRows { log.Printf("Id not found") return api.Activity{}, ErrIDNotFound } @@ -470,7 +474,8 @@ Now I can add my `-list` endpoint. It follows a similar pattern as Retrieve (`-g ~~~{.go captionb="internal/server/activity.go"} func (c *Activities) List(offset int) ([]api.Activity, error) { - rows, err := c.db.Query("SELECT * FROM activities WHERE ID > ? ORDER BY id DESC LIMIT 100", offset) + rows, err := c.db.Query( + "SELECT * FROM activities WHERE ID > ? ORDER BY id DESC LIMIT 100", offset) if err != nil { return nil, err } @@ -620,7 +625,7 @@ Here it is in GitHub Actions:
-Now my activity service has a persistence layer, and I learned quite a bit about how `database/sql`, `sqlite3`, `sqlite-utils` and `github.com/mattn/go-sqlite3` work. Thank you for coming along on the journey with me. I didn't show all the code changes here, but you can find the [diff](https://github.com/adamgordonbell/cloudservices/commit/9398c7251af9ef3d61a3ac32a5535cb7e71985fb) and the [full code](https://github.com/adamgordonbell/cloudservices/tree/v3-sqlite) on GitHub. +Now my activity service has a persistence layer, and I learned quite a bit about how `database/sql`, `sqlite3`, `sqlite-utils` and `github.com/mattn/go-sqlite3` work. Thank you for coming along on the journey with me. I didn't show all the code changes here, but you can find the [diff](https://github.com/earthly/cloud-services-example/commit/9398c7251af9ef3d61a3ac32a5535cb7e71985fb) and the [full code](https://github.com/earthly/cloud-services-example/tree/v3-sqlite) on GitHub. ## What's Next diff --git a/blog/_posts/2022-02-16-golang-grpc-example.md b/blog/_posts/2022-02-16-golang-grpc-example.md index 31606833b..979263b48 100644 --- a/blog/_posts/2022-02-16-golang-grpc-example.md +++ b/blog/_posts/2022-02-16-golang-grpc-example.md @@ -28,7 +28,7 @@ last_modified_at: 2023-09-19 Welcome back. I'm an experienced developer, learning Golang by building an activity tracker. Last time I added SQLite persistence. Today, I'm going to be porting everything to gRPC. -If you're curious about gRPC – how it works, when to use it, what example code might look like – well, you are in luck because I'm going to be building a grpc client, a grpc server, and the protobuf files for my activity tracker. The full code is on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v4-grpc). +If you're curious about gRPC – how it works, when to use it, what example code might look like – well, you are in luck because I'm going to be building a grpc client, a grpc server, and the protobuf files for my activity tracker. The full code is on [GitHub](https://github.com/earthly/cloud-services-example/tree/v4-grpc). ## Why gRPC @@ -247,8 +247,8 @@ If you recall from when I was adding the `sqlite` feature, Activities handles al ~~~{.diff caption="internal/server/activity.go "} import -- api "github.com/adamgordonbell/cloudservices/activity-log" -+ api "github.com/adamgordonbell/cloudservices/activity-log/api/v1" +- api "github.com/earthly/cloud-services-example/activity-log" ++ api "github.com/earthly/cloud-services-example/activity-log/api/v1" + "google.golang.org/protobuf/types/known/timestamppb" ~~~ @@ -278,7 +278,7 @@ func (c *Activities) Insert(activity *api.Activity) (int, error) { } ~~~ -And that is the only persistence layer change we need to make to switch from our hand-rolled struct to the `protoc` generated one. Again, you can see the full thing on [github](https://github.com/adamgordonbell/cloudservices/blob/v4-grpc/activity-log/internal/server/activity.go). +And that is the only persistence layer change we need to make to switch from our hand-rolled struct to the `protoc` generated one. Again, you can see the full thing on [github](https://github.com/earthly/cloud-services-example/blob/v4-grpc/activity-log/internal/server/activity.go). ### GRPC Service @@ -339,7 +339,7 @@ $ go run cmd/server/main.go ~~~ ~~~{.bash .merge-code} -# github.com/adamgordonbell/cloudservices/activity-log/internal/server +# github.com/earthly/cloud-services-example/activity-log/internal/server internal/server/server.go:30:39: cannot use &srv (type *grpcServer) as type api_v1.Activity_LogServer in argument to api_v1.RegisterActivity_LogServer: *grpcServer does not implement api_v1.Activity_LogServer (missing api_v1.mustEmbedUnimplementedActivity_LogServer method) ~~~ @@ -516,7 +516,7 @@ func (s *grpcServer) Insert(ctx context.Context, activity *api.Activity) (*api.I } ~~~ -I can repeat this for [`List` and `Retrieve`](https://github.com/adamgordonbell/cloudservices/blob/v4-grpc/activity-log/internal/server/server.go), and I have a working solution. (Though the error handling has room for improvement. I'll get back to that later on in the article). +I can repeat this for [`List` and `Retrieve`](https://github.com/earthly/cloud-services-example/blob/v4-grpc/activity-log/internal/server/server.go), and I have a working solution. (Though the error handling has room for improvement. I'll get back to that later on in the article). ### Testing A gRPC Server @@ -755,7 +755,7 @@ func (c *Activities) Retrieve(ctx context.Context, id int) (*api.Activity, error } ~~~ -And with that implementation [in place](https://github.com/adamgordonbell/cloudservices/blob/v4-grpc/activity-client/internal/client/activity.go), the client works. Here is the Earthly build: +And with that implementation [in place](https://github.com/earthly/cloud-services-example/blob/v4-grpc/activity-client/internal/client/activity.go), the client works. Here is the Earthly build:
{% picture content-wide-nocrop {{site.pimages}}{{page.slug}}/3060.png --alt {{ gRPC Client Example Working }} %} diff --git a/blog/_posts/2022-03-17-golang-grpc-gateway.md b/blog/_posts/2022-03-17-golang-grpc-gateway.md index 2194da00b..39ee4f05b 100644 --- a/blog/_posts/2022-03-17-golang-grpc-gateway.md +++ b/blog/_posts/2022-03-17-golang-grpc-gateway.md @@ -12,7 +12,7 @@ internal-links: topic: go excerpt: | In this article, the author explores different ways to create a gRPC gateway that accepts HTTP requests and proxies them to a gRPC service. They cover building a proxy using grpc-gateway, creating a REST service based on the same proto file as the gRPC service, and combining REST and gRPC requests in a single service. The author also discusses TLS, certificate generation, and HTTP/2 in the context of these implementations. -last_modified_at: 2023-07-19 +last_modified_at: 2024-04-13 --- **Exploring gRPC gateway methods? Earthly simplifies your build process for gRPC services. [Check it out](https://cloud.earthly.dev/login/).** @@ -34,7 +34,7 @@ Ok, lets start. The first thing I need to do is get the gRPC gateway plugin: go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway ~~~ -Then I update my [protoc invocation](https://github.com/adamgordonbell/cloudservices/blob/v5-grpc-gateway/activity-log/Earthfile#L29) to use this plugin: +Then I update my [protoc invocation](https://github.com/earthly/cloud-services-example/blob/v5-grpc-gateway/activity-log/Earthfile#L29) to use this plugin: ~~~{.diff caption=">_"} protoc api/v1/*.proto \ @@ -80,7 +80,7 @@ import ( "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" - api "github.com/adamgordonbell/cloudservices/activity-log/api/v1" + api "github.com/earthly/cloud-services-example/activity-log/api/v1" ) ~~~ @@ -206,7 +206,7 @@ Which I can view in a more human readable form using the online [swagger editor]
Creating a Swagger Doc for a gRPC service proxy
-You can find the code for the above gRPC proxy on [GitHub](https://github.com/adamgordonbell/cloudservices/blob/v5-grpc-gateway/grpc-proxy/main.go), and If you have the proto files for a gRPC then all you need to do is generate the proxy and swagger files with `protoc` and adapt the one file service to your needs. +You can find the code for the above gRPC proxy on [GitHub](https://github.com/earthly/cloud-services-example/blob/v5-grpc-gateway/grpc-proxy/main.go), and If you have the proto files for a gRPC then all you need to do is generate the proxy and swagger files with `protoc` and adapt the one file service to your needs. Let's move on to the next gRPC gateway example. @@ -258,7 +258,7 @@ The big change is calling `RegisterActivity_LogHandlerServer` instead of `Regist ### SideNote: SQLite -My [toy example](https://github.com/adamgordonbell/cloudservices/tree/v5-grpc-gateway) is using SQLite, which probably isn't a great fit for this solution because it involves multiple services writing to the database. With a network-based database, however, this could work quite well. +My [toy example](https://github.com/earthly/cloud-services-example/tree/v5-grpc-gateway) is using SQLite, which probably isn't a great fit for this solution because it involves multiple services writing to the database. With a network-based database, however, this could work quite well. And practically, the reason I'm showing this solution is a half step toward the final solution: responding to HTTP rest requests and gRPC requests in a single service. So lets go there next. @@ -280,12 +280,12 @@ import ( "net/http" "strings" - "github.com/adamgordonbell/cloudservices/activity-log/internal/server" + "github.com/earthly/cloud-services-example/activity-log/internal/server" "github.com/grpc-ecosystem/grpc-gateway/v2/runtime" "google.golang.org/grpc" "google.golang.org/grpc/reflection" - api "github.com/adamgordonbell/cloudservices/activity-log/api/v1" + api "github.com/earthly/cloud-services-example/activity-log/api/v1" ) func main() { @@ -460,7 +460,7 @@ $ cfssl gencert -ca ca.pem -ca-key=ca-key.pem -config ca-config.json \ -profile=server server-csr.json | cfssljson -bare server ~~~ -With all that generation in place, and wrapped up in a nice [Earthfile target](https://github.com/adamgordonbell/cloudservices/blob/v5-grpc-gateway/activity-log/Earthfile#L44), my side quest is over, and I can head back to my activity-log service. +With all that generation in place, and wrapped up in a nice [Earthfile target](https://github.com/earthly/cloud-services-example/blob/v5-grpc-gateway/activity-log/Earthfile#L44), my side quest is over, and I can head back to my activity-log service. ### TLS Time @@ -501,7 +501,22 @@ $ grpcurl -insecure localhost:8080 api.v1.Activity_Log/List } ~~~ -And for curl, I can use `-k`: +I can also specify the cert like this: + +~~~{.bash caption=">_"} +$ grpcurl -cacert=./certs/ca.pem localhost:8080 api.v1.Activity_Log/List +{ + "activities": [ + { + "id": 2, + "time": "1970-01-01T00:00:00Z", + "description": "christmas eve bike class" + } + ] +} +~~~ + +See the [updated test.sh](https://github.com/earthly/cloud-services-example/blob/v5-grpc-gateway/activity-log/test.sh) for more details, and for curl, I can use `-k`: ~~~{.bash caption=">_"} curl -k -X POST -s https://localhost:8080/api.v1.Activity_Log/List -d \ @@ -611,9 +626,11 @@ func main() { +### Using `grpcurl` With TLS + ## Conclusion -There we have it. Rest to gRPC in three ways, with all the complicated bits documented in a runnable [Earthfile](https://github.com/adamgordonbell/cloudservices/blob/v5-grpc-gateway/Earthfile). All the code is on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v5-grpc-gateway). And with the certs in place, this gRPC + REST service is not even that big of a lift from a standard gRPC end-point. In fact, this approach is in use in [`etcd`](https://github.com/etcd-io/etcd/blob/main/server/embed/serve.go) and [Istio](https://github.com/istio/istio/blob/f46f821fb13b7fc24b5d29193e2ad7c5c0a46877/pilot/pkg/bootstrap/server.go#L469). +There we have it. Rest to gRPC in three ways, with all the complicated bits documented in a runnable [Earthfile](https://github.com/earthly/cloud-services-example/blob/v5-grpc-gateway/Earthfile). All the code is on [GitHub](https://github.com/earthly/cloud-services-example/tree/v5-grpc-gateway). And with the certs in place, this gRPC + REST service is not even that big of a lift from a standard gRPC end-point. In fact, this approach is in use in [`etcd`](https://github.com/etcd-io/etcd/blob/main/server/embed/serve.go) and [Istio](https://github.com/istio/istio/blob/f46f821fb13b7fc24b5d29193e2ad7c5c0a46877/pilot/pkg/bootstrap/server.go#L469). And if you enjoyed the build process for your gRPC gateway, consider diving deeper with [Earthly](https://cloud.earthly.dev/login) to further simplify your build process. diff --git a/blog/_posts/2022-03-18-buf-protobuf.md b/blog/_posts/2022-03-18-buf-protobuf.md index c79f2d50a..6acfce7dc 100644 --- a/blog/_posts/2022-03-18-buf-protobuf.md +++ b/blog/_posts/2022-03-18-buf-protobuf.md @@ -10,7 +10,7 @@ internal-links: - buf excerpt: | Learn how to avoid common pitfalls when working with Protobuf using Buf, a suite of tools that simplifies dealing with protocol buffers. Discover how to use Buf's linting, breaking change detection, and code generation features to improve your development process. -last_modified_at: 2023-07-19 +last_modified_at: 2024-04-13 --- **This article explores the `buf` tool. Earthly can enhance your `buf` toolchain by ensuring reproducible builds for your gRPC services. [Check it out](https://cloud.earthly.dev/login).** @@ -24,7 +24,7 @@ So before I roll this service out and start actively using it, I will take my ex ## Background -The current [activity tracking code](https://github.com/adamgordonbell/cloudservices/tree/v5-grpc-gateway) and [final version](https://github.com/adamgordonbell/cloudservices/tree/v6-buf) are on GitHub but the main thing you need to know for this walk-though is that my gRPC service is defined like this: +The current [activity tracking code](https://github.com/earthly/cloud-services-example/tree/v5-grpc-gateway) and [final version](https://github.com/earthly/cloud-services-example/tree/v6-buf) are on GitHub but the main thing you need to know for this walk-though is that my gRPC service is defined like this: ~~~{.protobuf caption="activity-log/api/v1/activity.proto"} service ActivityLogService { @@ -101,7 +101,7 @@ As my activity service evolves – as I add new features and roll out new versio ~~~{.bash caption=">_"} -> buf breaking --against "https://github.com/adamgordonbell/cloudservices.git#branch=main,subdir=activity-log" +> buf breaking --against "https://github.com/earthly/cloud-services-example.git#branch=main,subdir=activity-log" ~~~
Using `buf breaking` to compare against main branch
@@ -197,7 +197,7 @@ proto: - --proto_path=. + COPY buf.* . + RUN buf lint -+ RUN buf breaking --against "https://github.com/adamgordonbell/cloudservices.git#branch=buf,subdir=activity-log" ++ RUN buf breaking --against "https://github.com/earthly/cloud-services-example.git#branch=buf,subdir=activity-log" + RUN buf generate SAVE ARTIFACT ./api AS LOCAL ./api ~~~ diff --git a/blog/_posts/2022-05-04-aws-lambda-golang.md b/blog/_posts/2022-05-04-aws-lambda-golang.md index 09f435c80..66bd84e02 100644 --- a/blog/_posts/2022-05-04-aws-lambda-golang.md +++ b/blog/_posts/2022-05-04-aws-lambda-golang.md @@ -481,7 +481,7 @@ It's surprising to me how much faster the Golang version is. Lynx and the readab ## Conclusion -So there you go, containerized serverless Golang. We built a program in Go that has some OS level dependencies (lynx), we've wrapped it up into a container, ran it in an AWS Lambda, and then also used S3 get and puts for caching. And the whole up to a REST end point. You can [test it out](https://earthly-tools.com/text-mode) or use it for your own purposes and the complete source code is on [github](https://github.com/adamgordonbell/cloudservices/tree/aws-lambda-1). +So there you go, containerized serverless Golang. We built a program in Go that has some OS level dependencies (lynx), we've wrapped it up into a container, ran it in an AWS Lambda, and then also used S3 get and puts for caching. And the whole up to a REST end point. You can [test it out](https://earthly-tools.com/text-mode) or use it for your own purposes and the complete source code is on [github](https://github.com/earthly/cloud-services-example/tree/aws-lambda-1). And if you liked how I put together this project, take a look at [Earthly](https://cloud.earthly.dev/login). diff --git a/blog/_posts/2022-06-13-aws-lambda-api-proxy.md b/blog/_posts/2022-06-13-aws-lambda-api-proxy.md index 2276bd3b1..df0b33ea1 100644 --- a/blog/_posts/2022-06-13-aws-lambda-api-proxy.md +++ b/blog/_posts/2022-06-13-aws-lambda-api-proxy.md @@ -8,7 +8,7 @@ sidebar: nav: "lambdas" excerpt: | Learn how to run a full REST HTTP API in a single AWS Lambda using Golang. Discover the advantages of this approach and how to handle routing and requests using the AWS Lambda Go API Proxy. -last_modified_at: 2023-07-14 +last_modified_at: 2024-04-13 --- **This article explains the creation of Lambda-based REST APIs. Earthly streamlines the CI pipeline for GoLang developers using AWS Lambda functions. [Learn more about Earthly](https://cloud.earthly.dev/login).** @@ -108,7 +108,7 @@ func main() { } ~~~ -It proxies the requests and responses, converts them to the proper format, and communicates with the Lambda runtime. I'm going port the [Text-mode service](/blog/text-mode/) to use this framework, and if you want to skip ahead, the [code is on GitHub](https://github.com/adamgordonbell/cloudservices/tree/aws-lambda-2). +It proxies the requests and responses, converts them to the proper format, and communicates with the Lambda runtime. I'm going port the [Text-mode service](/blog/text-mode/) to use this framework, and if you want to skip ahead, the [code is on GitHub](https://github.com/earthly/cloud-services-example/tree/aws-lambda-2). > Lambda is Greek for CGI script [^3] @@ -315,7 +315,7 @@ $ curl localhost:8080/default/text-mode That is because I'm still running the lambda runtime locally, which expects JSON events. You can use this locally, as seen in [this article](/blog/aws-lambda-docker/), but it's a bit cumbersome. -To correct this, I need to modify the image (`public.ecr.aws/lambda/go:latest`) that I'm running. So, I create a second image with an updated entrypoint, in my [Earthfile](https://github.com/adamgordonbell/cloudservices/blob/aws-lambda-2/lambda-api/Earthfile): +To correct this, I need to modify the image (`public.ecr.aws/lambda/go:latest`) that I'm running. So, I create a second image with an updated entrypoint, in my [Earthfile](https://github.com/earthly/cloud-services-example/blob/aws-lambda-2/lambda-api/Earthfile): ~~~{.docker caption="Earthfile"} local-image: @@ -380,7 +380,7 @@ Earthly.dev Presents: |_| |_| \___/ \__,_| \___| ~~~ -The complete source code is [on GitHub](https://github.com/adamgordonbell/cloudservices/tree/aws-lambda-2/lambda-api) , and the code for previous versions. This solution should work for any HTTP service in go, whether written using gorrilaMux, the standard lib, or whatever HTTP framework you prefer. +The complete source code is [on GitHub](https://github.com/earthly/cloud-services-example/tree/aws-lambda-2/lambda-api) , and the code for previous versions. This solution should work for any HTTP service in go, whether written using gorrilaMux, the standard lib, or whatever HTTP framework you prefer. I think this can be a powerful model for deploying stateless HTTP services without getting too intertwined and locked into AWS-specific features. It's just a container and a proxy lib. Everything else works just like you are used to. diff --git a/blog/_posts/2022-07-20-terraform-lambda.md b/blog/_posts/2022-07-20-terraform-lambda.md index 0566662f7..f1411cf28 100644 --- a/blog/_posts/2022-07-20-terraform-lambda.md +++ b/blog/_posts/2022-07-20-terraform-lambda.md @@ -855,6 +855,6 @@ Initially, I found working with Terraform to be a challenge. But I now have the The Terraform code written here for sure could be improved. I could be extracting my code into separate modules and separating out variables and using workspaces and lifecycles to create a better factored infrastructure as code solution. But doing it from first principles and keeping things simple has been instructive and hopefully reading this conversion build log is valuable for you. -You can find the full all the source on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/terraform-import). +You can find the full all the source on [GitHub](https://github.com/earthly/cloud-services-example/tree/terraform-import). {% include_html cta/bottom-cta.html %} diff --git a/blog/_posts/2022-08-02-terraform-route53.md b/blog/_posts/2022-08-02-terraform-route53.md index 181e62198..4d0be13f4 100644 --- a/blog/_posts/2022-08-02-terraform-route53.md +++ b/blog/_posts/2022-08-02-terraform-route53.md @@ -197,6 +197,6 @@ Earthly.dev Presents: |_| |_| \___/ \__,_| \___| ~~~ -The code is on [GitHub](https://github.com/adamgordonbell/cloudservices/commit/e213af302cf6372aca4c099419bc6d2a0896ae7a) +The code is on [GitHub](https://github.com/earthly/cloud-services-example/commit/e213af302cf6372aca4c099419bc6d2a0896ae7a) {% include_html cta/bottom-cta.html %}