Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Article Updates #836

Merged
merged 9 commits into from
May 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion blog/_data/navigation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ activity-tracker:
url: /golang-sqlite/
- title: "Part 4: gRPC"
url: /golang-grpc-example/
- title: "Part 4: gRPC Gateway"
- title: "Part 5: gRPC Gateway"
url: /golang-grpc-gateway/
- title: "Side Quest: Protobuf's"
url: /buf-protobuf/
Expand Down
20 changes: 10 additions & 10 deletions blog/_posts/2021-12-20-golang-http.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ internal-links:
- golang http
excerpt: |
Learn how to build a JSON HTTP server using Golang in this tutorial. Discover the basics of creating a Golang web service, handling HTTP requests, and working with JSON data.
last_modified_at: 2023-09-19
last_modified_at: 2024-04-07
---
**This article explains Golang JSON services. Earthly simplifies the build and test processes for Go web services. [Learn more](https://cloud.earthly.dev/login).**

Expand Down Expand Up @@ -190,7 +190,7 @@ test:

<figcaption>Test my containerized service</figcaption>

What I've built up so far is on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/WebServer), but it doesn't do much. For my activity tracker to be useful, it will need to understand and store activities.
What I've built up so far is on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/WebServer), but it doesn't do much. For my activity tracker to be useful, it will need to understand and store activities.

So let's move on to my activity data structures.

Expand Down Expand Up @@ -228,7 +228,7 @@ Doing that, I can write an insert function like this:

~~~{.go captionb="internal/server/activity.go"}
func (c *Activities) Insert(activity Activity) uint64 {
activity.ID = uint64(len(c.activities))
activity.ID = uint64(len(c.activities)) + 1
c.activities = append(c.activities, activity)
return activity.ID
}
Expand All @@ -238,10 +238,10 @@ And retrieve is super simple as well:

~~~{.go captionb="internal/server/activity.go"}
func (c *Activities) Retrieve(id uint64) (Activity, error) {
if id >= uint64(len(c.activities)) {
if id > uint64(len(c.activities)) {
return Activity{}, ErrIDNotFound
}
return c.activities[id], nil
return c.activities[id-1], nil
}
~~~

Expand Down Expand Up @@ -360,7 +360,7 @@ We now have half our API working!
~~~{.bash caption=">_"}
> curl -X POST localhost:8080 -d \
'{"activity": {"description": "christmas eve class",
time:"2021-12-24T12:42:31Z"}}'
"time":"2021-12-24T12:42:31Z"}}'
~~~

~~~{.merge-code}
Expand Down Expand Up @@ -428,7 +428,7 @@ curl -X POST localhost:8080 -d \
class","id":15}
~~~

the whole thing, including some edge cases I left out is on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/ActivityLog).
the whole thing, including some edge cases I left out is on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/ActivityLog).

If fact, I can now update my shell script `test.sh` to exercise these endpoints.

Expand Down Expand Up @@ -467,12 +467,12 @@ And now, since I wrote that `Earthfile` that starts up the service and runs `tes

<div class="wide">
{% picture content-wide-nocrop {{site.pimages}}{{page.slug}}/1010.png --alt {{ GitHub Actions }} %}
<figcaption>Passing End to End tests on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/ActivityLog)</figcaption>
<figcaption>Passing End to End tests on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/ActivityLog)</figcaption>
</div>

## That's a Wrap

There we go. I have a working service that I've put up on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v1-http-server/ActivityLog) with an active CI process. It doesn't persist its data, it doesn't allow me to access my activity log in any other way than by id, and it doesn't have a UI, but I'm starting to get a feel for how web services are built in Golang, which was the point.
There we go. I have a working service that I've put up on [GitHub](https://github.com/earthly/cloud-services-example/tree/v1-http-server/ActivityLog) with an active CI process. It doesn't persist its data, it doesn't allow me to access my activity log in any other way than by id, and it doesn't have a UI, but I'm starting to get a feel for how web services are built in Golang, which was the point.

As an activity tracker, what I have so far is pretty weak. But as a learning lesson, I've found it valuable.

Expand All @@ -484,7 +484,7 @@ Now I just have to start being active! Maybe building a command line client for

### Linting

In the first version of this example I used `Id` everywhere instead of `ID`, which is incorrect capitalization (per `go lint` and [Alex](/blog/authors/Alex/)). To prevent further style issues like this as I continue building this application I'm linting my code going forward using [`golangci-lint`](https://golangci-lint.run/) which with the [right configuration](https://github.com/adamgordonbell/cloudservices/blob/v1-http-server/ActivityLog/.golangci.yml) calls several go linters, including `go lint`.
In the first version of this example I used `Id` everywhere instead of `ID`, which is incorrect capitalization (per `go lint` and [Alex](/blog/authors/Alex/)). To prevent further style issues like this as I continue building this application I'm linting my code going forward using [`golangci-lint`](https://golangci-lint.run/) which with the [right configuration](https://github.com/earthly/cloud-services-example/blob/v1-http-server/ActivityLog/.golangci.yml) calls several go linters, including `go lint`.

### Errors

Expand Down
22 changes: 11 additions & 11 deletions blog/_posts/2022-01-11-golang-command-line.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ topic: cli
funnel: 2
excerpt: |
Learn how to build a command-line JSON client in Golang to interact with a REST service for storing workout activities. The article covers topics such as parsing command-line flags, making HTTP requests, handling errors, and testing the client.
last_modified_at: 2023-09-19
last_modified_at: 2024-04-07
---
**This article is about Golang activity tracking. Earthly can streamline your build process. [Check it out](https://cloud.earthly.dev/login).**

Expand All @@ -40,7 +40,7 @@ The existing backend doesn't support `list` yet, so we will skip that one for no
First, I create a new folder for my client:

~~~{.bash caption=">_"}
$ go mod init github.com/adamgordonbell/cloudservices/activityclient
$ go mod init github.com/earthly/cloud-services-example/activityclient
~~~

## Command Line Flags
Expand Down Expand Up @@ -334,7 +334,7 @@ package client
import (
...

api "github.com/adamgordonbell/cloudservices/activity-log"
api "github.com/earthly/cloud-services-example/activity-log"
)

~~~
Expand Down Expand Up @@ -489,7 +489,7 @@ Then I just need to `json.Unmarshall` my activity document:
return document.Activity, nil
~~~

And with that, I have a [working](https://github.com/adamgordonbell/cloudservices/tree/v2-cli/activity-client), though basic, client. So I'm going to add some light testing and then call it a day.
And with that, I have a [working](https://github.com/earthly/cloud-services-example/tree/v2-cli/activity-client), though basic, client. So I'm going to add some light testing and then call it a day.

## Testing the Happy Path

Expand All @@ -512,7 +512,7 @@ Assuming the backend service is up, and the client is built, this will test that

## Continuous Integration

I can quickly hook this happy path up to CI by extending my previous [Earthfile](https://github.com/adamgordonbell/cloudservices/blob/v2-cli/Earthfile).
I can quickly hook this happy path up to CI by extending my previous [Earthfile](https://github.com/earthly/cloud-services-example/blob/v2-cli/Earthfile).

I'll create a test target for my activity client (`ac-test`), and copy in client binary and the test script:

Expand All @@ -526,7 +526,7 @@ test:
Then I'll start-up the docker container for the service (using its GitHub path) and run `test.sh`:

~~~{.dockerfile captionb="Earthfile"}
WITH DOCKER --load agbell/cloudservices/activityserver=github.com/adamgordonbell/cloudservices/ActivityLog+docker
WITH DOCKER --load agbell/cloudservices/activityserver=github.com/earthly/cloud-services-example/ActivityLog+docker
RUN docker run -d -p 8080:8080 agbell/cloudservices/activityserver && \
./test.sh
END
Expand Down Expand Up @@ -565,7 +565,7 @@ func (c *Activities) Insert(activity api.Activity) int {

My initial attempts to import the JSON service types into the CLI client were a failure. Problems encountered included:

* **Problem:** module `module github.com/adamgordonbell/cloudservices/activitylog` was in a folder called `ActivityLog`. This caused inconsistency caused problems when importing.
* **Problem:** module `module github.com/earthly/cloud-services-example/activitylog` was in a folder called `ActivityLog`. This caused inconsistency caused problems when importing.

**Solution** I renamed all packages to be kebab-cased. `ActivityLog` is now `activity-log`. Problem solved!
* **Problem:** Backend using uint64 and frontend using int leading to `cannot use id (type int) as type uint64 in field value` everywhere.
Expand All @@ -575,20 +575,20 @@ My initial attempts to import the JSON service types into the CLI client were a
**Solution** use `replace` in `go.mod` to use local version of `activity-log` in `activity-client`.

~~~
module github.com/adamgordonbell/cloudservices/activity-client
module github.com/earthly/cloud-services-example/activity-client

go 1.17

require github.com/adamgordonbell/cloudservices/activity-log v0.0.0
require github.com/earthly/cloud-services-example/activity-log v0.0.0

replace github.com/adamgordonbell/cloudservices/activity-log => ../activity-log
replace github.com/earthly/cloud-services-example/activity-log => ../activity-log
~~~

</div>

### What's Next

So now I've learned the basics of building a command-line tool that calls a JSON web-service in GoLang. It went pretty smoothly, and the amount of code I had to write was [pretty minimal](https://github.com/adamgordonbell/cloudservices/tree/v2-cli/activity-client).
So now I've learned the basics of building a command-line tool that calls a JSON web-service in GoLang. It went pretty smoothly, and the amount of code I had to write was [pretty minimal](https://github.com/earthly/cloud-services-example/tree/v2-cli/activity-client).

There are two things I want to add to the activity tracker next. First, since all that calls to backend service are in this client, I want to move to GRPC. Second, I need some persistence - right now, the service holds everything in memory. I can't have a power outage erasing all of my hard work.

Expand Down
27 changes: 16 additions & 11 deletions blog/_posts/2022-02-03-golang-sqlite.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Welcome back. I'm an experienced developer, learning Golang by building an activ

**If you're curious about the basics of storing persistent data into a SQL database using Golang, this tutorial will be helpful for you.** I'm going to be using `sqlite3`, but I'll add lots of headings, so you can skip ahead if `sqlite` is not your thing.

My plan is to add SQLite persistence to [the backend service](https://github.com/adamgordonbell/cloudservices) so that my workouts aren't lost if the service goes down. And once I have that, I'll add the `--list` command to my command line client and add an end point for it. it's the type of feature that is simple to do with a SQL backend.
My plan is to add SQLite persistence to [the backend service](https://github.com/earthly/cloud-services-example) so that my workouts aren't lost if the service goes down. And once I have that, I'll add the `--list` command to my command line client and add an end point for it. it's the type of feature that is simple to do with a SQL backend.

## Install SQLite

Expand Down Expand Up @@ -195,8 +195,6 @@ go get github.com/mattn/go-sqlite3

Finally, let's jump into the Golang code.

{% include_html cta/bottom-cta.html %}

## Golang SQL Repository

Previously `server.Activities` contained a slice of `api.Activity`. Now I'm going to update it to contain a pointer to a `sql.DB`. This will be my database handle. It will be how I store and retrieve the activity records. Here is a diff:
Expand Down Expand Up @@ -240,7 +238,7 @@ import (
"log"
"sync"

api "github.com/adamgordonbell/cloudservices/activity-log"
api "github.com/earthly/cloud-services-example/activity-log"
+ _ "github.com/mattn/go-sqlite3"
)
~~~
Expand Down Expand Up @@ -285,7 +283,7 @@ func NewActivities() (*Activities, error) {
}
~~~

<figcaption>Initialize the [database](https://github.com/adamgordonbell/cloudservices/blob/v3-sqlite/activity-log/internal/server/activity.go)</figcaption>
<figcaption>Initialize the [database](https://github.com/earthly/cloud-services-example/blob/v3-sqlite/activity-log/internal/server/activity.go)</figcaption>

## Golang Insert Into Database

Expand Down Expand Up @@ -365,8 +363,10 @@ Thankfully, I can use [`sql.QueryRow`](https://github.com/golang/go/blob/2580d0e
My usage looks like this:

~~~{.go captionb="internal/server/activity.go"}
row := c.db.QueryRow("SELECT id, time, description FROM activities WHERE id=?", id)

row := c.db.QueryRow(`
SELECT id, time, description
FROM activities
WHERE id=?`, id)
~~~

To convert the database values into my struct `api.Activity` I used `row.Scan` ( or `row.Scan` for multiple rows). It copies columns from the row into the value pointed at by each of its arguments.
Expand All @@ -378,12 +378,16 @@ func (c *Activities) Retrieve(id int) (api.Activity, error) {
log.Printf("Getting %d", id)

// Query DB row based on ID
row := c.db.QueryRow("SELECT id, time, description FROM activities WHERE id=?", id)
row := c.db.QueryRow(`
SELECT id, time, description
FROM activities
WHERE id=?`, id)

// Parse row into Activity struct
activity := api.Activity{}
var err error
if err = row.Scan(&activity.ID, &activity.Time, &activity.Description); err == sql.ErrNoRows {
if err = row.Scan(&activity.ID, &activity.Time, &activity.Description);
err == sql.ErrNoRows {
log.Printf("Id not found")
return api.Activity{}, ErrIDNotFound
}
Expand Down Expand Up @@ -470,7 +474,8 @@ Now I can add my `-list` endpoint. It follows a similar pattern as Retrieve (`-g

~~~{.go captionb="internal/server/activity.go"}
func (c *Activities) List(offset int) ([]api.Activity, error) {
rows, err := c.db.Query("SELECT * FROM activities WHERE ID > ? ORDER BY id DESC LIMIT 100", offset)
rows, err := c.db.Query(
"SELECT * FROM activities WHERE ID > ? ORDER BY id DESC LIMIT 100", offset)
if err != nil {
return nil, err
}
Expand Down Expand Up @@ -620,7 +625,7 @@ Here it is in GitHub Actions:
<figcaption></figcaption>
</div>

Now my activity service has a persistence layer, and I learned quite a bit about how `database/sql`, `sqlite3`, `sqlite-utils` and `github.com/mattn/go-sqlite3` work. Thank you for coming along on the journey with me. I didn't show all the code changes here, but you can find the [diff](https://github.com/adamgordonbell/cloudservices/commit/9398c7251af9ef3d61a3ac32a5535cb7e71985fb) and the [full code](https://github.com/adamgordonbell/cloudservices/tree/v3-sqlite) on GitHub.
Now my activity service has a persistence layer, and I learned quite a bit about how `database/sql`, `sqlite3`, `sqlite-utils` and `github.com/mattn/go-sqlite3` work. Thank you for coming along on the journey with me. I didn't show all the code changes here, but you can find the [diff](https://github.com/earthly/cloud-services-example/commit/9398c7251af9ef3d61a3ac32a5535cb7e71985fb) and the [full code](https://github.com/earthly/cloud-services-example/tree/v3-sqlite) on GitHub.

## What's Next

Expand Down
14 changes: 7 additions & 7 deletions blog/_posts/2022-02-16-golang-grpc-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ last_modified_at: 2023-09-19
<!-- markdownlint-disable MD036 -->
Welcome back. I'm an experienced developer, learning Golang by building an activity tracker. Last time I added SQLite persistence. Today, I'm going to be porting everything to gRPC.

If you're curious about gRPC – how it works, when to use it, what example code might look like – well, you are in luck because I'm going to be building a grpc client, a grpc server, and the protobuf files for my activity tracker. The full code is on [GitHub](https://github.com/adamgordonbell/cloudservices/tree/v4-grpc).
If you're curious about gRPC – how it works, when to use it, what example code might look like – well, you are in luck because I'm going to be building a grpc client, a grpc server, and the protobuf files for my activity tracker. The full code is on [GitHub](https://github.com/earthly/cloud-services-example/tree/v4-grpc).

## Why gRPC

Expand Down Expand Up @@ -247,8 +247,8 @@ If you recall from when I was adding the `sqlite` feature, Activities handles al

~~~{.diff caption="internal/server/activity.go "}
import
- api "github.com/adamgordonbell/cloudservices/activity-log"
+ api "github.com/adamgordonbell/cloudservices/activity-log/api/v1"
- api "github.com/earthly/cloud-services-example/activity-log"
+ api "github.com/earthly/cloud-services-example/activity-log/api/v1"
+ "google.golang.org/protobuf/types/known/timestamppb"
~~~

Expand Down Expand Up @@ -278,7 +278,7 @@ func (c *Activities) Insert(activity *api.Activity) (int, error) {
}
~~~

And that is the only persistence layer change we need to make to switch from our hand-rolled struct to the `protoc` generated one. Again, you can see the full thing on [github](https://github.com/adamgordonbell/cloudservices/blob/v4-grpc/activity-log/internal/server/activity.go).
And that is the only persistence layer change we need to make to switch from our hand-rolled struct to the `protoc` generated one. Again, you can see the full thing on [github](https://github.com/earthly/cloud-services-example/blob/v4-grpc/activity-log/internal/server/activity.go).

### GRPC Service

Expand Down Expand Up @@ -339,7 +339,7 @@ $ go run cmd/server/main.go
~~~

~~~{.bash .merge-code}
# github.com/adamgordonbell/cloudservices/activity-log/internal/server
# github.com/earthly/cloud-services-example/activity-log/internal/server
internal/server/server.go:30:39: cannot use &srv (type *grpcServer) as type api_v1.Activity_LogServer in argument to api_v1.RegisterActivity_LogServer:
*grpcServer does not implement api_v1.Activity_LogServer (missing api_v1.mustEmbedUnimplementedActivity_LogServer method)
~~~
Expand Down Expand Up @@ -516,7 +516,7 @@ func (s *grpcServer) Insert(ctx context.Context, activity *api.Activity) (*api.I
}
~~~

I can repeat this for [`List` and `Retrieve`](https://github.com/adamgordonbell/cloudservices/blob/v4-grpc/activity-log/internal/server/server.go), and I have a working solution. (Though the error handling has room for improvement. I'll get back to that later on in the article).
I can repeat this for [`List` and `Retrieve`](https://github.com/earthly/cloud-services-example/blob/v4-grpc/activity-log/internal/server/server.go), and I have a working solution. (Though the error handling has room for improvement. I'll get back to that later on in the article).

### Testing A gRPC Server

Expand Down Expand Up @@ -755,7 +755,7 @@ func (c *Activities) Retrieve(ctx context.Context, id int) (*api.Activity, error
}
~~~

And with that implementation [in place](https://github.com/adamgordonbell/cloudservices/blob/v4-grpc/activity-client/internal/client/activity.go), the client works. Here is the Earthly build:
And with that implementation [in place](https://github.com/earthly/cloud-services-example/blob/v4-grpc/activity-client/internal/client/activity.go), the client works. Here is the Earthly build:

<div class="wide">
{% picture content-wide-nocrop {{site.pimages}}{{page.slug}}/3060.png --alt {{ gRPC Client Example Working }} %}
Expand Down
Loading
Loading