✔️ Discover and use an ORM
✔️ Understand the Go interface system
✔️ Implement a clean architecture
⚠️ This half-day is particularly long and contains a lot of concepts to explore.
Here are some tips for you to advance as far as possible:
- Be sure to read the statements and concepts covered in the steps
- Read the documentation provided in each exercise
- Don't stay stuck for a long time if you encounter a problem, quickly ask the staff for help 😄
Estimated time: 2 minutes
- In the
day02folder of your pool directory, create a folderORM.
mkdir -p day02/ORM- Initialize a
SoftwareGoDay2module.
Estimated time: 10 minutes
This morning, you learned to use SQL to insert or read data, sort these data, linking certain data to each other, etc.
You have most likely noticed two important problems:
-
Writing and executing these queries by hand is a tedious task 🙁
-
When you wanted to search for artists by type of music, you had to write the raw type in your query.
But how to use the same kind of request for another model, especially if it is unknown in advance?
In short, the queries were not dynamic.
Apart from maintenance or debugging needs, a developer very rarely uses SQL commands in CRUD.
You will therefore have to create and interact again with a database containing different tables:
RecordCompany: publishers.Artist: artists.Contact: contact sheets.Music: music.
Here's a schema to clear your mind:
But unlike this morning, you're not going to use any SQL commands or queries, but learn how to use an ORM which makes these SQL requests for you, directly from your Go code.
When a developer writes an application that uses an SQL database, he must make an important choice regarding how to communicate with this database:
RawSQL client: A client that performs dynamic queries very close to the basic syntax.Query Builder: A tool to build queries more easily, using the language of the application.ORM: A tool to abstain from the SQL query system.
Note:
As usual in the development world, each method has pros and cons, depending on the application's needs.
You can learn more about this with this article.
Since you now know how to make SQL queries by hand, and for simplicity reasons, you will use the 3rd method, an ORM
Estimated time: 30 minutes
What is an ORM?
"Object-relational mappers", or ORM, are software dedicated to translation between data representations in relational databases and in-memory representation used with object-oriented programming (OOP).
The ORM provides an object-oriented interface to the database data, trying to use familiar programming concepts and reduce the amount of code needed to accelerate development.
In general, ORMs serve as an abstraction layer to help developers work with databases without radically changing the object-oriented paradigm.
This can be useful to reduce the mental load of adapting to the specificities of a database's storage format.
Don't worry, you will gradually discover what it means by practicing 🚀
The ORM you will use today is called ENT.
It's one of Go's ORM by excellence, it's very practical and well documented.
But that's not all.
Indeed, ENT as a specificity compared to others, which is to generate code for you.
ENT allows you to quickly set up the CRUD for your database according to schemas that you'll define directly in your Go code 🤩
Is it really useful to automatically generate the CRUD?
It depends on your use case, but if you want to quickly manipulate a database without edge effects (that is to say in a classic way), then yes it is useful! Imagine the time that you'll win!
This exercise will therefore be carried out in four parts:
- Configuring your PostgreSQL database
- Installation of ENT
- Creation of your first schema (data model)
- Connect to the database with ENT
Even before implementing the ORM, we must created the database we'll use with this ORM.
Like this morning, we will be using a PostgreSQL database.
Create a .env file in which you will put the environment variables related to your db:
DB_USER: the username of your database.DB_PASS: your user's password for your database.DB_HOST: the host to connect.You must put
localhosthere.DB_PORT: the listening port of your database.DB_NAME: the name of your database.DB_URL: the login url, it groups all the above information.You must put
postgresql://$DB_USER:$DB_PASS@$DB_HOST:$DB_PORT/$DB_NAME
To create a new database, simply execute the following command:
docker run --name [DB_NAME] -e POSTGRES_PASSWORD=[DB_PASS] -e POSTGRES_USER=[DB_USER] -e POSTGRES_DB=[DB_NAME] -p [DB_PORT]:[DB_PORT] -d postgres:alpineOr :
source .env
docker run --name $DB_NAME -e POSTGRES_PASSWORD=$DB_PASS -e POSTGRES_USER=$DB_USER -e POSTGRES_DB=$DB_NAME -p $DB_PORT:$DB_PORT -d postgres:alpine💡 It's possible to create several databases in a single postgres instance, that's why each db is given a name.
Install ENT with the command:
go get -d entgo.io/ent/cmd/ent-
Run the command
go run entgo.io/ent/cmd/ent init Artist.A folder will be generated in which you will find the
schema/artist.gofile.This file corresponds to the schema that will define your model and create a db table according to this model.
-
Modify the
Artistmodel to have thenameandnationalityattributes.💡 For ENT:
-
Run the
go generate ./entcommand to start generating ENT code.You will see many folders and files appear, here are some particularly interesting to know:
ent/client.go: The ENT client for your database.ent/artist.go: The template of your Artist defined by your Artist schema.ent/artist_create.go: Functions/methods for creating an artistent/artist_query.go: Functions/methods that allow you to query an artistent/artist_update.go: The functions/methods that allow you to modify an artistent/artist_delete.go: The functions/methods that allow you to delete an artist
-
Still in the
schema/artist.gofile, add theidfield to your schema so that ENT uses a UUID instead of anint🙂In a relational database, each table contains a unique identifier, so that each element of that table can be distinguished even if their data are the same.
In our case, this identifier isid.ENT generates this identifier automatically, so no need to create it! It is also common for security to use a
uuidrather than a simple index.ENT has an integrated function specifically for this purpose. More details.
- Create the database package, where we will manage our connection and calls to our database.
- Create the
database/database.go.- Create a
Databasestructure containing the following fields:Client, a pointer to an ENT client (*ent.Client).- Any field that seems useful to you (URL of the database, named logger...)
- Create a
NewDatabasefunction that returns a completely initialized Database and an error in case of failure
- Create a
- With ENT, open the database, initialize the connection and synchronize the schemas to create the tables in your database.
- Create the
- Create the file
main.go.- Retrieve your environment variables needed to initialize your database.
- Call the
NewDatabasefunction of thedatabasepackage:- If successful, write:
Database is ready. - If the operation fails, write:
Failed to initialize database:followed by the error message.
- If successful, write:
You can also connect to your database with DataGrip and see your newly created table 🚀
- ENT - Your First Schema
- ENT - Connect to PostgreSQL
- ENT - Fields
- ENT - Edges
- ENT - ID Field
- Go - Load a .env file automatically
- Go - Retrieve environment variables
Estimated time: 45 minutes
⚠️ Respect EXACTLY the prototypes of the methods given to you.
Now it's time to develop the functions to read, add, modify and delete an artist. In other words, the CRUD 💥
The code generated by ENT provides you with many methods and functions that allow you to interact with your database:
💡 For this exercise, assume that the parameters sent to you are valid.
In the database folder, create a file artist.go.
Then inside it you can add a CreateArtist method on the Database type that takes as parameters a context and the attributes of
the Artist:
name: the name of the artistnationality: the origin country of the artist
package database
func (d Database) CreateArtist(ctx context.Context, name, nationality string) (*ent.Artist, error) {
... = d.Client...
return ...
}The method must create a new artist and return it once saved in the db.
You can test it by calling it directly from the
mainfunction 😉
Create a GetArtists method on the Database type that returns all the Artist saved in db.
package database
func (d Database) GetArtists(ctx context.Context) ([]*ent.Artist, error) {
... = d.Client...
return ...
}Then you can also add a GetArtistByID method that takes an uuid as parameter and returns the artist that matches this id.
package database
import "github.com/google/uuid"
func (d Database) GetArtistByID(ctx context.Context, id uuid.UUID) (*ent.Artist, error) {
... = d.Client...
return ...
}If the artist doesn't exist, you must return an error ❌
Create an UpdateArtist method that takes this parameter:
artist: a pointer to the already modified artist whose changes must be saved in the database.
package database
func (d Database) UpdateArtist(ctx context.Context, artist *ent.Artist) (*ent.Artist, error) {
... = d.Client...
return ...
}It must save the modified artist in your database and then send it back.
If the artist doesn't exist, you must return an error ❌
Create a DeleteArtist method that takes an id parameter and removes the selected artist.
package database
import "github.com/google/uuid"
func (d Database) DeleteArtist(ctx context.Context, id uuid.UUID) (*ent.Artist, error) {
... = d.Client...
return ...
}If the artist does not exist, you must return an error.
⚠️ Make sure that you have respected the prototype of the methods given above.
Testing is important, even more so when it comes to resource manipulation. If something is broken in it, your whole app will be broken.
You're lucky, we've prepared a whole bunch of tests for you!
For each step, you should have a tests/ folder to place in your app.
You can run them with the following command:
go test ./tests
# ok SoftwareGoDay2/tests 0.069sEstimated time: 45 minutes
It is essential for a developer to break down their application into different logical parts that will have a different responsibility.
This is called the Separation of Concern.There are several design patterns to split your application:
- MVC
- Domain Driven Design
- Clean Architecture
- ...
Like yesterday, you will implement a part of the MVC design pattern to introduce you to several concepts.
This architecture will be divided into two layers:
- The
databaselayer will contain the unit functions related to the database. (Units do one thing and only one 😉).
You have just implemented it in the previous exercise 🎉
- The
controllerlayer will contain the functions that will control your logic and orchestrate the methods of the database to perform your logic.
Like yesterday, you will use a package controller in which you will store all the functions/methods to interact with the database.
However, we will use it a little differently 👀
There are many business cases where the logic checks need to be performed before executing an action.
Let's take an example:
In a company, a manager of a team requests access to another user's personal information.
However, the company has decided that a manager is allowed to access only the information of his team members, who are under his responsibility.
Before returning the requested information, we must therefore verify:
- If the user requesting the information is indeed a Manager, that is to say that his "Role" is allowed to see the information of other users.
- If the employee requested is in the team of this manager, that is to say in the perimeter of access/vision of this manager.
- If the employee is the manager's responsibility.
When do we need to verify this information, in other words, make a logic?
Your database layer is only responsible for creating, retrieving, modifying or deleting database data. Which means only interact with it.
Its operation can become very complex as your application grows over time.
As you may have noticed, your database layer is completely linked to ENT. Imagine that you have implemented your logic in this layer.
If one day, for performance or comfort, you no longer use ENT but another ORM, you should rewrite all the code containing your logic 😦
This doesn't follow the SOLID principles.
It will therefore be your controller's responsibility to perform your logic using the functions of your database layer.
As this day is already very long, you will only implement a very basic logic: the validation of the received data 😄
Create a controller package and write the constructor below in the controller/controller.go file.
package controller
import "SoftwareGoDay2/database"
type Controller struct {
*database. Database
// Add some fields if necessary
}
func NewController(db *database.Database) *Controller {
// Return a pointer to a `Controller` struct filled with the database
return &Controller{Database: db}
}This type Controller will be used to implement your logic using the methods of your type Database that you give it as a parameter when using the constructor.
Create a new file controller/artist.go
Create a CreateArtist method on the Controller type that takes a context and the attributes of the Artist:
namenationality
package controller
import (
"context"
"github.com/pkg/errors"
)
var InvalidArtistName = errors.New("artist name is invalid")
func (c Controller) CreateArtist(ctx context.Context, name, nationality string) (*ent.Artist, error) {
... = c.Database.CreateArtist(...)
return ...
}When a developer creates a new feature that interacts with a database, he must ask himself the following questions:
- What data is absolutely necessary for my model (here:
Artist) to be considered valid and instantiated in the DB?- In what form/state may the information will be transmitted to me and in what cases could it invalidate my model?
Let's say an Artist must have a non-empty name and their nationality could be unknown
The method must verify that the Artist's name is correct and return the InvalidArtistName error if not.
Create a GetArtists method on the Controller type that returns the list of all Artist saved in the database.
package controller
func (c Controller) GetArtists(ctx context.Context) ([]*ent.Artist, error) {
...
}Create a GetArtistByID method on the Controller type that takes an id as parameter and returns an artist if its id matches the one
given as parameter.
package controller
func (c Controller) GetArtistByID(ctx context.Context, id string) (*ent.Artist, error) {
...
}The method must verify that the Artist's id sent as a string is a UUID and transform it to send it to the Database method.
Create an UpdateArtist method on the Controller type that takes the following parameters:
id: the identifier of the artist to be modifiedname: the name of the artist to be modified (can be blank)nationality: the nationality of the artist to be modified (may be empty)
controller package
func (c Controller) UpdateArtist(ctx context.Context, id, name, nationality string) (*ent.Artist, error) {
...
}It must:
- Retrieve the artist from the database
- Modify artist attributes that are not invalid and are different from the recovered artist
- Save the changes in the db and return the modified artist
Create a DeleteArtist method on the Controller type that takes an id parameter and removes the selected artist.
controller package
func (c Controller) DeleteArtist(ctx context.Context, id string) (*ent.Artist, error) {
...
}Now you know how to:
- Basically use the CRUD of an ORM for a model
- Organize your code for more flexibility
- Test your logic and your database
Congratulations, that already really cool 🥳
In the relational database, there are 3 types of relationships:
- One to One: One entity related to another
Example: AnArtisthas only one contact and a contact will only serve an artist - One to Many: An entity that can be linked to multiple copies of another entity
Example: aRecordCompanycan produce multiple artists - Many to Many: Several entities linked to several other entities of another table
Example: AnArtistcan write several musics and a music can be created by several artists in collaboration.
Here's a schema to illustrate these relationships:
To create these relationships with ENT, you will need to declare edges that will serve as a link between your models.
The database at the end of this exercise will look like this:
Artist:
idNameNationalityContact
Contact:
idPhoneEmail
-
Create a new
Contacttemplate:go run entgo.io/ent/cmd/ent init Contact
-
Give the following attributes to your
Contactmodel:- id (uuid)
- email (string)
- phone (string)
-
Create a relationship from your
Artistmodel to aContact(the relationship must beUnique) -
Create the reverse relationship from your
Contactmodel to anArtist(the relationship must beUnique)
Resource: OneToOne relationship with ENT
You will need to modify your CRUD to support operations on your new model.
As for your artists, create 2 new files:
database/contact.gowhich will contain all your Database methods that will interact with ENT for theContacttype.controller/contact.gowhich will contain all your methods of theControllertype which will implement your logic for theContacttype.
Following the CRUD example on artists, implement/modify the methods for your Database type and your Controller type 😄
Create the CreateContact methods that take the following parameters:
context: A context for ENTartistID: the unique number (uuid) of the artist to whom this contact is attachedphone: The contact's telephone number.email: The email of the contact.
This method must create a contact and binds it to the Artist whose ID is passed as parameter. If the artist does not exist, it must return an error.
A contact being completely linked to an artist, it doesn't make sense to create a method to recover a contact.
Change the GetArtistByID and GetArtists methods of your type to refer to artists with their contact.
💡 You don't need to modify the prototype of these methods
An artist's contact will be loaded into the Artist.Edges.Contact field.
Resource: Eager Loading with ENT
Create the UpdateContact method on the Controller type that takes the following parameters:
context: A context for ENTid: the artist ID to be modifiedphone: The contact's phone number (could be empty/invalid).email: Contact email (could be empty/invalid).
Create the DeleteContact methods that take the following parameters:
context: A context for ENTid: the identifier of the contact to be deleted
If the contact does not exist, you must return an error ❌
You have just implemented your first relationship on OneToOne artists 🥳
Now it's time to produce your artist, through a Music Label 😄
This time, this relationship will be of the OneToMany type
At the end of this exercise, the database will look like this:
Artist:
idNameNationalityContactRecordCompany
Contact:
idPhoneEmail
RecordCompany:
-
Name -
Artists -
Create a new RecordCompany template:
go run entgo.io/ent/cmd/ent init RecordCompany
-
Give your RecordCompany template the following attributes:
- id
- name
-
Create a relationship from your RecordCompany template to an Artist
-
Create the reverse relationship from your
Artist(From) model to aRecordCompany(the relationship must beUnique)
Resource: OneToMany relationship with ENT
You begin to understand the principle, create 2 new files:
database/record_company.gowhich will contain all your Database methods that will interact with ENT for theRecordCompanytype.controller/record_company.gowhich will contain all your Controller methods that will implement your logic for theRecordCompanytype.
Add the CreateRecordCompany method that take the following parameters:
context: A context for ENTartistID: the unique number (uuid) of the artist to whom this label is attachedphone: The phone number of the label.email: The email of the label.
This method must create a label and binds it to the Artist whose ID is passed as parameter.
Create the GetRecordCompanies and GetRecordCompanyByID methods that also reference all the artists associated with these labels.
Don't forget to take a look at Eager Loading with ENT 😉
Create the UpdateRecordCompany method on the Controller type that takes the following parameters:
context: A context for ENTid: the label identifier to be modifiedname: The label name (can be empty/invalid).
Create the UpdateRecordCompany method on the Database type that takes the following parameters:
context: A context for ENTlabel: a pointer to the already modified label whose changes must be saved in database.
If the label does not exist, you must return an error ❌
Create the DeleteRecordCompany method that takes the following parameters:
context: A context for ENTid: the label identifier to be deleted
If the label does not exist, you must return an error
Create an AddArtistToRecordCompany method.
It must take as parameters:
artistID: Artist identifierrecordCompanyID: Record company identifier
Your function will connect these entities by following their ID.
If one of them doesn't exist, return an error ❌
Create a RemoveArtistFromRecordCompany method.
It must take as parameters:
artistID: Artist identifierrecordCompanyID: Record company identifier
Your function will disconnect these entities according to their identifier.
If one of them does not exist, return an error.
Perhaps you have heard about various arguments? Wouldn't it be practical to give a list of artists to add instead of one by one? Ent give you access to many functions... 😉
Finally, we are near the end! There is only one entity left: Music 😄
And only one relationship type remains: the ManyToMany.
This step is intentionally less guided, if you have reached it, you should have understood everything about relational databases, CRUD and how you should code it.
The final database will look like this:
Artist:
idNameNationalityContactRecordCompanyMusics
Contact:
idPhoneEmail
RecordCompany:
NameArtists
Music:
NameLinkArtists*### 📌 Task Create theMusictemplate with ENT.
It will consist of the following properties:
id: unique identifiername: name of musiclink: public link to the music (it could be whatever you want, like YouTube, Spotify...)artists: Artists who collaborated for this music
You will need to create a ManyToMany relationship between Artist and Music.
Remember to update other templates if necessary 😉
We want to:
- create
- read
- updating
- delete
- Link an artist to a music
- Removing an artist from a music
Don't forget to handle errors 😄
Resources
Congratulations for completing this day! This one was particularly long, we're really proud of you 🥳
If you still have some energy though, here are some bonuses for you 😃
Yesterday, you created a cool application following a strong and resilient architecture called MVC.
An advantage of MVC is his layered architecture, you can easily update a part of your application without refactor everything.
If you remembered well, your data storage was a simple JSON file, what
about replacing it if a relational database?
You know how MVC works, you know how to store data in a relational database, you know an easy way to interact with database directly from your code...
Let's mix all your knowledge to add a permanent storage to your application 🚀
Indeed, you are free to add/update models, refactor the application and do whatever you find useful.
Have fun!
You've noticed that your architecture has become more complex.
If this continues, you risk, when modifying a feature, to break other essential functionalities.
Now that you have used the given tests, it may be the time to implement your own!
Take a look at the tests for each steps and try to improve them 😄
If you want to learn more about databases, here are some interesting links:
🚀 Don't hesitate to follow us on our different networks, and put a star 🌟 on
PoC'srepositories.