Canvas for Backend Technical Test at Scalingo (THEO BELOEIL GUIA)

Before running the project

• We need a github token to run the project, this is needed because we crush github api rate limit quickly using go multithreading (we have only 60 requests per hour beeing unauthenticated).

• To get a token, go to https://github.com/settings/tokens and generate a token with the public_repo scope.

This token must be used in any request to the API (in the Authorization header). Authorization: Bearer your_token

Project requirements

• 🟢 Use Go
• 🟢 Use Docker
• 🟢 Use concurrency
• 🟢 Fetch repositories from the github API based on language and multiple filters
• 🟢 Fetch requested language data of each repository
• 🟢 Return the aggregated data in the response

Points to consider

• The request MUST contain a language query parameter, this is the point to me, do a search public repositories on a specific language.
• The response IS paginated, 100 repositories per page (but can be less, per_page parameter)

Usage of API

The API uses the same filters as the one used in github API. https://docs.github.com/fr/search-github/searching-on-github/searching-for-repositories

a simple query looks like this:

http://localhost:5000/repos?q=language:javascript+size:1..10

• a filter is a key:value pair, the key is the filter name and the value is a combination of operator + value (can be a range, a number, a date, etc combined with a : or <: or >: etc...).
• the + character is used to separate filters.

I did not implement all filters, but most of them are supported, the rest can be implemented easily Filters available.

⚠️ I did not implement a check of license and language filters validity, so if you provide a wrong license of language filter, it will return a poor error message. I should the fetch the data from github to check if the license or language is valid and make a proper error message. (or create a map of valid licenses and languages but this is hacky and won't allow us to scale accordingly with the api)⚠️

The APi offers only one endpoint:

• GET /repos

Filter Support

The query parameters are:

• q - the query string

---

• q is the combination of the following filters

• size - 1..10||>=10||<=10||:20

• topics - 1..10||>=10||<=10||:20

• stars - 1..10||>=10||<=10||:20

• followers - 1..10||>=10||<=10||:20

• forks - 1..10||>=10||<=10||:20

• license - MIT||GPL||BSD

• language - javascript || python || go || rust

• created - >=2024-01-01||<=2024-01-01||:2024-01-01

• pushed - >=2024-01-01||<=2024-01-01||:2024-01-01

---

optional (default to 100)

• per_page - number of items per page (default: 100, max 100)

---

optional (default to 1)

• page - number of the page (default: 1)

Examples

• search public repositories with the word scalingo (in name, description or topics) and the language javascript and the size of the repository is between 1 and 10 KB
  • http://localhost:5000/repos?q=scalingo+language:javascript+size:1..10
• search public repositories with the language javascript and the size of the repository is smaller or equal to 100 KB
  • <http://localhost:5000/repos?q=language:javascript+size:<=100>
• search public repositories with the language go and the followers is 10 and stars is greater than 10
  • http://localhost:5000/repos?q=language:go+followers:10+stars:10>
• search public repositories with the language rust and the size of the repository is between 1 and 10 KB and the number of stars is 10 and the number of followers is greater or equal to 100
  • http://localhost:5000/repos?q=language:rust+size:1..10+stars:10+followers:=100>
• search public repositories with the language ruston page 2 with 45 items per request
  • http://localhost:5000/repos?q=language:rust&per_page=45&page=2

⚠️ Do not forget to add the token in the Authorization header. ⚠️

Testing

all the code is tested, and we get close to 100% coverage. Each layer are tested using mocks for "childs" layer but the repository layer should be tested with a real infra environment but this takes time so i did a naive testing approach -> testing my code only

Project structure

I do use the clean architecture pattern, so the project is divided into 4 layers in the src folder:

• controllers contains the handlers for the API endpoints.
• repositories contains the implementation of the repository interface (the data layer, infra, db ...).
• usecases contains the business logic of the API (validate of query mainly).
• models contains the data models of the API (the models of the data in repository).

Each layer can access to the inner layers but not the opposite.

You can find the entry point of the API in main.go.

Clean architecture is a pattern that helps to separate the concerns of the application, it helps to make the code more testable and more maintainable, allowing to write unit tests easily and perform mocking easily. This pattern is usefull for separation of concerns, it helps to make the code more modular and easier to maintain.