Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/implement redis #651

Merged
merged 44 commits into from
Nov 18, 2024
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
637baf8
Add last updated queries
ja573 Nov 13, 2024
50aeb38
Allow copying timestamps
ja573 Nov 13, 2024
ad55c3b
Implement parsing timestamps
ja573 Nov 13, 2024
3053e4d
Add redis
ja573 Nov 13, 2024
49c6b9b
Add redis
ja573 Nov 13, 2024
300714a
Add chrono and redis errors
ja573 Nov 13, 2024
5c865ac
Implement deadpool redis
ja573 Nov 14, 2024
343ee92
Implement deadpool redis
ja573 Nov 14, 2024
9a8500c
Implement deadpool redis
ja573 Nov 14, 2024
b0a4754
Timestamp now implements Copy
ja573 Nov 14, 2024
b74ffe2
Update changelog
ja573 Nov 14, 2024
72f2d30
Add redis pool to server
ja573 Nov 14, 2024
3206aa8
Update changelog
ja573 Nov 14, 2024
2da19d2
Update changelog
ja573 Nov 14, 2024
f40d6ed
Implement Copy for all enums
ja573 Nov 14, 2024
b1d319c
Implement Copy for all enums
ja573 Nov 14, 2024
0cd85b1
WIP Implement redis caching in export api
ja573 Nov 14, 2024
aeb07b6
Update doc tests
ja573 Nov 14, 2024
8488785
Implement caching in metadata record
ja573 Nov 14, 2024
a05a3b1
Merge branch 'develop' into feature/implement_redis
ja573 Nov 14, 2024
a85c7aa
Update changelog
ja573 Nov 14, 2024
8ea6258
Test timestamp parsing
ja573 Nov 15, 2024
ce47825
Test redis module
ja573 Nov 15, 2024
25cc97e
Test redis module
ja573 Nov 15, 2024
944c920
Convert connection errors to thoth
ja573 Nov 15, 2024
c346feb
Restructure database functions
ja573 Nov 15, 2024
e5b28d7
Refactor db module
ja573 Nov 15, 2024
adc03c8
Update changelog
ja573 Nov 15, 2024
2a68c06
Refactor db module
ja573 Nov 15, 2024
825217c
Handle errors idiomatically
ja573 Nov 15, 2024
eea8db3
Handle errors idiomatically
ja573 Nov 15, 2024
5a5a6c1
Handle errors idiomatically
ja573 Nov 15, 2024
12cf4bc
Handle errors idiomatically
ja573 Nov 15, 2024
6c009f0
Set redis url from arg
ja573 Nov 15, 2024
ba70a39
Get last updated for published books only
ja573 Nov 15, 2024
59baa85
Merge branch 'develop' into feature/implement_redis
ja573 Nov 18, 2024
419f0bd
Transfer copyright to Thoth
ja573 Nov 18, 2024
8921c9b
Remove travis badge
ja573 Nov 18, 2024
ac628e4
Merge branch 'feature/implement_redis' of github.com:thoth-pub/thoth …
ja573 Nov 18, 2024
23fddee
Replace logo image
ja573 Nov 18, 2024
bd999f3
Use github container registry
ja573 Nov 18, 2024
fe45f43
Update changelog
ja573 Nov 18, 2024
917e1ec
Fix variable name
ja573 Nov 18, 2024
432f2fa
Reinstate commnet
ja573 Nov 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ THOTH_EXPORT_API=http://localhost:8181
THOTH_DOMAIN=localhost
# Full postgres URL
DATABASE_URL=postgres://thoth:thoth@localhost/thoth
# Full redis URL
REDIS_URL=redis://localhost:6379
# Authentication cookie secret key
SECRET_KEY=an_up_to_255_bytes_random_key
# Logging level
Expand Down
11 changes: 11 additions & 0 deletions .github/workflows/build_test_and_check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ env:
CARGO_TERM_COLOR: always
THOTH_GRAPHQL_API: https://api.thoth.pub
THOTH_EXPORT_API: https://export.thoth.pub
TEST_REDIS_URL: redis://localhost:6379

jobs:
build:
Expand All @@ -46,6 +47,16 @@ jobs:
run: cargo build -vv
test:
runs-on: ubuntu-latest
services:
redis:
image: redis:alpine
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 6379:6379
steps:
- uses: actions/checkout@v4
- uses: actions/cache@v4
Expand Down
14 changes: 9 additions & 5 deletions .github/workflows/docker_build_and_push_to_dockerhub.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
pull_request:
workflow_dispatch:

env:
REGISTRY: ghcr.io

jobs:
build_and_push_staging_docker_image:
runs-on: ubuntu-latest
Expand All @@ -16,19 +19,20 @@ jobs:
with:
# list of Docker images to use as base name for tags
images: |
openbookpublishers/thoth
${{ env.REGISTRY }}/thoth-pub/thoth
# generate Docker tags based on the following events/attributes
tags: |
type=ref,event=pr,prefix=staging-pr-
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
- name: Login to Container registry
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
Expand All @@ -54,7 +58,7 @@ jobs:
uses: docker/build-push-action@v5
with:
push: false
tags: openbookpublishers/thoth:latest
tags: thoth-pub/thoth:latest
file: Dockerfile.dev
build-args: |
THOTH_GRAPHQL_API=https://api.thoth.pub
Expand Down
12 changes: 8 additions & 4 deletions .github/workflows/docker_build_and_push_to_dockerhub_release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@ on:
release:
types: [published]

env:
REGISTRY: ghcr.io

jobs:
build_and_push_docker_image:
runs-on: ubuntu-latest
Expand All @@ -16,7 +19,7 @@ jobs:
with:
# list of Docker images to use as base name for tags
images: |
openbookpublishers/thoth
${{ env.REGISTRY }}/thoth-pub/thoth
# generate Docker tags based on the following events/attributes
tags: |
type=semver,pattern={{version}}
Expand All @@ -26,11 +29,12 @@ jobs:
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to DockerHub
- name: Login to Container registry
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: docker_build
uses: docker/build-push-action@v5
Expand Down
12 changes: 11 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,17 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]
### Added
- [648](https://github.com/thoth-pub/thoth/issues/648) - Added new `LocationPlatform`, `THOTH`, for Locations where file is hosted directly by Thoth on S3.
- [651](https://github.com/thoth-pub/thoth/pull/651) - Implement Redis connection pools using `deadpool-redis`
- [651](https://github.com/thoth-pub/thoth/pull/651) - Implement Redis caching in export API
- [651](https://github.com/thoth-pub/thoth/pull/651) - Added `WorkLastUpdatedQuery` and `WorksLastUpdatedQuery` queries to thoth-client
- [651](https://github.com/thoth-pub/thoth/pull/651) - Implement `Copy`, `Ord` and `PartialOrd` traits for `Timestamp`
- [651](https://github.com/thoth-pub/thoth/pull/651) - Implement parsing from and to RFC 3339 strings for `Timestamp`
- [651](https://github.com/thoth-pub/thoth/pull/651) - Implement `Copy` trait for `WorkType`, `WorkStatus`, `PublicationType`, `CountryCode`, `LanguageRelation`, `LanguageCode`, `LocationPlatform`, `LengthUnit`, `WeightUnit`, `CurrencyCode`, and `SeriesType`
- [651](https://github.com/thoth-pub/thoth/pull/651) - Allow supplying `DATABASE_URL` as binary argument
- [648](https://github.com/thoth-pub/thoth/issues/648) - Added new `LocationPlatform`, `THOTH`, for Locations where file is hosted directly by Thoth on S3.

### Changed
- [651](https://github.com/thoth-pub/thoth/pull/651) - Use Github Container registry instead of DockerHub

### Fixed
- [631](https://github.com/thoth-pub/thoth/issues/631) - Fix slow loading of Contributor dropdown in Contribution form
Expand Down
104 changes: 100 additions & 4 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 1 addition & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
[package]
name = "thoth"
version = "0.12.14"
authors = ["Javier Arias <javi@openbookpublishers.com>", "Ross Higman <ross@openbookpublishers.com>"]
authors = ["Javier Arias <javi@thoth.pub>", "Ross Higman <ross@thoth.pub>"]
edition = "2021"
license = "Apache-2.0"
description = "GraphQL API for bibliographic data"
repository = "https://github.com/thoth-pub/thoth"
readme = "README.md"

[badges]
travis-ci = { repository = "openbookpublishers/thoth" }
maintenance = { status = "actively-developed" }

[workspace]
Expand Down
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright 2020 Open Book Publishers
Copyright 2020 Thoth Open Metadata

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
4 changes: 4 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
docker-dev-build \
docker-dev-run \
docker-dev-db \
docker-dev-redis \
build \
test \
clippy \
Expand Down Expand Up @@ -44,6 +45,9 @@ docker-dev-run:
docker-dev-db:
docker compose -f docker-compose.dev.yml up db

docker-dev-redis:
docker compose -f docker-compose.dev.yml up redis

build:
cargo build -vv

Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<div align="center">
<img src="https://cdn.openbookpublishers.com/thoth_logo_1141bc9a6b.png" height="400" />
<img src="https://cdn.thoth.pub/thoth_logo.png" height="400" />

<h1>Thoth</h1>

Expand Down Expand Up @@ -127,7 +127,7 @@ The wasm APP needs to know the endpoint the API will be running at compile time,
docker build \
--build-arg THOTH_GRAPHQL_API=https://api.thoth.pub \
--build-arg THOTH_EXPORT_API=https://export.thoth.pub \
. -t openbookpublishers/thoth
. -t thoth-pub/thoth
```

## Acknowledgements
Expand Down
6 changes: 6 additions & 0 deletions docker-compose.dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ services:
env_file:
- .env

redis:
image: redis:alpine
container_name: "thoth_redis"
ports:
- "6379:6379"

graphql-api:
build:
context: .
Expand Down
Loading
Loading