Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Jedis and Lettuce benchmarks #1

Closed
wants to merge 34 commits into from
Closed
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
06574ea
Add Jedis and Lettuce benchmarks
jonathanl-bq Sep 7, 2023
5990767
Start ignoring .gradle files
jonathanl-bq Sep 8, 2023
6ae93f5
Update gitignore and remove generated files from git
acarbonetto Oct 3, 2023
8203c4d
Update gitignore and remove generated files from git
acarbonetto Sep 8, 2023
e517744
Update gitignore and remove generated files from git
acarbonetto Sep 8, 2023
2a11e9a
Add benchmarks for GET non-existing
jonathanl-bq Sep 8, 2023
cbb0dcb
Revert "Update gitignore and remove generated files from git"
acarbonetto Sep 8, 2023
5f51a5b
fix redis-rs submodules
acarbonetto Sep 8, 2023
52df672
Randomize commands in Java benchmarks
jonathanl-bq Sep 13, 2023
84f0efc
rename chooseAction to randomAction
jonathanl-bq Sep 14, 2023
1983974
Add a Java benchmarking app (#7)
acarbonetto Sep 20, 2023
6c1fb45
Add Readme and update install_and_test script to runJava
acarbonetto Sep 20, 2023
46d0cf6
Add Readme and update install_and_test script to runJava
acarbonetto Sep 20, 2023
d526f96
Combine java pipeline and java benchmarks (#8)
Yury-Fridlyand Sep 22, 2023
436da8f
Add sync and async clients both to tests. (#12)
Yury-Fridlyand Sep 25, 2023
016f5f6
Move duplicated logic in benchmark JS scripts to a single file, and c…
acarbonetto Sep 25, 2023
e57c1ff
Add dataSize option to java benchmark. (#11)
Yury-Fridlyand Sep 26, 2023
8664d05
Rename jabushka to javababushka (#14)
acarbonetto Sep 26, 2023
b15f93e
Add missing renames. (#17)
Yury-Fridlyand Sep 27, 2023
1bab56a
Add option to run tests on multiple clients in concurrency (#16)
Yury-Fridlyand Oct 5, 2023
d99d27a
Update redis-rs to match main branch
acarbonetto Oct 6, 2023
540f49a
Create clients only once per iteration (#19)
acarbonetto Oct 6, 2023
65090b4
Updated ClientCount to client_count for uniformity.
SanHalacogluImproving Oct 17, 2023
8a0449d
Convert client count to snake case for rust benchmark. (#27)
SanHalacogluImproving Oct 18, 2023
c3d235a
Add json reporting. (#10)
Yury-Fridlyand Oct 20, 2023
fe9bb98
Merge remote-tracking branch 'upstream/main' into java_benchmarks
Yury-Fridlyand Oct 25, 2023
d533b7f
Typo fix.
Yury-Fridlyand Oct 25, 2023
9cbc9c2
Typo fix.
Yury-Fridlyand Oct 25, 2023
b00a205
Merge branch 'java_benchmarks' of github.com:Bit-Quill/babushka into …
Yury-Fridlyand Oct 25, 2023
a62fe92
fix java install_and_test script variables
acarbonetto Nov 6, 2023
30f2f62
Merge branch 'main' into java_benchmarks
acarbonetto Nov 6, 2023
05590b0
Java benchmarks clusters (#34)
acarbonetto Nov 14, 2023
bcf188c
Clean up timer
acarbonetto Nov 14, 2023
b50d57e
Java client jni netty (#32)
Yury-Fridlyand Nov 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .github/workflows/java-benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: Java client benchmarks

on:
workflow_dispatch:
inputs:
name:
required: false
type: string

run-name: ${{ inputs.name == '' && format('{0} @ {1}', github.ref_name, github.sha) || inputs.name }}

jobs:
java-benchmark:
strategy:
matrix:
java:
- 11
- 17
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
submodules: recursive

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
with:
distribution: "temurin"
java-version: ${{ matrix.java }}

- name: Start Redis
run: docker run -p 6379:6379 -p 8001:8001 -d redis/redis-stack

- name: Run benchmarks
working-directory: java
run: ./gradlew :benchmark:run

- name: Upload test reports
if: always()
continue-on-error: true
uses: actions/upload-artifact@v3
with:
name: test-reports-${{ matrix.java }}
path: |
java/benchmarks/build/reports/**
63 changes: 63 additions & 0 deletions .github/workflows/java.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
name: SQL Java CI

on:
pull_request:
push:
paths:
- "java/**"
- ".github/workflows/java.yml"

# Run only most latest job on a branch and cancel previous ones
concurrency:
group: ${{ github.head_ref || github.ref }}
cancel-in-progress: true

jobs:
build-and-test-java-client:
strategy:
# Run all jobs
fail-fast: false
matrix:
java:
- 11
- 17
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v4
with:
submodules: recursive

- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v3
with:
distribution: "temurin"
java-version: ${{ matrix.java }}

- name: Install and run protoc (protobuf)
run: |
sudo apt update
sudo apt install -y protobuf-compiler
mkdir -p java/client/src/main/java/org/babushka/javababushka/generated
protoc -Iprotobuf=babushka-core/src/protobuf/ --java_out=java/client/src/main/java/org/babushka/javababushka/generated babushka-core/src/protobuf/*.proto

- name: Build rust part
working-directory: java
run: cargo build

- name: Start Redis
run: docker run -p 6379:6379 -p 8001:8001 -d redis/redis-stack

- name: Build java part
working-directory: java
run: ./gradlew --continue build

- name: Upload test reports
if: always()
continue-on-error: true
uses: actions/upload-artifact@v3
with:
name: test-reports-${{ matrix.java }}
path: |
java/client/build/reports/**
java/integTest/build/reports/**
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@
debug/
target/

# Git stuff
.worktrees

# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock
Expand All @@ -17,6 +20,14 @@ dump.rdb
.env
benchmarks/results

# IDE generaged files
.vs
.vscode
.idea

# MacOS metadata
.DS_Store

# lock files

yarn.lock
Expand Down
42 changes: 0 additions & 42 deletions .vscode/launch.json

This file was deleted.

37 changes: 0 additions & 37 deletions .vscode/settings.json

This file was deleted.

41 changes: 0 additions & 41 deletions .vscode/tasks.json

This file was deleted.

57 changes: 55 additions & 2 deletions benchmarks/install_and_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,14 @@ runAllBenchmarks=1
runPython=0
runNode=0
runCsharp=0
runJava=0
runRust=0
concurrentTasks="1 10 100 1000"
dataSize="100 4000"
clientCount="1"
chosenClients="all"
host="localhost"
port=6379
tlsFlag="--tls"

function runPythonBenchmark(){
Expand Down Expand Up @@ -68,6 +70,22 @@ function runCSharpBenchmark(){
dotnet run --configuration Release --resultsFile=../$1 --dataSize $2 --concurrentTasks $concurrentTasks --clients $chosenClients --host $host --clientCount $clientCount $tlsFlag $portFlag
}

function runJavaBenchmark(){
cd ${BENCH_FOLDER}/../java
echo "./gradlew run --args=\"--resultsFile=${BENCH_FOLDER}/$1 --clients $chosenClients --host $host --port $port\""
# ./gradlew run --args="--resultsFile=../$1 --dataSize $2 --concurrentTasks $concurrentTasks --clients $chosenClients --host $host --port $port --clientCount $clientCount $tlsFlag"
./gradlew run --args="--resultsFile=${BENCH_FOLDER}/$1 --clients $chosenClients --host $host --port $port"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reminder for

  • --dataSize
  • --concurrentTasks
  • --clientCount
  • $tlsFlag

cd ${BENCH_FOLDER}/java
}

function runJavaBenchmark(){
cd ${BENCH_FOLDER}/../java
echo "./gradlew run --args=\"--resultsFile=${BENCH_FOLDER}/$1 --clients $chosenClients --host $host --port $port\""
# ./gradlew run --args="--resultsFile=../$1 --dataSize $2 --concurrentTasks $concurrentTasks --clients $chosenClients --host $host --port $port --clientCount $clientCount $tlsFlag"
./gradlew run --args="--resultsFile=${BENCH_FOLDER}/$1 --clients $chosenClients --host $host --port $port"
cd ${BENCH_FOLDER}/java
}

function runRustBenchmark(){
rustConcurrentTasks=
for value in $concurrentTasks
Expand Down Expand Up @@ -185,6 +203,36 @@ do
runAllBenchmarks=0
runNode=1
;;
-java)
runAllBenchmarks=0
runJava=1
chosenClients="Babushka"
;;
-lettuce)
runAllBenchmarks=0
runJava=1
chosenClients="Lettuce"
;;
-lettuce)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
-lettuce)
-jedis)

runAllBenchmarks=0
runJava=1
chosenClients="Jedis"
;;
-java)
runAllBenchmarks=0
runJava=1
chosenClients="Babushka"
;;
-lettuce)
runAllBenchmarks=0
runJava=1
chosenClients="Lettuce"
;;
-lettuce)
runAllBenchmarks=0
runJava=1
chosenClients="Jedis"
;;
-csharp)
runAllBenchmarks=0
runCsharp=1
Expand Down Expand Up @@ -242,6 +290,13 @@ do
runCSharpBenchmark $csharpResults $currentDataSize
fi

if [ $runAllBenchmarks == 1 ] || [ $runJava == 1 ];
then
javaResults=$(resultFileName java $currentDataSize)
resultFiles+=$javaResults" "
runJavaBenchmark $javaResults $currentDataSize
fi

if [ $runAllBenchmarks == 1 ] || [ $runRust == 1 ];
then
rustResults=$(resultFileName rust $currentDataSize)
Expand All @@ -250,8 +305,6 @@ do
fi
done



flushDB

if [ $writeResultsCSV == 1 ];
Expand Down
3 changes: 3 additions & 0 deletions java/.cargo/config.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[env]
BABUSHKA_NAME = { value = "BabushkaPy", force = true }

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rename

BABUSHKA_VERSION = "0.1.0"
8 changes: 8 additions & 0 deletions java/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Ignore Gradle project-specific cache directory
.gradle

# Ignore Gradle build output directory
build

# Ignore generated files (e.g. protobuf)
generated
21 changes: 21 additions & 0 deletions java/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[package]
name = "javababushka"
version = "0.0.0"
edition = "2021"
license = "BSD-3-Clause"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[lib]
name = "javababushka"
crate-type = ["cdylib"]

[dependencies]
redis = { path = "../submodules/redis-rs/redis", features = ["aio", "tokio-comp", "connection-manager", "tls", "tokio-rustls-comp"] }
babushka = { path = "../babushka-core" }
tokio = { version = "^1", features = ["rt", "macros", "rt-multi-thread", "time"] }
logger_core = {path = "../logger_core"}
tracing-subscriber = "0.3.16"

[profile.release]
lto = true
debug = true
Loading
Loading