Skip to content

Commit

Permalink
Enable nightly join fuzzer job to verify against Presto (facebookincu…
Browse files Browse the repository at this point in the history
…bator#10663)

Summary:
Pull Request resolved: facebookincubator#10663

Make the nightly join fuzzer job on GitHub Actions to verifiy Velox results against Presto.

Reviewed By: kgpai

Differential Revision: D60768415

fbshipit-source-id: d88de670b6be2a06b3a99837a7b5a5f730d59f9b
  • Loading branch information
kagamiori authored and facebook-github-bot committed Sep 9, 2024
1 parent 24dd2e9 commit cc10dc2
Showing 1 changed file with 30 additions and 4 deletions.
34 changes: 30 additions & 4 deletions .github/workflows/scheduled.yml
Original file line number Diff line number Diff line change
Expand Up @@ -573,21 +573,44 @@ jobs:
path: |
/tmp/spark_fuzzer_repro
join-fuzzer-run:
presto-java-join-fuzzer-run:
name: Join Fuzzer
runs-on: ubuntu-latest
container: ghcr.io/facebookincubator/velox-dev:centos9
container: ghcr.io/facebookincubator/velox-dev:presto-java
needs: compile
timeout-minutes: 120
env:
CCACHE_DIR: "${{ github.workspace }}/.ccache/"
LINUX_DISTRO: "centos"
steps:

- name: Download join fuzzer
uses: actions/download-artifact@v4
with:
name: join

- name: "Checkout Repo"
uses: actions/checkout@v4
with:
path: velox
submodules: 'recursive'
ref: "${{ inputs.ref }}"

- name: Fix git permissions
# Usually actions/checkout does this but as we run in a container
# it doesn't work
run: git config --global --add safe.directory ${GITHUB_WORKSPACE}/velox

- name: Run Join Fuzzer
run: |
cd velox
cp ./scripts/presto/etc/hive.properties $PRESTO_HOME/etc/catalog
ls -lR $PRESTO_HOME/etc
$PRESTO_HOME/bin/launcher run -v > /tmp/server.log 2>&1 &
# Sleep for 60 seconds to allow Presto server to start.
sleep 60
/opt/presto-cli --server 127.0.0.1:8080 --execute 'CREATE SCHEMA hive.tpch;'
cd -
mkdir -p /tmp/join_fuzzer_repro/logs/
chmod -R 777 /tmp/join_fuzzer_repro
chmod +x velox_join_fuzzer_test
Expand All @@ -597,15 +620,18 @@ jobs:
--minloglevel=0 \
--stderrthreshold=2 \
--log_dir=/tmp/join_fuzzer_repro/logs \
--presto_url=http://127.0.0.1:8080 \
--req_timeout_ms=2000 \
&& echo -e "\n\nJoin fuzzer run finished successfully."
- name: Archive aggregate production artifacts
- name: Archive join production artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v4
with:
name: join-fuzzer-failure-artifacts
name: presto-sot-join-fuzzer-failure-artifacts
path: |
/tmp/join_fuzzer_repro
/tmp/server.log
exchange-fuzzer-run:
name: Exchange Fuzzer
Expand Down

0 comments on commit cc10dc2

Please sign in to comment.