Skip to content
This repository has been archived by the owner on Jun 6, 2024. It is now read-only.

cmu-db/noisepage-pilot

Repository files navigation

NoisePage Pilot

This repository contains the pilot components for the NoisePage DBMS.

Quickstart

  1. Init all submodules git submodule update --init --recursive.
  2. Install necessary packages.
    • cd behavior/modeling/featurewiz && pip3 install --upgrade -r requirements.txt
    • pip3 install --upgrade -r requirements.txt
  3. List all the tasks.
    • doit list
  4. Select and run a doit task from the task list, e.g. doit action_recommendation. Task dependencies are executed automatically.

Sample Doit Tasks

The following is a list of some relevant doit tasks and their parameters:

  • Action Recommendation
    • The following task will begin picking indexes: doit action_recommendation.
  • Behavior Modeling
    • doit behavior_datagen runs behavior model training data generation using TScout, Benchbase and Postgres. Requires sudo permissions for TScout. Also performs training data differencing (subtracting child-plan costs).
    • doit behavior_train trains, evaluates, and serializes models along with their evaluations and predictions. It depends on behavior_datagen having been run first.
    • Configure data generation, model training, Benchbase, and Postgres in noisepage-pilot/config/behavior.
    • Training data differencing and model training default to using the most recent experiment data.
    • Additional behavior modeling documentation is available at noisepage-pilot/behavior/README.md.

Background

  • Self-Driving DBMS = Workload Forecasting + Behavior Modeling + Action Planning.
    • Workload Forecasting: forecast folder.
    • Modeling: WIP.
    • Action Planning: action folder.
    • See 1, 2 for more details.

References

Footnotes

  1. Make Your Database System Dream of Electric Sheep: Towards Self-Driving Operation.

    @article{pavlo21,
    author = {Pavlo, Andrew and Butrovich, Matthew and Ma, Lin and Lim, Wan Shen and Menon, Prashanth and Van Aken, Dana and Zhang, William},
    title = {Make Your Database System Dream of Electric Sheep: Towards Self-Driving Operation},
    journal = {Proc. {VLDB} Endow.},
    volume = {14},
    number = {12},
    pages = {3211--3221},
    year = {2021},
    url = {https://db.cs.cmu.edu/papers/2021/p3211-pavlo.pdf},
    }
    
  2. https://15799.courses.cs.cmu.edu/spring2022/