This repository has been archived by the owner on Sep 27, 2019. It is now read-only.
[15721]Index Tuning with RL #1338
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
…oton into brain_rl_testing_framework
…-Lee/peloton into brain_rl_testing_framework
llvm for mac Basic classes added the configuration enumeration files Add Whatif API Add optimizer cost query func skeleton Complete what if API implementation. Testing pending. 1. Add test file in brain for what-if API. 2. Implement a basic test to insert some tuples and hypothetical indexes and get the cost. (Not working) Ignore query planning Analyze tables was missing. Fixed it fix the query add comments, fix some code style Fix whatif API test run formatter Add index selection module skeleton skeleton for admissible column parsing adding cost model classes cleanup and reorganize the code Intermediate changes. Query parser not complete. Intermediate changes. Query parser not complete. removed cost model class Add IndexObject Pool Memoization support completed Complete query parser Complete query parser multi column index, wip Add tests for admissible indexes Fix what if index and admissive indexes test added outline for naive enumeration method Fix get admissible indexes test Fix get admissible indexes test Added the IndexConfiguration set difference Minor BUg Fix Split computing and getting const Fix compilation error and typos Finish Configuration Enumeration module Fix the main index selection algorithm Finish Merging Merge cleanup Restructure code More refactoring added comments to index selection context Added the comparator for the candidate index enumeration Adding comments Restructure generate candidate indexes Fix merge partial test for multi columnindex generation Add candidate index gen test Minor change to ComputeCost. Formatting and comments. Add comments comments More formatting and comments. more comments brief comments. rename pl_assert to peloton_assert Remove GetCost and rename ComputeCost to GetCost fix multicolumnindex generation minor fixes Fix admissible index and candidate pruning tests Fix unused variables Add more tests to WhatIfAPI and IndexSelection Implement the suggestions mentioned in the code review Uncomment the choose best plan call Fix tests Add support for multi-column index Conflicts: src/optimizer/rule_impls.cpp src/optimizer/stats_calculator.cpp Fix conflicts after merge nit fixes Fix what-if index tests Add more multi-column index sets in the test cases. Add testing utility class for index suggestion tests Add to cmake for the files in the previous commit Modify what-if tests to use the utility class Fix formatting Code review fix fix tests nit Fix memory leaks and misc nit fixes fixed the test temportarily for the index bug Rename IndexObject to HypotheticalIndexObject debugging the shared pointer issue Fix segfault. Some more Renames check the exact indexes Fix the tests to use the util fixing the index selection Fix formatting Rebase and fix conflicts while rebasing latest tests Better tests Add get workload support to the testing utility class. Fix stray Comment out the debug code in optimizer Add index suggestion task skeleton Add query history catalog GET methods. Fix formatting Update index suggestion task Add new workload Add new test - incomplete Add more than 3 columns cost model test Fix join query parsing for table name extraction Add more queries to workload D DEBUG -> TRACE Fix compilation error Complete the index suggestion task - RPC is pending. Changed the columns from a set to vector Get args at RPC handler Fix compilation issue and list serialization Refactored the tests Complete RPC handler Fix compilation error in peloton-bin fix logs run brain and server together in one process for testing Add dropIndex RPC MOved tunable knobs into a separate structure Fix index selection job -- rename some stuff changed the arguments of the constructor completed the refactor Rename more stuff minor style changes More renames Fix DML statement handling in workload Fix cost model bug for more than 2 column indexes Add an extensive test on multi-column optimizer cost model test Add drop indexes RPC Run formatter concrete test case to show the issues with non-deterministic set of indexes Fix drop indexes Fix a bug in config enumeration for case where no index is better Fix the test that is failing non-deteministically due to the optimizer cost evaluation module Fix formatter issue Fix travis error Use only one transaction for the entire run of the job. Also, generate stats for every run of the job hopefully, final version of the algorithm added multiple choices for the output more index selection tests Add missing populate index Consider non-equality predicates for index scan in the cost model fixed precision issues Drop the indexes only if it is not suggested this time minor fixes Fix the AnalyzeStats crash Fix: Index Selection returns empty set because the catalog cache eviction is not done properly. Fix a bug during where clause parsing to make it work with TPCC Fix the compilation error Address some of the code review comments Fix create/drop index -- running TPCC Fix analyze stats crash. Fix query history logging for PREPARED statements Change knobs More misc addressing commits Restructure code Reformat code small correction to make it compile in debug mode remove the unnecessary commented parts of test and code remove #define Restructure code, fix nits fix errors for compilation in debug mode Restructure code Run formatter fix query logger test trying to pass the compilation on travis change debug logging to trace level logging Fix warning in IndexConfigComparator warning: the specified comparator type does not provide a const call operator [-Wuser-defined-warnings] trace-->debug Hack to make travis pass the build. DEFUALT_SCHEMA_NAME can't be found error. Fix this when merging with master. Hack to make travis pass the build. DEFUALT_SCHEMA_NAME can't be found error. Fix this when merging with master. remove multiple of unnecessary debug statements
…-Lee/peloton into brain_rl_testing_framework
saatviks
force-pushed
the
brain_rl_testing_framework
branch
from
9ba691e
to
58a4ab5
Compare
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Least-Squares Policy Iteration based Index tuning - Inspired from Cost-Model Oblivious DB Tuning by Basu et. al..
All the main source files are under
src/brain/indextune:Add/Drop candidates, featurization, etc.We currently only have a few sanity tests for the end to end setup, but have tested the
CompressedIndexConfig*classes andRLSEModelsufficiently.