Initial commit

.env.test

@@ -0,0 +1,4 @@
+#copy and paste this file into a .env file to run scripts locally. You will also need to add the DUNE_API_KEY into the repo settings under "Secrets and Variables"
+
+#add a dune API key - you can create one under team settings (https://dune.com/settings/teams/manage/{team_name}/api). You must be on the premium plan.
+DUNE_API_KEY=

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,17 @@
+---
+name: Bug Report
+about: Tell us about the data bug or broken query you are seeing
+title: "[BUG] Query data quality issue"
+labels: 'topic: bug'
+assignees: ''
+
+---
+
+**What is the query id, and describe the bug**
+A clear and concise description of what the bug is, with query id linked.
+
+**Examples of the bug**
+Please provide links to etherscan or your own queries showing the errors/discrepencies.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.

.github/ISSUE_TEMPLATE/chart-improvment.md

@@ -0,0 +1,17 @@
+---
+name: Chart Improvment
+about: Suggest an idea for improving a chart
+title: "[CHART] Suggestion for improving a chart"
+labels: 'topic: chart_improvements'
+assignees: ''
+
+---
+
+**Which query would you like to improve, and why?"**
+A clear and concise description of what the problem is, with query id (if this is an improvement to an existing chart)
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Any work or examples you've seen that are similar to the solution**
+Links and screenshots to any guiding charts/tables/queries you've seen.

.github/ISSUE_TEMPLATE/generic-issue.md

@@ -0,0 +1,11 @@
+---
+name: Generic Issue
+about: Tell us about your question
+title: "[GENERIC] Question"
+labels: 'topic: generic'
+assignees: ''
+
+---
+
+**What is your question?**
+I don't quite understand how/why this query does XYZ...

.github/ISSUE_TEMPLATE/query-improvement.md

@@ -0,0 +1,17 @@
+---
+name: Query Improvement
+about: Suggest an idea for improving a query
+title: "[QUERY] Suggestion for improving a chart"
+labels: 'topic: query_improvements'
+assignees: ''
+
+---
+
+**Which query would you like to improve, and why?"**
+A clear and concise description of what the problem is, with query id (if this is an improvement to an existing chart)
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen. Please suggest data sources/tables that this enhancement could come from.
+
+**Any work or examples you've seen that are similar to the solution**
+Links and screenshots to any guiding charts/tables/queries you've seen.

.github/workflows/push_to_dune.yml

@@ -0,0 +1,33 @@
+name: Push to Dune on Commit
+
+on:
+  push:
+    branches:
+      - main
+    paths:
+      - 'queries/**'
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v2
+
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.9'
+
+      - name: Log directory structure
+        run: |
+          pwd
+          ls -R
+    
+      - name: pip requirements
+        run: pip install -r requirements.txt
+
+      - name: Update all queries from Dune, by overwriting queries with repo query text
+        env:
+          DUNE_API_KEY: ${{ secrets.DUNE_API_KEY }}
+          DUNE_API_BASE_URL: ${{ secrets.DUNE_API_BASE_URL }}
+        run: python -u scripts/push_to_dune.py

.gitignore

@@ -0,0 +1,2 @@
+
+.env

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 Dune
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,22 @@
+**Is this linked to an existing issue**
+If so, link that issue(s) here.
+
+**Fill out the following table describing your edits:**
+
+| Original Query | Updated Query | Change | Reasoning |
+|---|---|---|---|
+| [3237745](https://dune.com/queries/3237745) | [3237938](https://dune.com/queries/3238935) | Remove sandwich traders using dex.sandwiches | We should only care about traders who are not doing MEV |
+| [3237723](https://dune.com/queries/3237745) | ... | ... | ... |
+
+**Provide any other context or screenshots that explain or justify the changes above:**
+
+"`tx_from` may not be the most accurate, we might want to use `tx_to` instead since that identifies the MEV contract being utilized. But this requires further digging"
+
+---
+
+**Note to Contributor:**
+
+Make sure your PR edits the original `query_id.sql` file with the new query text. If you are proposing adding a new query completely, make sure to add it to `queries.yml` and as a file in `/queries` as well.
+
+Thanks for contributing! 🙏
+

README.md

@@ -0,0 +1,57 @@
+# Dune Query Repo
+
+A template for creating repos to manage your Dune queries (using the [Dune CRUD API](https://dune.com/docs/api/api-reference/edit-queries/)). The main flow I've created this template for is to turn any dashboard you own into a repository of queries. But you can extend it however you want.
+
+### Setup Your Repo
+
+1. Generate an API key from your Dune account and put that in both your `.env` file and [github action secrets](https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository) (name it `DUNE_API_KEY`). You can create a key under your Dune team settings. *The key must be from a premium plan for this repo to work.*
+
+2. Then, go to the dashboard you want to create a repo for (must be owned by you/your team). Click on the "github" button in the top right of your dashboard to see the query ids.
+
+3. Copy and paste that list into the `queries.yml` file inside the `/queries` folder. You can paste any list of query ids, doesn't have to be linked to a dashboard.
+
+4. Then, run `pull_from_dune.py` to bring in all queries into `/query_{id}.sql` files within the `/queries` folder. Directions to setup and run this script are below.
+
+5. Make any changes you need to directly in the repo. Any time you push a commit to MAIN branch, `push_to_dune.py` will save your changes into Dune directly.
+
+---
+
+### Query Management Scripts
+
+You'll need python and pip installed to run the script commands. If you don't have a package manager set up, then use either [conda](https://www.anaconda.com/download) or [poetry](https://python-poetry.org/) . Then install the required packages:
+
+```
+pip install -r requirements.txt
+```
+
+| Script | Action | Command |
+|---|---|---|
+| `pull_from_dune.py` | updates/adds queries to your repo based on ids in `queries.yml` | `python scripts/pull_from_dune.py` |
+| `push_to_dune.py` | updates queries to Dune based on files in your `/queries` folder | `python scripts/push_to_dune.py` |
+| `preview_query.py` | gives you the first 20 rows of results by running a query from your `/queries` folder. Specify the id. | `python scripts/preview_query.py 2615782` |
+
+---
+
+### Things to be aware of
+
+💡: Names of queries are pulled into the file name the first time `pull_from_dune.py` is run. Changing the file name in app or in folder will not affect each other (they aren't synced). **Make sure you leave the `___id.sql` at the end of the file, otherwise the scripts will break!**
+
+🟧: Make sure to leave in the comment `-- already part of a query repo` at the top of your file. This will hopefully help prevent others from using it in more than one repo.
+
+🔒: Queries must be owned by the team the API key was created under - otherwise you won't be able to update them from the repo.
+
+➕: If you want to add a query, add it in Dune app first then pull the query id (from URL `dune.com/queries/{id}/other_stuff`) into `queries.yml`
+
+🛑: If you accidently merge a PR or push a commit that messes up your query in Dune, you can roll back any changes using [query version history](https://dune.com/docs/app/query-editor/version-history).
+
+---
+
+### For Contributors
+
+I've set up four types of issues right now:
+- `bugs`: This is for data quality issues like miscalculations or broken queries.
+- `chart_improvements`: This is for suggesting improvements to the visualizations.
+- `query_improvements`: This is for suggesting improvements to the query itself, such as adding an extra column or table that enhances the results.
+- `generic`: This is a catch all for other questions or suggestions you may have about the dashboard.
+
+If you want to contribute, either start an issue or go directly into making a PR (using the same labels as above). Once the PR is merged, the queries will get updated in the frontend.

queries/queries.yml

@@ -0,0 +1,3 @@
+query_ids:
+  - 2615782
+  - 2615781