docs(dspy): setting up base structure for mkdocs based (versioned) do…

…cumentation
stanfordnlp · Jan 18, 2024 · a91c17c · a91c17c
1 parent 18cee06
commit a91c17c
Show file tree

Hide file tree

Showing 9 changed files with 6,184 additions and 0 deletions.
diff --git a/docs/custom.css b/docs/custom.css
@@ -0,0 +1,169 @@
+.green-title {
+  color: green;
+  display: inline;
+  font-size: small;
+}
+
+.platform-badge,
+.version-badge {
+  padding: 5px;
+  border-radius: 5px;
+  margin-left: 10px;
+  display: inline;
+}
+
+.platform-badge {
+  background-color: #007bff;
+  color: white;
+  font-size: smaller;
+}
+
+.version-badge {
+  background-color: #f39d12ae;
+  color: rgb(0, 0, 0);
+  font-size: 10px;
+  padding: 10px;
+}
+
+.team-members {
+  display: flex;
+  justify-content: space-around;
+}
+
+.team-member {
+  text-align: center;
+  /* flex: 1; */
+  width: 40px;
+  /* height: 40px; */
+}
+
+.title-separator,
+.comments-separator {
+  border-top: 1px solid #ccc;
+  margin: 10px 0;
+}
+
+.comments-section {
+  margin-top: 10px;
+  font-size: smaller;
+}
+
+.warning-message {
+  background-color: #f44336;
+  color: white;
+  padding: 10px;
+  border-radius: 5px;
+  display: none;
+}
+
+.warning-message.show {
+  display: block;
+}
+
+.success-message {
+  background-color: #4caf50;
+  color: white;
+  padding: 10px;
+  border-radius: 5px;
+}
+
+.priority-section {
+  display: flex;
+  align-items: center;
+}
+
+.warning-message-below {
+  background-color: rgba(244, 67, 54, 0.7); /* Red with alpha 0.7 */
+  color: white;
+  padding: 10px;
+  border-radius: 5px;
+  margin-top: 10px;
+  font-size: smaller;
+}
+
+.collapsible-info {
+  background-color: #2baa55ce; /* Blue */
+  color: white;
+  padding: 10px;
+  border-radius: 5px;
+  margin-top: 10px;
+  display: block;
+  font-size: smaller;
+}
+
+.collapsible-info.show {
+  display: block;
+}
+.title-badge {
+  font-size: 1.2em;
+  background-color: #4caf50; /* Green */
+  color: white;
+  padding: 10px;
+  border-radius: 5px;
+  display: inline;
+}
+
+.model-image {
+  width: 130px; /* Set the width */
+  max-width: 100%; /* Make sure it scales down if the container is smaller */
+  height: auto; /* Maintain aspect ratio */
+  display: block; /* Change from inline to block */
+  margin: auto; /* Center the image */
+}
+
+.model-badge {
+  font-size: 1.2em;
+  background-color: #4caf50; /* Green */
+  color: white;
+  padding: 10px;
+  border-radius: 5px;
+  display: inline;
+  transition: background-color 0.3s ease; /* Smooth transition */
+}
+
+.model-badge:hover {
+  background-color: #4a53ec; /* Darker green */
+}
+
+.title-platform-section {
+  display: flex;
+  align-items: center;
+}
+
+.status-priority-section {
+  display: flex;
+  align-items: center;
+  justify-content: space-between;
+}
+
+.card-grid {
+  display: flex;
+  flex-wrap: wrap;
+  gap: 16px;
+}
+
+.card {
+  flex: 1;
+  border: 1px solid #ccc;
+  border-radius: 8px;
+  padding: 16px;
+  margin: 16px;
+  min-width: calc(45% - 20px); /* For 3 cards per row */
+  max-width: calc(45% - 20px);
+  max-height: fit-content;
+  margin: 10px;
+}
+
+button {
+  background-color: #023b04c0;
+  color: white;
+  padding: 14px 20px;
+  margin: 8px 0;
+  border: none;
+  cursor: pointer;
+}
+
+button:disabled {
+  background-color: #ccc;
+  cursor: not-allowed;
+}
diff --git a/docs/docs_requirements.txt b/docs/docs_requirements.txt
@@ -0,0 +1,5 @@
+mkdocs
+mkdocs-gen-files
+mkdocs-material
+mkdocs-material-extensions
+mkdocstrings-python
diff --git a/docs/index.md b/docs/index.md
@@ -0,0 +1,58 @@
+# 🌟👋 Welcome to DSPy -- The framework for programming—not prompting—foundation models 🌐🚀
+
+<!DOCTYPE html>
+<html>
+<head>
+    <style>
+        /* Define the animation */
+        @keyframes bounceRotate {
+            0% {
+                transform: rotateY(0deg);
+            }
+            50% {
+                transform: rotateY(20deg);
+            }
+            100% {
+                transform: rotateY(0deg);
+            }
+        }
+
+        /* Apply the animation to the image */
+        .bounce-rotate-logo {
+            animation: bounceRotate 7s ease-in-out infinite; /* Bounce back and forth infinitely with a duration of 5 seconds */
+        }
+    </style>
+
+</head>
+<body>
+
+<p style="text-align:center;">
+  <img class="bounce-rotate-logo" src="./docs/images/DSPy8.png" width="50%">
+</p>
+
+</body>
+</html>
+
+## 🎯 The Vision Behind DSPy
+
+**DSPy** is a framework for developing **high-quality systems** with LMs. While prompting LMs can quickly build (brittle) demos, the best LM systems generally break down problems into steps and tune the prompts or LM weights of each step well. As a bonus, these systems use small LMs to save costs.
+
+This is hard as we usually don't have data to tune each of these steps. **DSPy** treats prompts and LM weights as parameters to be optimized in LM pipelines, given the metrics you want to maximize.
+
+To make this possible:
+
+- [x] **DSPy** provides **composable and declarative modules** for instructing LMs in a familiar Pythonic syntax. It upgrades "prompting techniques" like chain-of-thought and self-reflection from hand-adapted _string manipulation tricks_ into truly modular _generalized operations that learn to adapt to your task_.
+
+- [x] **DSPy** introduces an **automatic compiler that teaches LMs** how to conduct the declarative steps in your program. Specifically, the **DSPy compiler** will internally _trace_ your program and then **craft high-quality prompts for large LMs (or train automatic finetunes for small LMs)** to teach them the steps of your task.
+
+- [x] **DSPy** has many modules and optimizers built-in and we want you to add more. Think of this like PyTorch but for LM pipelines, not DNNs. The **DSPy compiler** _bootstraps_ prompts and finetunes from minimal data **without needing manual labels for the intermediate steps** in your program. Instead of brittle "prompt engineering" with hacky string manipulation, you can explore a systematic space of modular and trainable pieces.
+
+- [x] For complex tasks, **DSPy** can routinely teach powerful models like `GPT-3.5` and local models like `T5-base` or `Llama2-13b` to be much more reliable at tasks. **DSPy** will compile the _same program_ into different few-shot prompts and/or finetunes for each LM.
+
+## 🚀 Analogy to Neural Networks
+
+When we build neural networks, we don't write manual _for-loops_ over lists of _hand-tuned_ floats. Instead, you might use a framework like [PyTorch](https://pytorch.org/) to compose declarative layers (e.g., `Convolution` or `Dropout`) and then use optimizers (e.g., SGD or Adam) to learn the parameters of the network.
+
+Ditto! **DSPy** gives you the right general-purpose modules (e.g., `ChainOfThought`, `Retrieve`, etc.) and takes care of optimizing their prompts _for your program_ and your metric, whatever they aim to do. Whenever you modify your code, your data, or your validation constraints, you can _compile_ your program again and **DSPy** will create new effective prompts that fit your changes.
+
+**Welcome to the future of LLMs programmig! 🌟🌐**
diff --git a/docs/repo/contributing.md b/docs/repo/contributing.md
@@ -0,0 +1,71 @@
+# ⚙️ Setting-up working envirionment
+
+## 💻 Env Setup
+
+```
+conda create --name dspy python=3.11
+```
+
+or
+
+```
+python3 -m venv dspy
+```
+
+## 🚀 Pre-commit hook
+
+Before using pre-commit hook you need to install it in your python environment.
+
+```
+conda install -c conda-forge pre-commit
+```
+
+go to the root folder and then activate it as follows (it will first download all required dependencies):
+
+```
+pre-commit install
+```
+
+> Pre-commit hooks will attept to fix all your files and so you will need to (add + commit) them once the fixes are done !
+
+!!! info "Optionally"
+
+    Generally the pre-commit will run automatically before each of your commit,
+    but you can also manually trigger it, as follows:
+
+    ```python
+    pre-commit run --all-files
+    ```
+
+## 📝 Commit with Style
+
+Use standarized commit message:
+
+`{LABEL}(ACRONYM): {message}`
+
+This is very important for the automatic releases and a clean history on the `main` branch.
+
+!!! Labels-types
+
+    | Label | Usage |
+    | ----- | ----- |
+    | break| `break` is used to identify changes related to old compatibility or functionality that breaks the current usage (major) |
+    | feat | `feat` is used to identify changes related to new backward-compatible abilities or functionality (minor) |
+    | init | `init` is used to indentify the starting related to the project (minor) |
+    | enh | `enh` is used to indentify changes related to amelioration of abilities or functionality (patch) |
+    | build | `build` (also known as `chore`) is used to identify **development** changes related to the build system (involving scripts, configurations, or tools) and package dependencies (patch) |
+    | ci | `ci` is used to identify **development** changes related to the continuous integration and deployment system - involving scripts, configurations, or tools (minor) |
+    | docs | `docs`  is used to identify documentation changes related to the project; whether intended externally for the end-users or internally for the developers (patch) |
+    | perf | `perf`  is used to identify changes related to backward-compatible **performance improvements** (patch) |
+    | refactor | `refactor` is used to identify changes related to modifying the codebase, which neither adds a feature nor fixes a bug - such as removing redundant code, simplifying the code, renaming variables, etc.<br />i.e. handy for your wip ; ) (patch) |
+    | style | `style`  is used to identify **development** changes related to styling the codebase, regardless of the meaning - such as indentations, semi-colons, quotes, trailing commas, and so on (patch) |
+    | test | `test` is used to identify **development** changes related to tests - such as refactoring existing tests or adding new tests. (minor) |
+    | fix | `fix`  is used to identify changes related to backward-compatible bug fixes. (patch) |
+    | ops | `ops` is used to identify changes related to deployment files like `values.yml`, `gateway.yml,` or `Jenkinsfile` in the **ops** directory. (minor) |
+    | hotfix | `hotfix` is used to identify **production** changes related to backward-compatible bug fixes (patch) |
+    | revert | `revert` is used to identify backward changes (patch) |
+    | maint | `maint` is used to identify **maintenance** changes related to project (patch) |
+
+```
+
+```
diff --git a/docs/repo/documentation.md b/docs/repo/documentation.md
@@ -0,0 +1,82 @@
+# 📃 Documentation
+
+We are using MkDocs to build the documentation, you can find more info about all possibilites here: [Examples](https://squidfunk.github.io/mkdocs-material/reference/#setting-the-page-title). It is basically a combination of Markdown and Mermaid for graph.
+
+!!! info
+You can read more about [Mermaid](https://github.com/mermaid-js/mermaid) with all its possibilities !
+If you would like to test your mermaid FlowChart online without having to install any required libraries, you can refere to [Online Schema Editor](https://mermaid-js.github.io/mermaid-live-editor)
+
+## ➕ Extending the documentation
+
+- [x] start from the `dev` branch and create a new branch with the correct naming convention, see: [how to contribute](contributing.md)
+
+- [x] add additional '.md' files to the documentation directory: `/docs`
+
+- [x] add the new entry into the navigation bar and connect it with your `md` file. This can be done in: [`root/mkdocs.yml`](mkdocs.yml)
+
+- [x] You can interactively test the documentation locally by using the following command: `mkdocs serve`
+
+  > You will need to have all local docs-related requirements installed (see: [tool.poetry.group.doc.dependencies]):
+
+  ```python
+  mkdocs = ">=1.5.3"
+  mkdocs-material = ">=9.0.6"
+  mkdocs-material-extensions = ">=1.3.1"
+  mkdocs-gen-files = "^0.5.0"
+  mkdocstrings-python = "^1.7.5"
+  mkdocstrings = {extras = ["python"], version = ">=0.20.0"}
+  mike = ">=2.0.0"
+  ```
+
+- [x] Once you are done, create a new Merge Request to the `dev` branch.
+
+- [x] When your MR gets approved merge it into the `dev` following the well know conventions [how to contribute](contributing.md)
+
+- [x] New documentation will be automatically deployed once your MR gets merged !
+
+!!! warning
+In some cases you may need to deploy the new doc to Github-pages immediatly, this can be done using the following command: `mkdocs gh-deploy` (while being in a right venv)
+
+## 🔍 Documenting code
+
+Documenting code is done using dedicated docstrings, which are then automatically parsed and converted into the documentation.
+
+In order to document your code, you need to use the following syntax:
+
+```python
+# 🗺️ PARAGRAPG NAME
+
+::: dspy.predict.predict.Predict
+    handler: python
+    options:
+        show_root_heading: true
+        show_source: true
+```
+
+and the Predict class documentation needs to foollow the Google Style Guide, see: [Google Style Guide](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings)
+
+!!! example
+
+    ```python
+    """Send a static HTML report along with a message to the Slack channel.
+
+        Args:
+            project_name (str): The name of the project for which the job was submitted.
+            html_file_path (str): The file path of the HTML report to be sent.
+            message (str): The message to send along with the HTML report.
+
+        Example:
+            ```python
+            from theparrot.notify import SlackBot
+
+            bot = SlackBot()
+            bot.send_html_report(
+                html_file_path="path/to/your/report.html",
+                message="Check out this report!",
+            )
+            ```
+        """
+    ```
+
+This approach allows to handle documentation directly from the code, which is a great way to keep it up to date.
+It also allows to version the documentation, which is a great way to keep track of changes and handle multiple versions of the package.