Skip to content

Commit

Permalink
Update README
Browse files Browse the repository at this point in the history
  • Loading branch information
@Yucheng-Jiang
Yucheng-Jiang committed Sep 25, 2024
1 parent 0f16ece commit 0734f7f
Show file tree
Hide file tree
Showing 2 changed files with 166 additions and 39 deletions.
205 changes: 166 additions & 39 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,14 @@
# STORM: Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking

<p align="center">
| <a href="http://storm.genie.stanford.edu"><b>Research preview</b></a> | <a href="https://arxiv.org/abs/2402.14207"><b>Paper</b></a> | <a href="https://storm-project.stanford.edu/"><b>Website</b></a> |
| <a href="http://storm.genie.stanford.edu"><b>Research preview</b></a> | <a href="https://arxiv.org/abs/2402.14207"><b>STORM Paper</b></a>| <a href="https://www.arxiv.org/abs/2408.15232"><b>Co-STORM Paper</b></a> | <a href="https://storm-project.stanford.edu/"><b>Website</b></a> |
</p>

**Latest News** 🔥

- [2024/09] Release Co-STORM codebase and intergrated into knowledge-storm python package.

- [2024/09] Introduce collaborative STORM (Co-STORM) to support human-AI collaborative knowledge curation! [Co-STORM Paper](https://www.arxiv.org/abs/2408.15232) accepted to 2024 EMNLP.

- [2024/07] You can now install our package with `pip install knowledge-storm`!
- [2024/07] We add `VectorRM` to support grounding on user-provided documents, complementing existing support of search engines (`YouRM`, `BingSearch`). (check out [#58](https://github.com/stanford-oval/storm/pull/58))
- [2024/07] We release demo light for developers a minimal user interface built with streamlit framework in Python, handy for local development and demo hosting (checkout [#54](https://github.com/stanford-oval/storm/pull/54))
Expand All @@ -24,17 +27,20 @@
<p align="center">
<img src="assets/overview.svg" style="width: 90%; height: auto;">
</p>
STORM is a LLM system that writes Wikipedia-like articles from scratch based on Internet search.
STORM is a LLM system that writes Wikipedia-like articles from scratch based on Internet search. Co-STORM further enhanced its feature by enabling human to collaborative LLM system to support more aligned and preferred information seeking and knowledge curation.

While the system cannot produce publication-ready articles that often require a significant number of edits, experienced Wikipedia editors have found it helpful in their pre-writing stage.

**Try out our [live research preview](https://storm.genie.stanford.edu/) to see how STORM can help your knowledge exploration journey and please provide feedback to help us improve the system 🙏!**
**More than 70,000 people have tried our [live research preview](https://storm.genie.stanford.edu/). Try it out to see how STORM can help your knowledge exploration journey and please provide feedback to help us improve the system 🙏!**



## How STORM / Co-STORM works

## How STORM works
### STORM

STORM breaks down generating long articles with citations into two steps:

1. **Pre-writing stage**: The system conducts Internet-based research to collect references and generates an outline.
2. **Writing stage**: The system uses the outline and references to generate the full-length article with citations.
<p align="center">
Expand All @@ -47,7 +53,21 @@ STORM identifies the core of automating the research process as automatically co

Based on the separation of the two stages, STORM is implemented in a highly modular way using [dspy](https://github.com/stanfordnlp/dspy).

### CO-STORM

Co-STORM proposes collaborative discourse protocol which implements conversation turn policy management controlling smooth collaboration among

- **Co-STORM LLM experts**: This type of agent generate answers grounded on external knowledge sources and/or raise follow up questions based on discourse history.
- **Moderator**: This agent generate thought provoking questions taking inspiring from collection information provided by Co-STORM LLM experts.
- **Human user**: Human user will take the initiative to either (1) observe the discourse to gain deeper understanding of the topic, or (2) actively engaging in the conversation by injecting utterances to steer the discussion focus

<p align="center">
<img src="assets/co-storm-workflow.jpg" style="width: 60%; height: auto;">
</p>

Co-STORM also maintains a dynamic updated **mind map**, which organize collected information into hirarchical concept structure, aiming to build a shared conceptual space between the human user and the system while reducing the mental load when having long in depth discourse.

Co-STORM is also implemented in a highly modular way using [dspy](https://github.com/stanfordnlp/dspy).

## Installation

Expand All @@ -70,9 +90,20 @@ You could also install the source code which allows you to modify the behavior o


## API
The STORM knowledge curation engine is defined as a simple Python `STORMWikiRunner` class.

As STORM is working in the information curation layer, you need to set up the information retrieval module and language model module to create a `STORMWikiRunner` instance. Here is an example of using You.com search engine and OpenAI models.
Currently, our package support:

- `OpenAIModel`, `AzureOpenAIModel`, `ClaudeModel`, `VLLMClient`, `TGIClient`, `TogetherClient`, `OllamaClient`, `GoogleModel`, `DeepSeekModel`, `GroqModel` as language model components
- `YouRM`, `BingSearch`, `VectorRM`, `SerperRM`, `BraveRM`, `SearXNG`, `DuckDuckGoSearchRM`, `TavilySearchRM`, `GoogleSearch` as retrieval module components

:star2: **PRs for integrating more language models into [knowledge_storm/lm.py](knowledge_storm/lm.py) and search engines/retrievers into [knowledge_storm/rm.py](knowledge_storm/rm.py) are highly appreciated!**

Both STORM and Co-STORM are working in the information curation layer, you need to set up the information retrieval module and language model module to create their `Runner` classes respectively.

### STORM

The STORM knowledge curation engine is defined as a simple Python `STORMWikiRunner` class. Here is an example of using You.com search engine and OpenAI models.

```python
import os
from knowledge_storm import STORMWikiRunnerArguments, STORMWikiRunner, STORMWikiLMConfigs
Expand Down Expand Up @@ -101,12 +132,6 @@ rm = YouRM(ydc_api_key=os.getenv('YDC_API_KEY'), k=engine_args.search_top_k)
runner = STORMWikiRunner(engine_args, lm_configs, rm)
```

Currently, our package support:
- `OpenAIModel`, `AzureOpenAIModel`, `ClaudeModel`, `VLLMClient`, `TGIClient`, `TogetherClient`, `OllamaClient`, `GoogleModel`, `DeepSeekModel`, `GroqModel` as language model components
- `YouRM`, `BingSearch`, `VectorRM`, `SerperRM`, `BraveRM`, `SearXNG`, `DuckDuckGoSearchRM`, `TavilySearchRM`, `GoogleSearch` as retrieval module components

:star2: **PRs for integrating more language models into [knowledge_storm/lm.py](knowledge_storm/lm.py) and search engines/retrievers into [knowledge_storm/rm.py](knowledge_storm/rm.py) are highly appreciated!**

The `STORMWikiRunner` instance can be evoked with the simple `run` method:
```python
topic = input('Topic: ')
Expand All @@ -125,41 +150,126 @@ runner.summary()
- `do_generate_article`: if True, generate an article for the topic based on the outline and the collected information; otherwise, load the results.
- `do_polish_article`: if True, polish the article by adding a summarization section and (optionally) removing duplicate content; otherwise, load the results.

### Co-STORM

The Co-STORM knowledge curation engine is defined as a simple Python `STORMWikiRunner` class. Here is an example of using Bing search engine and OpenAI models.

```python
from knowledge_storm.collaborative_storm.engine import CollaborativeStormLMConfigs, RunnerArgument, CoStormRunner
from knowledge_storm.lm import OpenAIModel
from knowledge_storm.logging_wrapper import LoggingWrapper
from knowledge_storm.rm import BingSearch
# Co-STORM adopts the same multi LM system paradigm as STORM
lm_config: CollaborativeStormLMConfigs = CollaborativeStormLMConfigs()
openai_kwargs = {
"api_key": os.getenv("OPENAI_API_KEY"),
"api_provider": "openai",
"temperature": 1.0,
"top_p": 0.9,
"api_base": None,
}
question_answering_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=1000, **openai_kwargs)
discourse_manage_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=500, **openai_kwargs)
utterance_polishing_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=2000, **openai_kwargs)
warmstart_outline_gen_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=500, **openai_kwargs)
question_asking_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=300, **openai_kwargs)
knowledge_base_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=1000, **openai_kwargs)
lm_config.set_question_answering_lm(question_answering_lm)
lm_config.set_discourse_manage_lm(discourse_manage_lm)
lm_config.set_utterance_polishing_lm(utterance_polishing_lm)
lm_config.set_warmstart_outline_gen_lm(warmstart_outline_gen_lm)
lm_config.set_question_asking_lm(question_asking_lm)
lm_config.set_knowledge_base_lm(knowledge_base_lm)
# Check out the CoSTORM's RunnerArguments class for more configurations.
topic = input('Topic: ')
runner_argument = RunnerArgument(topic=topic, ...)
logging_wrapper = LoggingWrapper(lm_config)
bing_rm = BingSearch(bing_search_api_key=os.environ.get("BING_SEARCH_API_KEY"),
k=runner_argument.retrieve_top_k)
costorm_runner = CoStormRunner(lm_config=lm_config,
runner_argument=runner_argument,
logging_wrapper=logging_wrapper,
rm=bing_rm)
```

The `CoStormRunner` instance can be evoked with the `warmstart()` and `step(...)` methods.
```python
# warm start the system to build shared conceptual space between Co-STORM and users
costorm_runner.warm_start()

# step through the collaborative discourse
# running either of code below in any order for as many times as you would like
# Either by observing the conversation via running
conv_turn = costorm_runner.step()
# Or by actively engaging in the conversation to inject your utterance via running
costorm_runner.step(user_utterance="YOUR UTTERANCE HERE")

# generate report
costorm_runner.knowledge_base.reogranize()
article = costorm_runner.generate_report()
print(article)
```
## Quick Start with Example Scripts
We provide scripts in our [examples folder](examples) as a quick start to run STORM with different configurations.
We provide scripts in our [examples folder](examples) as a quick start to run STORM and Co-STORM with different configurations.
We suggest using `secrets.toml` to set up the API keys. Create a file `secrets.toml` under the root directory and add the following content:
```shell
# Set up OpenAI API key.
OPENAI_API_KEY="your_openai_api_key"
# If you are using the API service provided by OpenAI, include the following line:
OPENAI_API_TYPE="openai"
# If you are using the API service provided by Microsoft Azure, include the following lines:
OPENAI_API_TYPE="azure"
AZURE_API_BASE="your_azure_api_base_url"
AZURE_API_VERSION="your_azure_api_version"
# Set up You.com search API key.
YDC_API_KEY="your_youcom_api_key"
```
### STORM examples
**To run STORM with `gpt` family models with default configurations:**
1. We suggest using `secrets.toml` to set up the API keys. Create a file `secrets.toml` under the root directory and add the following content:
```shell
# Set up OpenAI API key.
OPENAI_API_KEY="your_openai_api_key"
# If you are using the API service provided by OpenAI, include the following line:
OPENAI_API_TYPE="openai"
# If you are using the API service provided by Microsoft Azure, include the following lines:
OPENAI_API_TYPE="azure"
AZURE_API_BASE="your_azure_api_base_url"
AZURE_API_VERSION="your_azure_api_version"
# Set up You.com search API key.
YDC_API_KEY="your_youcom_api_key"
```
2. Run the following command.
```
python examples/run_storm_wiki_gpt.py \
--output-dir $OUTPUT_DIR \
--retriever you \
--do-research \
--do-generate-outline \
--do-generate-article \
--do-polish-article
```
Run the following command.
```bash
python examples/storm_examples/run_storm_wiki_gpt.py \
--output-dir $OUTPUT_DIR \
--retriever you \
--do-research \
--do-generate-outline \
--do-generate-article \
--do-polish-article
```
**To run STORM using your favorite language models or grounding on your own corpus:** Check out [examples/README.md](examples/README.md).
### Co-STORM examples
To run Co-STORM with `gpt` family models with default configurations,
1. Add `BING_SEARCH_API_KEY="xxx"`to `secrets.toml`
2. Run the following command
```bash
python examples/costorm_examples/run_costorm_gpt.py \
--output-dir $OUTPUT_DIR \
--retriever bing
```
## Customization of the Pipeline
### STORM
If you have installed the source code, you can customize STORM based on your own use case. STORM engine consists of 4 modules:
1. Knowledge Curation Module: Collects a broad coverage of information about the given topic.
Expand All @@ -169,12 +279,19 @@ If you have installed the source code, you can customize STORM based on your own
The interface for each module is defined in `knowledge_storm/interface.py`, while their implementations are instantiated in `knowledge_storm/storm_wiki/modules/*`. These modules can be customized according to your specific requirements (e.g., generating sections in bullet point format instead of full paragraphs).
### Co-STORM
If you have installed the source code, you can customize Co-STORM based on your own use case
1. Co-STORM introduces multiple LLM agent types (i.e. CoSTORM experts and Moderator). LLM agent interface is defined in `knowledge_storm/interface.py` , while its implementation are instantiated in `knowledge_storm/collaborative_storm/modules/co_storm_agents.py`. Different LLM agent policy can be customized.
2. Co-STORM introduces collaborative discourse protocol and the core function is turn policy management. We provide an example implementation of turn policy management through DiscourseManager in `knowledge_storm/collaborative_storm/engine.py`. It can be customized and further improved.
## Replicate NAACL2024 result
Please switch to the branch `NAACL-2024-code-backup` [here](https://github.com/stanford-oval/storm/tree/NAACL-2024-code-backup).
## Replicate Replicate STORM & Co-STORM paper result
For STORM paper experiments, please switch to the branch `NAACL-2024-code-backup` [here](https://github.com/stanford-oval/storm/tree/NAACL-2024-code-backup).
For Co-STORM paper experiments, please switch to the branch `EMNLP-2024-code-backup` (place holder right now, will be updated soon).
## Roadmap & Contributions
Our team is actively working on:
Expand All @@ -193,6 +310,16 @@ We are very grateful to [Michelle Lam](https://michelle123lam.github.io/) for de
## Citation
Please cite our paper if you use this code or part of it in your work:
```bibtex
@misc{jiang2024unknownunknowns,
title={Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations},
author={Yucheng Jiang and Yijia Shao and Dekun Ma and Sina J. Semnani and Monica S. Lam},
year={2024},
eprint={2408.15232},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2408.15232},
}
@inproceedings{shao2024assisting,
title={{Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models}},
author={Yijia Shao and Yucheng Jiang and Theodore A. Kanell and Peter Xu and Omar Khattab and Monica S. Lam},
Expand Down
Binary file added assets/co-storm-workflow.jpg
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 0734f7f

Please sign in to comment.