diff --git a/README.md b/README.md index be17ef34..8b03be28 100644 --- a/README.md +++ b/README.md @@ -5,11 +5,14 @@ # STORM: Synthesis of Topic Outlines through Retrieval and Multi-perspective Question Asking

-| Research preview | Paper | Website | +| Research preview | STORM Paper| Co-STORM Paper | Website |

- **Latest News** 🔥 +- [2024/09] Release Co-STORM codebase and intergrated into knowledge-storm python package. + +- [2024/09] Introduce collaborative STORM (Co-STORM) to support human-AI collaborative knowledge curation! [Co-STORM Paper](https://www.arxiv.org/abs/2408.15232) accepted to 2024 EMNLP. + - [2024/07] You can now install our package with `pip install knowledge-storm`! - [2024/07] We add `VectorRM` to support grounding on user-provided documents, complementing existing support of search engines (`YouRM`, `BingSearch`). (check out [#58](https://github.com/stanford-oval/storm/pull/58)) - [2024/07] We release demo light for developers a minimal user interface built with streamlit framework in Python, handy for local development and demo hosting (checkout [#54](https://github.com/stanford-oval/storm/pull/54)) @@ -24,17 +27,20 @@

-STORM is a LLM system that writes Wikipedia-like articles from scratch based on Internet search. +STORM is a LLM system that writes Wikipedia-like articles from scratch based on Internet search. Co-STORM further enhanced its feature by enabling human to collaborative LLM system to support more aligned and preferred information seeking and knowledge curation. While the system cannot produce publication-ready articles that often require a significant number of edits, experienced Wikipedia editors have found it helpful in their pre-writing stage. -**Try out our [live research preview](https://storm.genie.stanford.edu/) to see how STORM can help your knowledge exploration journey and please provide feedback to help us improve the system 🙏!** +**More than 70,000 people have tried our [live research preview](https://storm.genie.stanford.edu/). Try it out to see how STORM can help your knowledge exploration journey and please provide feedback to help us improve the system 🙏!** + +## How STORM / Co-STORM works -## How STORM works +### STORM STORM breaks down generating long articles with citations into two steps: + 1. **Pre-writing stage**: The system conducts Internet-based research to collect references and generates an outline. 2. **Writing stage**: The system uses the outline and references to generate the full-length article with citations.

@@ -47,7 +53,21 @@ STORM identifies the core of automating the research process as automatically co Based on the separation of the two stages, STORM is implemented in a highly modular way using [dspy](https://github.com/stanfordnlp/dspy). +### CO-STORM + +Co-STORM proposes collaborative discourse protocol which implements conversation turn policy management controlling smooth collaboration among +- **Co-STORM LLM experts**: This type of agent generate answers grounded on external knowledge sources and/or raise follow up questions based on discourse history. +- **Moderator**: This agent generate thought provoking questions taking inspiring from collection information provided by Co-STORM LLM experts. +- **Human user**: Human user will take the initiative to either (1) observe the discourse to gain deeper understanding of the topic, or (2) actively engaging in the conversation by injecting utterances to steer the discussion focus + +

+ +

+ +Co-STORM also maintains a dynamic updated **mind map**, which organize collected information into hirarchical concept structure, aiming to build a shared conceptual space between the human user and the system while reducing the mental load when having long in depth discourse. + +Co-STORM is also implemented in a highly modular way using [dspy](https://github.com/stanfordnlp/dspy). ## Installation @@ -70,9 +90,20 @@ You could also install the source code which allows you to modify the behavior o ## API -The STORM knowledge curation engine is defined as a simple Python `STORMWikiRunner` class. -As STORM is working in the information curation layer, you need to set up the information retrieval module and language model module to create a `STORMWikiRunner` instance. Here is an example of using You.com search engine and OpenAI models. +Currently, our package support: + +- `OpenAIModel`, `AzureOpenAIModel`, `ClaudeModel`, `VLLMClient`, `TGIClient`, `TogetherClient`, `OllamaClient`, `GoogleModel`, `DeepSeekModel`, `GroqModel` as language model components +- `YouRM`, `BingSearch`, `VectorRM`, `SerperRM`, `BraveRM`, `SearXNG`, `DuckDuckGoSearchRM`, `TavilySearchRM`, `GoogleSearch` as retrieval module components + +:star2: **PRs for integrating more language models into [knowledge_storm/lm.py](knowledge_storm/lm.py) and search engines/retrievers into [knowledge_storm/rm.py](knowledge_storm/rm.py) are highly appreciated!** + +Both STORM and Co-STORM are working in the information curation layer, you need to set up the information retrieval module and language model module to create their `Runner` classes respectively. + +### STORM + +The STORM knowledge curation engine is defined as a simple Python `STORMWikiRunner` class. Here is an example of using You.com search engine and OpenAI models. + ```python import os from knowledge_storm import STORMWikiRunnerArguments, STORMWikiRunner, STORMWikiLMConfigs @@ -101,12 +132,6 @@ rm = YouRM(ydc_api_key=os.getenv('YDC_API_KEY'), k=engine_args.search_top_k) runner = STORMWikiRunner(engine_args, lm_configs, rm) ``` -Currently, our package support: -- `OpenAIModel`, `AzureOpenAIModel`, `ClaudeModel`, `VLLMClient`, `TGIClient`, `TogetherClient`, `OllamaClient`, `GoogleModel`, `DeepSeekModel`, `GroqModel` as language model components -- `YouRM`, `BingSearch`, `VectorRM`, `SerperRM`, `BraveRM`, `SearXNG`, `DuckDuckGoSearchRM`, `TavilySearchRM`, `GoogleSearch` as retrieval module components - -:star2: **PRs for integrating more language models into [knowledge_storm/lm.py](knowledge_storm/lm.py) and search engines/retrievers into [knowledge_storm/rm.py](knowledge_storm/rm.py) are highly appreciated!** - The `STORMWikiRunner` instance can be evoked with the simple `run` method: ```python topic = input('Topic: ') @@ -125,41 +150,126 @@ runner.summary() - `do_generate_article`: if True, generate an article for the topic based on the outline and the collected information; otherwise, load the results. - `do_polish_article`: if True, polish the article by adding a summarization section and (optionally) removing duplicate content; otherwise, load the results. +### Co-STORM + +The Co-STORM knowledge curation engine is defined as a simple Python `STORMWikiRunner` class. Here is an example of using Bing search engine and OpenAI models. + +```python +from knowledge_storm.collaborative_storm.engine import CollaborativeStormLMConfigs, RunnerArgument, CoStormRunner +from knowledge_storm.lm import OpenAIModel +from knowledge_storm.logging_wrapper import LoggingWrapper +from knowledge_storm.rm import BingSearch + +# Co-STORM adopts the same multi LM system paradigm as STORM +lm_config: CollaborativeStormLMConfigs = CollaborativeStormLMConfigs() +openai_kwargs = { + "api_key": os.getenv("OPENAI_API_KEY"), + "api_provider": "openai", + "temperature": 1.0, + "top_p": 0.9, + "api_base": None, +} +question_answering_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=1000, **openai_kwargs) +discourse_manage_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=500, **openai_kwargs) +utterance_polishing_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=2000, **openai_kwargs) +warmstart_outline_gen_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=500, **openai_kwargs) +question_asking_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=300, **openai_kwargs) +knowledge_base_lm = OpenAIModel(model=gpt_4o_model_name, max_tokens=1000, **openai_kwargs) + +lm_config.set_question_answering_lm(question_answering_lm) +lm_config.set_discourse_manage_lm(discourse_manage_lm) +lm_config.set_utterance_polishing_lm(utterance_polishing_lm) +lm_config.set_warmstart_outline_gen_lm(warmstart_outline_gen_lm) +lm_config.set_question_asking_lm(question_asking_lm) +lm_config.set_knowledge_base_lm(knowledge_base_lm) + +# Check out the CoSTORM's RunnerArguments class for more configurations. +topic = input('Topic: ') +runner_argument = RunnerArgument(topic=topic, ...) +logging_wrapper = LoggingWrapper(lm_config) +bing_rm = BingSearch(bing_search_api_key=os.environ.get("BING_SEARCH_API_KEY"), + k=runner_argument.retrieve_top_k) +costorm_runner = CoStormRunner(lm_config=lm_config, + runner_argument=runner_argument, + logging_wrapper=logging_wrapper, + rm=bing_rm) +``` + +The `CoStormRunner` instance can be evoked with the `warmstart()` and `step(...)` methods. + +```python +# warm start the system to build shared conceptual space between Co-STORM and users +costorm_runner.warm_start() + +# step through the collaborative discourse +# running either of code below in any order for as many times as you would like +# Either by observing the conversation via running +conv_turn = costorm_runner.step() +# Or by actively engaging in the conversation to inject your utterance via running +costorm_runner.step(user_utterance="YOUR UTTERANCE HERE") + +# generate report +costorm_runner.knowledge_base.reogranize() +article = costorm_runner.generate_report() +print(article) +``` + + ## Quick Start with Example Scripts -We provide scripts in our [examples folder](examples) as a quick start to run STORM with different configurations. +We provide scripts in our [examples folder](examples) as a quick start to run STORM and Co-STORM with different configurations. + +We suggest using `secrets.toml` to set up the API keys. Create a file `secrets.toml` under the root directory and add the following content: + +```shell +# Set up OpenAI API key. +OPENAI_API_KEY="your_openai_api_key" +# If you are using the API service provided by OpenAI, include the following line: +OPENAI_API_TYPE="openai" +# If you are using the API service provided by Microsoft Azure, include the following lines: +OPENAI_API_TYPE="azure" +AZURE_API_BASE="your_azure_api_base_url" +AZURE_API_VERSION="your_azure_api_version" +# Set up You.com search API key. +YDC_API_KEY="your_youcom_api_key" +``` + +### STORM examples **To run STORM with `gpt` family models with default configurations:** -1. We suggest using `secrets.toml` to set up the API keys. Create a file `secrets.toml` under the root directory and add the following content: - ```shell - # Set up OpenAI API key. - OPENAI_API_KEY="your_openai_api_key" - # If you are using the API service provided by OpenAI, include the following line: - OPENAI_API_TYPE="openai" - # If you are using the API service provided by Microsoft Azure, include the following lines: - OPENAI_API_TYPE="azure" - AZURE_API_BASE="your_azure_api_base_url" - AZURE_API_VERSION="your_azure_api_version" - # Set up You.com search API key. - YDC_API_KEY="your_youcom_api_key" - ``` -2. Run the following command. - ``` - python examples/run_storm_wiki_gpt.py \ - --output-dir $OUTPUT_DIR \ - --retriever you \ - --do-research \ - --do-generate-outline \ - --do-generate-article \ - --do-polish-article - ``` + +Run the following command. +```bash +python examples/storm_examples/run_storm_wiki_gpt.py \ + --output-dir $OUTPUT_DIR \ + --retriever you \ + --do-research \ + --do-generate-outline \ + --do-generate-article \ + --do-polish-article +``` **To run STORM using your favorite language models or grounding on your own corpus:** Check out [examples/README.md](examples/README.md). +### Co-STORM examples + +To run Co-STORM with `gpt` family models with default configurations, + +1. Add `BING_SEARCH_API_KEY="xxx"`to `secrets.toml` +2. Run the following command + +```bash +python examples/costorm_examples/run_costorm_gpt.py \ + --output-dir $OUTPUT_DIR \ + --retriever bing +``` + ## Customization of the Pipeline +### STORM + If you have installed the source code, you can customize STORM based on your own use case. STORM engine consists of 4 modules: 1. Knowledge Curation Module: Collects a broad coverage of information about the given topic. @@ -169,12 +279,19 @@ If you have installed the source code, you can customize STORM based on your own The interface for each module is defined in `knowledge_storm/interface.py`, while their implementations are instantiated in `knowledge_storm/storm_wiki/modules/*`. These modules can be customized according to your specific requirements (e.g., generating sections in bullet point format instead of full paragraphs). +### Co-STORM + +If you have installed the source code, you can customize Co-STORM based on your own use case + +1. Co-STORM introduces multiple LLM agent types (i.e. CoSTORM experts and Moderator). LLM agent interface is defined in `knowledge_storm/interface.py` , while its implementation are instantiated in `knowledge_storm/collaborative_storm/modules/co_storm_agents.py`. Different LLM agent policy can be customized. +2. Co-STORM introduces collaborative discourse protocol and the core function is turn policy management. We provide an example implementation of turn policy management through DiscourseManager in `knowledge_storm/collaborative_storm/engine.py`. It can be customized and further improved. -## Replicate NAACL2024 result -Please switch to the branch `NAACL-2024-code-backup` [here](https://github.com/stanford-oval/storm/tree/NAACL-2024-code-backup). +## Replicate Replicate STORM & Co-STORM paper result +For STORM paper experiments, please switch to the branch `NAACL-2024-code-backup` [here](https://github.com/stanford-oval/storm/tree/NAACL-2024-code-backup). +For Co-STORM paper experiments, please switch to the branch `EMNLP-2024-code-backup` (place holder right now, will be updated soon). ## Roadmap & Contributions Our team is actively working on: @@ -193,6 +310,16 @@ We are very grateful to [Michelle Lam](https://michelle123lam.github.io/) for de ## Citation Please cite our paper if you use this code or part of it in your work: ```bibtex +@misc{jiang2024unknownunknowns, + title={Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations}, + author={Yucheng Jiang and Yijia Shao and Dekun Ma and Sina J. Semnani and Monica S. Lam}, + year={2024}, + eprint={2408.15232}, + archivePrefix={arXiv}, + primaryClass={cs.CL}, + url={https://arxiv.org/abs/2408.15232}, +} + @inproceedings{shao2024assisting, title={{Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models}}, author={Yijia Shao and Yucheng Jiang and Theodore A. Kanell and Peter Xu and Omar Khattab and Monica S. Lam}, diff --git a/assets/co-storm-workflow.jpg b/assets/co-storm-workflow.jpg new file mode 100644 index 00000000..23b80926 Binary files /dev/null and b/assets/co-storm-workflow.jpg differ