Merge branch 'dev' into code_intepreter

Author: garylin2099

.github/workflows/build-package.yaml

@@ -0,0 +1,34 @@
+name: Build and upload python package
+
+on:
+  release:
+    types: [created]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python
+      uses: actions/setup-python@v4
+      with:
+        python-version: '3.9'
+        cache: 'pip'
+    - name: Install dependencies
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt
+        pip install -e.
+        pip install setuptools wheel twine
+    - name: Set package version
+      run: |
+        export VERSION="${GITHUB_REF#refs/tags/v}"
+        sed -i "s/version=.*/version=\"${VERSION}\",/" setup.py
+    - name: Build and publish
+      env:
+        TWINE_USERNAME: __token__
+        TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }}
+      run: |
+        python setup.py bdist_wheel sdist
+        twine upload dist/*

.github/workflows/unittest.yaml

@@ -50,6 +50,7 @@ jobs:
       run: |
         export ALLOW_OPENAI_API_CALL=0
         echo "${{ secrets.METAGPT_KEY_YAML }}" | base64 -d > config/key.yaml
+        mkdir -p ~/.metagpt && echo "${{ secrets.METAGPT_CONFIG2_YAML }}" | base64 -d > ~/.metagpt/config2.yaml
         pytest tests/ --doctest-modules --cov=./metagpt/ --cov-report=xml:cov.xml --cov-report=html:htmlcov --durations=20 | tee unittest.txt
     - name: Show coverage report
       run: |

.gitignore

@@ -176,5 +176,6 @@ htmlcov.*
 cov.xml
 *.dot
 *.pkl
+*.faiss
 *-structure.csv
 *-structure.json

README.md

@@ -6,16 +6,16 @@
 </p>
 
 <p align="center">
-<b>Assign different roles to GPTs to form a collaborative software entity for complex tasks.</b>
+<b>Assign different roles to GPTs to form a collaborative entity for complex tasks.</b>
 </p>
 
 <p align="center">
 <a href="docs/README_CN.md"><img src="https://img.shields.io/badge/文档-中文版-blue.svg" alt="CN doc"></a>
 <a href="README.md"><img src="https://img.shields.io/badge/document-English-blue.svg" alt="EN doc"></a>
 <a href="docs/README_JA.md"><img src="https://img.shields.io/badge/ドキュメント-日本語-blue.svg" alt="JA doc"></a>
-<a href="https://discord.gg/DYn29wFk9z"><img src="https://dcbadge.vercel.app/api/server/DYn29wFk9z?style=flat" alt="Discord Follow"></a>
 <a href="https://opensource.org/licenses/MIT"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT"></a>
 <a href="docs/ROADMAP.md"><img src="https://img.shields.io/badge/ROADMAP-路线图-blue" alt="roadmap"></a>
+<a href="https://discord.gg/DYn29wFk9z"><img src="https://dcbadge.vercel.app/api/server/DYn29wFk9z?style=flat" alt="Discord Follow"></a>
 <a href="https://twitter.com/MetaGPT_"><img src="https://img.shields.io/twitter/follow/MetaGPT?style=social" alt="Twitter Follow"></a>
 </p>
 
@@ -25,19 +25,31 @@
    <a href="https://huggingface.co/spaces/deepwisdom/MetaGPT" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20-Hugging%20Face-blue?color=blue&logoColor=white" /></a>
 </p>
 
+## News
+🚀 Jan. 16, 2024: Our paper [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
+](https://arxiv.org/abs/2308.00352) accepted for oral presentation **(top 1.2%)** at ICLR 2024, **ranking #1** in the LLM-based Agent category.
+
+🚀 Jan. 03, 2024: [v0.6.0](https://github.com/geekan/MetaGPT/releases/tag/v0.6.0) released, new features include serialization, upgraded OpenAI package and supported multiple LLM, provided [minimal example for debate](https://github.com/geekan/MetaGPT/blob/main/examples/debate_simple.py) etc.
+
+🚀 Dec. 15, 2023: [v0.5.0](https://github.com/geekan/MetaGPT/releases/tag/v0.5.0) released, introducing some experimental features such as **incremental development**, **multilingual**, **multiple programming languages**, etc.
+
+🔥 Nov. 08, 2023: MetaGPT is selected into [Open100: Top 100 Open Source achievements](https://www.benchcouncil.org/evaluation/opencs/annual.html).
+
+🔥 Sep. 01, 2023: MetaGPT tops GitHub Trending Monthly for the **17th time** in August 2023.
+
+🌟 Jun. 30, 2023: MetaGPT is now open source.
+
+🌟 Apr. 24, 2023: First line of MetaGPT code committed.
+
+## Software Company as Multi-Agent System
+
 1. MetaGPT takes a **one line requirement** as input and outputs **user stories / competitive analysis / requirements / data structures / APIs / documents, etc.**
 2. Internally, MetaGPT includes **product managers / architects / project managers / engineers.** It provides the entire process of a **software company along with carefully orchestrated SOPs.**
    1. `Code = SOP(Team)` is the core philosophy. We materialize SOP and apply it to teams composed of LLMs.
 
 ![A software company consists of LLM-based roles](docs/resources/software_company_cd.jpeg)
 
-<p align="center">Software Company Multi-Role Schematic (Gradually Implementing)</p>
-
-## News
-🚀 Jan 03: Here comes [v0.6.0](https://github.com/geekan/MetaGPT/releases/tag/v0.6.0)! In this version, we added serialization and deserialization of important objects and enabled breakpoint recovery. We upgraded OpenAI package to v1.6.0 and supported Gemini, ZhipuAI, Ollama, OpenLLM, etc. Moreover, we provided extremely simple examples where you need only 7 lines to implement a general election [debate](https://github.com/geekan/MetaGPT/blob/main/examples/debate_simple.py). Check out more details [here](https://github.com/geekan/MetaGPT/releases/tag/v0.6.0)!
-
-
-🚀 Dec 15: [v0.5.0](https://github.com/geekan/MetaGPT/releases/tag/v0.5.0) is released! We introduced **incremental development**, facilitating agents to build up larger projects on top of their previous efforts or existing codebase. We also launched a whole collection of important features, including **multilingual support** (experimental), multiple **programming languages support** (experimental), **incremental development** (experimental), CLI support, pip support, enhanced code review, documentation mechanism, and optimized messaging mechanism!
+<p align="center">Software Company Multi-Agent Schematic (Gradually Implementing)</p>
 
 ## Install
 

config/config.yaml

@@ -0,0 +1,3 @@
+llm:
+  api_key: "YOUR_API_KEY"
+  model: "gpt-3.5-turbo-1106"

config/config2.yaml

@@ -0,0 +1,42 @@
+llm:
+  api_type: "openai"
+  base_url: "YOUR_BASE_URL"
+  api_key: "YOUR_API_KEY"
+  model: "gpt-3.5-turbo-1106"  # or gpt-4-1106-preview
+
+proxy: "YOUR_PROXY"
+
+search:
+  api_type: "google"
+  api_key: "YOUR_API_KEY"
+  cse_id: "YOUR_CSE_ID"
+
+mermaid:
+  engine: "pyppeteer"
+  path: "/Applications/Google Chrome.app"
+
+redis:
+  host: "YOUR_HOST"
+  port: 32582
+  password: "YOUR_PASSWORD"
+  db: "0"
+
+s3:
+  access_key: "YOUR_ACCESS_KEY"
+  secret_key: "YOUR_SECRET_KEY"
+  endpoint: "YOUR_ENDPOINT"
+  secure: false
+  bucket: "test"
+
+
+AZURE_TTS_SUBSCRIPTION_KEY: "YOUR_SUBSCRIPTION_KEY"
+AZURE_TTS_REGION: "eastus"
+
+IFLYTEK_APP_ID: "YOUR_APP_ID"
+IFLYTEK_API_KEY: "YOUR_API_KEY"
+IFLYTEK_API_SECRET: "YOUR_API_SECRET"
+
+METAGPT_TEXT_TO_IMAGE_MODEL_URL: "YOUR_MODEL_URL"
+
+PYPPETEER_EXECUTABLE_PATH: "/Applications/Google Chrome.app"
+

config/config2.yaml.example

@@ -9,24 +9,22 @@ Enable MetaGPT to self-evolve, accomplishing self-training, fine-tuning, optimiz
 
 1. Become the multi-agent framework with the highest ROI.
 2. Support fully automatic implementation of medium-sized projects (around 2000 lines of code).
-3. Implement most identified tasks, reaching version 0.5.
+3. Implement most identified tasks, reaching version 1.0.
 
 ### Tasks
 
-To reach version v0.5, approximately 70% of the following tasks need to be completed.
-
 1. Usability
    1. ~~Release v0.01 pip package to try to solve issues like npm installation (though not necessarily successfully)~~ (v0.3.0)
-   2. Support for overall save and recovery of software companies
+   2. ~~Support for overall save and recovery of software companies~~ (v0.6.0)
    3. ~~Support human confirmation and modification during the process~~ (v0.3.0) New: Support human confirmation and modification with fewer constrainsts and a more user-friendly interface
    4. Support process caching: Consider carefully whether to add server caching mechanism
    5. ~~Resolve occasional failure to follow instruction under current prompts, causing code parsing errors, through stricter system prompts~~ (v0.4.0, with function call)
    6. Write documentation, describing the current features and usage at all levels (ongoing, continuously adding contents to [documentation site](https://docs.deepwisdom.ai/main/en/guide/get_started/introduction.html))
    7. ~~Support Docker~~
 2. Features
-   1. Support a more standard and stable parser (need to analyze the format that the current LLM is better at)
-   2. ~~Establish a separate output queue, differentiated from the message queue~~
-   3. Attempt to atomize all role work, but this may significantly increase token overhead
+   1. ~~Support a more standard and stable parser (need to analyze the format that the current LLM is better at)~~ (v0.5.0)
+   2. ~~Establish a separate output queue, differentiated from the message queue~~ (v0.5.0)
+   3. ~~Attempt to atomize all role work, but this may significantly increase token overhead~~ (v0.5.0)
    4. Complete the design and implementation of module breakdown
    5. Support various modes of memory: clearly distinguish between long-term and short-term memory
    6. Perfect the test role, and carry out necessary interactions with humans
@@ -43,10 +41,10 @@ To reach version v0.5, approximately 70% of the following tasks need to be compl
 4. Actions
    1. ~~Implementation: Search~~ (v0.2.1)
    2. Implementation: Knowledge search, supporting 10+ data formats
-   3. Implementation: Data EDA (expected v0.6.0)
-   4. Implementation: Review
-   5. ~~Implementation~~: Add Document (v0.5.0)
-   6. ~~Implementation~~: Delete Document (v0.5.0)
+   3. Implementation: Data EDA (expected v0.7.0)
+   4. Implementation: Review & Revise (expected v0.7.0)
+   5. ~~Implementation: Add Document~~ (v0.5.0)
+   6. ~~Implementation: Delete Document~~ (v0.5.0)
    7. Implementation: Self-training
    8. ~~Implementation: DebugError~~ (v0.2.1)
    9. Implementation: Generate reliable unit tests based on YAPI
@@ -64,23 +62,22 @@ To reach version v0.5, approximately 70% of the following tasks need to be compl
    3. ~~Support Playwright apis~~
 7. Roles
    1. Perfect the action pool/skill pool for each role
-   2. Red Book blogger
-   3. E-commerce seller
-   4. Data analyst (expected v0.6.0)
-   5. News observer
-   6. ~~Institutional researcher~~ (v0.2.1)
+   2. E-commerce seller
+   3. Data analyst (expected v0.7.0)
+   4. News observer
+   5. ~~Institutional researcher~~ (v0.2.1)
 8. Evaluation
    1. Support an evaluation on a game dataset (experimentation done with game agents)
    2. Reproduce papers, implement full skill acquisition for a single game role, achieving SOTA results (experimentation done with game agents)
-   3. Support an evaluation on a math dataset (expected v0.6.0)
+   3. Support an evaluation on a math dataset (expected v0.7.0)
    4. Reproduce papers, achieving SOTA results for current mathematical problem solving process
 9. LLM
    1. Support Claude underlying API
    2. ~~Support Azure asynchronous API~~
    3. Support streaming version of all APIs
    4. ~~Make gpt-3.5-turbo available (HARD)~~
 10. Other
-    1. Clean up existing unused code
-    2. Unify all code styles and establish contribution standards
-    3. Multi-language support
-    4. Multi-programming-language support
+    1. ~~Clean up existing unused code~~
+    2. ~~Unify all code styles and establish contribution standards~~
+    3. ~~Multi-language support~~
+    4. ~~Multi-programming-language support~~