1. PEER: Expertizing Domain-Specific Tasks with a Multi-Agent Framework and Tuning Methods
- Author
-
Wang, Yiying, Li, Xiaojing, Wang, Binzhu, Zhou, Yueyang, Lin, Yingru, Ji, Han, Chen, Hong, Zhang, Jinshi, Yu, Fei, Zhao, Zewei, Jin, Song, Gong, Renji, and Xu, Wanqing
- Subjects
Computer Science - Artificial Intelligence - Abstract
In domain-specific applications, GPT-4, augmented with precise prompts or Retrieval-Augmented Generation (RAG), shows notable potential but faces the critical tri-lemma of performance, cost, and data privacy. High performance requires sophisticated processing techniques, yet managing multiple agents within a complex workflow often proves costly and challenging. To address this, we introduce the PEER (Plan, Execute, Express, Review) multi-agent framework. This systematizes domain-specific tasks by integrating precise question decomposition, advanced information retrieval, comprehensive summarization, and rigorous self-assessment. Given the concerns of cost and data privacy, enterprises are shifting from proprietary models like GPT-4 to custom models, striking a balance between cost, security, and performance. We developed industrial practices leveraging online data and user feedback for efficient model tuning. This study provides best practice guidelines for applying multi-agent systems in domain-specific problem-solving and implementing effective agent tuning strategies. Our empirical studies, particularly in the financial question-answering domain, demonstrate that our approach achieves 95.0% of GPT-4's performance, while effectively managing costs and ensuring data privacy.
- Published
- 2024