1. Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models
- Author
-
Dhillon, Paramveer S., Molaei, Somayeh, Li, Jiaqi, Golub, Maximilian, Zheng, Shaochun, and Robert, Lionel P.
- Subjects
Computer Science - Human-Computer Interaction ,Computer Science - Computation and Language - Abstract
Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time). While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users. No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scaffolding mechanisms., Comment: Appearing at CHI 2024 (Honolulu, HI)
- Published
- 2024