Marcela M Matos, Elina Mattila, António L. Palmeira, Sofus C. Larsen, Berit L. Heitmann, Alice Dunning, Graham W. Horgan, R. James Stubbs, Lauren Ramsay, Jorge Encantado, Liisa Lähteenmäki, Marja Harjumaa, Susanne Hansen, Marta M. Marques, Miikka Ermes, Cristiana Duarte, Marlene N. Silva, Lise Bundgaard, Jake Turicchi, Ruairi O'Driscoll, Inês Santos, and Repositório da Universidade de Lisboa
©Elina Mattila, Susanne Hansen, Lise Bundgaard, Lauren Ramsey, Alice Dunning, Marlene N Silva, Marja Harjumaa, Miikka Ermes, Marta M Marques, Marcela Matos, Sofus C Larsen, Jorge Encantado, Inês Santos, Graham Horgan, Ruairi O'Driscoll, Jake Turicchi, Cristiana Duarte, António L Palmeira, R James Stubbs, Berit Lilienthal Heitmann, Liisa Lähteenmäki. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 10.01.2022. This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included., Background: Digital behavior change interventions (DBCIs) offer a promising channel for providing health promotion services. However, user experience largely determines whether they are used, which is a precondition for effectiveness. Objective: The primary aim of this study is to evaluate user experiences with the NoHoW Toolkit (TK)-a DBCI that targets weight loss maintenance-over a 12-month period by using a mixed methods approach and to identify the main strengths and weaknesses of the TK and the external factors affecting its adoption. The secondary aim is to objectively describe the measured use of the TK and its association with user experience. Methods: An 18-month, 2×2 factorial randomized controlled trial was conducted. The trial included 3 intervention arms receiving an 18-week active intervention and a control arm. The user experience of the TK was assessed quantitatively through electronic questionnaires after 1, 3, 6, and 12 months of use. The questionnaires also included open-ended items that were thematically analyzed. Focus group interviews were conducted after 6 months of use and thematically analyzed to gain deeper insight into the user experience. Log files of the TK were used to evaluate the number of visits to the TK, the total duration of time spent in the TK, and information on intervention completion. Results: The usability level of the TK was rated as satisfactory. User acceptance was rated as modest; this declined during the trial in all the arms, as did the objectively measured use of the TK. The most appreciated features were weekly emails, graphs, goal setting, and interactive exercises. The following 4 themes were identified in the qualitative data: engagement with features, decline in use, external factors affecting user experience, and suggestions for improvements. Conclusions: The long-term user experience of the TK highlighted the need to optimize the technical functioning, appearance, and content of the DBCI before and during the trial, similar to how a commercial app would be optimized. In a trial setting, the users should be made aware of how to use the intervention and what its requirements are, especially when there is more intensive intervention content., This project has received funding from the European Union’s Horizon 2020 research and innovation program under grant agreement number 643309.