Democratic elections need a well-informed public. With the media environment increasingly influenced by algorithms, science and society need new tools to evaluate such algorithmic influence. In this work in progress paper, we study search engine bias in the context of the German federal elections in 2021. We combine a representative survey (N = 1,641) focused on information strategies and large-scale automated retrieval of search results by agent-based testing (ABT; 293,474 searches with 2,498,764 search results) to introduce an audience-centered approach to algorithmic auditing. This approach allows us to study both how voters use search engines in the run-up to elections and the content of the results they are shown. Preliminary results point towards a strong influence of different information-seeking strategies (i.e., looking for specific party, candidate, or issue information, or more general election guidance) on the structural diversity of search results. We outline further analysis strategies based on our dataset to investigate potential effects of different individual pathways to information on the structural diversity of search results., The project and this paper were funded by the Bavarian Research Institure for Digital Transformation., {"references":["Atkin, C. (1973). Instrumental Utilities and Information Seeking. In P. Clarke (Ed.), New Models for Mass Communication (pp. 205–242). SAGE Publications.","Bandy, J. (2021). Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits. ArXiv:2102.04256 [Cs]. http://arxiv.org/abs/2102.04256.","Bozdag, E., & van den Hoven, J. (2015). Breaking the filter bubble: Democracy and design. Ethics and Information Technology, 17(4), 249–265. https://doi.org/10.1007/s10676-015-9380-y.","Campbell, A., Converse, P. E., Miller, W. E., & Stokes, D. E. (1960). The american voter. University of Chicago Press.","Dieter, P. (2021). Keine Vorteile mehr durch AMP: Google vollzieht Kehrtwende. t3n Magazin. https://t3n.de/news/google-amp-vs-page-experience-1379793.","Dommett, K., & Verovšek, P. J. (2021). Promoting Democracy in the Digital Public Sphere: Applying Theoretical Ideals to Online Political Communication. Javnost - The Public, 1–17. https://doi.org/10.1080/13183222.2021.1955206.","Dutton, W. H., Reisdorf, B. C., Dubois, E., & Blank, G. (2017). Search and Politics: The Uses and Impacts of Search in Britain, France, Germany, Italy, Poland, Spain, and the United States. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.2960697.","Epstein, R., & Robertson, R. E. (2015). The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the National Academy of Sciences, 112(33), E4512–E4521. https://doi.org/10.1073/pnas.1419828112.","Epstein, R., Robertson, R. E., Lazer, D., & Wilson, C. (2017). Suppressing the Search Engine Manipulation Effect (SEME). Proceedings of the ACM on Human-Computer Interaction, 1(CSCW), 1–22. https://doi.org/10.1145/3134677.","Erlass über die Bundeszentrale für politische Bildung. (2001). bpb.de. https://www.bpb.de/die-bpb/51244/der-bpb-erlass.","Haim, M. (2019). ScrapeBot. A Selenium-based tool for agent-based testing. https://github.com/MarHai/ScrapeBot.","Haim, M. (2020). Agent-based testing: An automated approach toward artificial reactions to human behavior. Journalism Studies, 21(7), 895–911. https://doi.org/10.1080/1461670X.2019.1702892.","Haim, M., & Jungblut, M. (2021). Politicians' Self-depiction and Their News Portrayal: Evidence from 28 Countries Using Visual Computational Analysis. Political Communication, 38(1–2), 55–74. https://doi.org/10.1080/10584609.2020.1753869","Helberger, N. (2019). On the Democratic Role of News Recommenders. Digital Journalism, 7(8), 993–1012. https://doi.org/10.1080/21670811.2019.1623700.","Hu, D., Jiang, S., E. Robertson, R., & Wilson, C. (2019). Auditing the Partisanship of Google Search Snippets. The World Wide Web Conference on - WWW '19, 693–704. https://doi.org/10.1145/3308558.3313654.","Introna, L. D., & Nissenbaum, H. (2000). Shaping the web: Why the politics of search engines matters. The Information Society, 16(3), 169–185. https://doi.org/10.1080/01972240050133634.","Kawakami, A., Umarova, K., & Mustafaraj, E. (2020). The Media Coverage of the 2020 US Presidential Election Candidates through the Lens of Google's Top Stories [Data set]. Harvard Dataverse. https://doi.org/10.7910/DVN/0ZLHOK.","Kitchin, R. (2017). Thinking critically about and researching algorithms. Information, Communication & Society, 20(1), 14–29. https://doi.org/10.1080/1369118X.2016.1154087.","Kliman-Silver, C., Hannak, A., Lazer, D., Wilson, C., & Mislove, A. (2015). Location, Location, Location: The Impact of Geolocation on Web Search Personalization. Proceedings of the 2015 Internet Measurement Conference, 121–127. https://doi.org/10.1145/2815675.2815714.","Kravets, D., & Toepfl, F. (2021). Gauging reference and source bias over time: How Russia's partially state-controlled search engine Yandex mediated an anti-regime protest event. Information, Communication & Society, 1–17. https://doi.org/10.1080/1369118X.2021.1933563.","Kulshrestha, J., Eslami, M., Messias, J., Zafar, M. B., Ghosh, S., Gummadi, K. P., & Karahalios, K. (2017). Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media. Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, 417–432. https://doi.org/10.1145/2998181.2998321.","Kulshrestha, J., Eslami, M., Messias, J., Zafar, M. B., Ghosh, S., Gummadi, K. P., & Karahalios, K. (2019). Search bias quantification: Investigating political bias in social media and web search. Information Retrieval Journal, 22(1–2), 188–227. https://doi.org/10.1007/s10791-018-9341-2.","Metaxa, D., Park, J. S., Landay, J. A., & Hancock, J. (2019). Search Media and Elections: A Longitudinal Investigation of Political Search Results. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1–17. https://doi.org/10.1145/3359231.","Mittelstadt, B. (2016). Auditing for Transparency in Content Personalization Systems. International Journal of Communication 10(2016), 4991–5002. https://ijoc.org/index.php/ijoc/article/view/6267/1808.","Möller, J., van de Velde, R. N., Merten, L., & Puschmann, C. (2020). Explaining Online News Engagement Based on Browsing Behavior: Creatures of Habit? Social Science Computer Review, 38(5), 616–632. https://doi.org/10.1177/0894439319828012.","More time, tools, and details on the page experience update. (2021). Google Developers. https://developers.google.com/search/blog/2021/04/more-details-page-experience.","Mustafaraj, E., Lurie, E., & Devine, C. (2020). The case for voter-centered audits of search engines during political elections. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 559–569. https://doi.org/10.1145/3351095.3372835.","Nechushtai, E., & Lewis, S. C. (2019). What kind of news gatekeepers do we want machines to be? Filter bubbles, fragmentation, and the normative dimensions of algorithmic recommendations. Computers in Human Behavior, 90, 298–307. https://doi.org/10.1016/j.chb.2018.07.043.","Nielsen, R. K. (2016). News media, search engines and social networking SITES as varieties of online gatekeepers. In Rethinking Journalism Again. Routledge.","Puschmann, C. (2019). Beyond the Bubble: Assessing the Diversity of Political Search Results. Digital Journalism, 7(6), 824–843. https://doi.org/10.1080/21670811.2018.1539626.","Robertson, R. E., Jiang, S., Joseph, K., Friedland, L., Lazer, D., & Wilson, C. (2018). Auditing Partisan Audience Bias within Google Search. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1–22. https://doi.org/10.1145/3274417.","Robertson, R. E., Lazer, D., & Wilson, C. (2018). Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages. Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18, 955–965. https://doi.org/10.1145/3178876.3186143.","Sandvig, C., Hamilton, K., Karahalios, K., & Langbort, C. (2014). Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms. Paper presented to \"Data and Discrimination: Converting Critical Concerns into Productive Inquiry,\" a preconference at the 64th Annual Meeting of the International Communication Association. May 22, 2014.","Search Engine Market Share Germany. (2021). StatCounter Global Stats. https://gs.statcounter.com/search-engine-market-share/all/germany.","Steiner, M., Magin, M., Stark, B., & Geiß, S. (2020). Seek and you shall find? A content analysis on the diversity of five search engines' results on political queries. Information, Communication & Society, 1–25. https://doi.org/10.1080/1369118X.2020.1776367.","Thurman, N., Moeller, J., Helberger, N., & Trilling, D. (2019). My Friends, Editors, Algorithms, and I: Examining audience attitudes to news selection. Digital Journalism, 7(4), 447–469. https://doi.org/10.1080/21670811.2018.1493936.","Unkel, J., & Haim, M. (2021). Googling Politics: Parties, Sources, and Issue Ownerships on Google in the 2017 German Federal Election Campaign. Social Science Computer Review, 39(5), 844–861. https://doi.org/10.1177/0894439319881634.","Urman, A., Makhortykh, M., & Ulloa, R. (2021). The Matter of Chance: Auditing Web Search Results Related to the 2020 U.S. Presidential Primary Elections Across Six Search Engines. Social Science Computer Review, 089443932110068. https://doi.org/10.1177/08944393211006863.","Van Aelst, P., Strömbäck, J., Aalberg, T., Esser, F., de Vreese, C., Matthes, J., Hopmann, D., Salgado, S., Hubé, N., Stępińska, A., Papathanassopoulos, S., Berganza, R., Legnante, G., Reinemann, C., Sheafer, T., & Stanyer, J. (2017). Political communication in a high-choice media environment: A challenge for democracy? Annals of the International Communication Association, 41(1), 3–27. https://doi.org/10.1080/23808985.2017.1288551."]}