Search

Large language models (LLMs) such as OpenAI's ChatGPT, Google's Bard, and Microsoft's Bing Chat have shown potential as educational tools in the medical and dental fields. This study evaluated their effectiveness using questions from the Japanese national dental hygienist examination, focusing on textual information only. We analyzed 73 questions from the 32nd Japanese national dental hygienist examination, conducted in March 2023, using LLMs ChatGPT-3.5, GPT-4, Bard, and Bing Chat. Each question was categorized into one of nine domains. Standardized prompts were used for all LLMs, and Fisher's exact test was applied for statistical analysis. GPT-4 achieved the highest accuracy (75.3%), followed by Bing (68.5%), Bard (66.7%), and GPT-3.5 (63.0%). There were no statistically significant differences between the LLMs. The performance varied across different question categories, with all models excelling in the 'Disease mechanism and promotion of recovery process' category (100% accuracy). GPT-4 generally outperformed other models, especially in multi-answer questions. GPT-4 demonstrated the highest overall accuracy among the LLMs tested, indicating its superior potential as an educational support tool in dental hygiene studies. The study highlights the varied performance of different LLMs across various question categories. While GPT-4 is currently the most effective, the capabilities of LLMs in educational settings are subject to continual change and improvement. [ABSTRACT FROM AUTHOR]

Recently, automatic question generation (AQG) has been researched extensively for educational purposes. Existing approaches generally lack relevant information on the authentic context and problem diversity with various difficulty levels, so we proposed a new AQG system for generating contextualized and personalized mathematic word problems (MWP) in authentic contexts using the Generative Pre-trained Transformers (GPT). Our proposed system comprises (1) authentic contextual information acquisition through image recognition by TensorFlow and augmented reality (AR) measurement by AR Core, (2) a personalized mechanism based on instructional prompts to generate three different difficulty levels for learners' different needs, and (3) MWP generation through GPT with authentic contextual information and personalized needs. We conducted a quasi-experiment with the participation of 52 students to evaluate the effectiveness of the proposed system on geometry learning performance. Further, the learning behaviors were analyzed in the aspects of authentic context, mathematics, and reflective behavior. The findings showed better results in geometry learning performances from students who learned with contextualized and personalized MWPs than those who were taught without contextualization and personalization on MWPs. Moreover, it was found that student's ability to comprehend the practical situation or scenario presented in a problem (problem context understanding) and students' ability to recognize relevant information from the problem context (identifying contextual information) significantly improved their learning performance. Moreover, students' ability to apply math concepts and solve medium-level MWP also contributes to the improvement of learning performance. Meanwhile, learners showed positive perceptions toward the proposed system in facilitating geometry learning. Therefore, it is useful to promote an authentic context setting for mathematical problem-solving. [ABSTRACT FROM AUTHOR]

This study aimed to determine the accuracy of large language models (LLMs) in answering oral and maxillofacial surgery (OMS) multiple choice questions. A total of 259 questions from the university's question bank were answered by the LLMs (GPT-3.5, GPT-4, Llama 2, Gemini, and Copilot). The scores per category as well as the total score out of 259 were recorded and evaluated, with the passing score set at 50%. The mean overall score amongst all LLMs was 62.5%. GPT-4 performed the best (76.8%, 95% confidence interval (CI) 71.4–82.2%), followed by Copilot (72.6%, 95% CI 67.2–78.0%), GPT-3.5 (62.2%, 95% CI 56.4–68.0%), Gemini (58.7%, 95% CI 52.9–64.5%), and Llama 2 (42.5%, 95% CI 37.1–48.6%). There was a statistically significant difference between the scores of the five LLMs overall (χ2 = 79.9, df = 4, P < 0.001) and within all categories except 'basic sciences' (P = 0.129), 'dentoalveolar and implant surgery' (P = 0.052), and 'oral medicine/pathology/radiology' (P = 0.801). The LLMs performed best in 'basic sciences' (68.9%) and poorest in 'pharmacology' (45.9%). The LLMs can be used as adjuncts in teaching, but should not be used for clinical decision-making until the models are further developed and validated. [ABSTRACT FROM AUTHOR]

The article presents the discussion on use of AI-based chatbots like OpenAI's ChatGPT in the chemical engineering industry, highlighting both the potential benefits and the challenges of deploying such technologies. Topics include recent updates to ChatGPT, differences between paid and free versions; and case studies demonstrating its application in process safety.

Large language models are capable of generating source code in a zero-shot manner to develop programs that meet user functional requirements. However, when faced with scenarios involving complex business requirements, the generated source code may fail to satisfy user needs. Addressing the challenge of understanding software requirements, we propose a two-stage code generation approach. Initially, the large language model generates pseudocode based on the user's functional requirements, refined through an iterative process with user feedback. Subsequently, the model generates source code based on the finalized pseudocode. We conducted empirical studies on an open code generation dataset, and experimental results with models such as GPT-4, Claude Sonnect 3, and Geminipro 1.5 demonstrate that our method outperforms zero-shot prompt learning in scenarios with complex user requirements, with improvements in PASS@K reaching up to 15%. [ABSTRACT FROM AUTHOR]

This article explores the capabilities of Microsoft Co-Pilot, with support from the AI tool Grammarly, in summarizing and synthesizing scholarly content. The authors prompt Co-Pilot to write a commentary using executive summaries of articles related to special education administration. The AI tool generates summaries that highlight the potential of AI in enhancing decision-making processes, resource allocation, and educational outcomes in special education administration. The article also discusses the integration of AI in special education from the perspective of students with disabilities, the assessment of intellectual disability, and the ethical considerations of AI adoption in instructional design. The authors conclude that while AI holds promise in special education leadership, a balanced approach that incorporates human judgment and ethical considerations is necessary. [Extracted from the article]

Business Process Model and Notation (BPMN) is a standard for formally modeling complex business processes. Manual creation of BPMN models can be time-consuming and error-prone, prompting a need for automation. Existing approaches, such as rule-based methods, machine learning, and machine translation, have progressed but face accuracy and real-world applicability challenges. In this research paper, we propose a novel method for automated extraction of BPMN models from textual descriptions using natural language processing (NLP) tools and deep learning models, including the spaCy library for text processing, a fine-tuned BERT model, and state-of-the-art large language models like GPT-3.5-Turbo and GPT-4. We utilize Graphviz, an open-source graph visualization software, to visualize the extracted processes. Our method supports representing tasks, exclusive gateways, parallel gateways, and start and end events in the generated BPMN models. The evaluation of 31 textual descriptions shows that our method generates process models with 96% accuracy using GPT-4 and 80% accuracy using GPT-3.5-Turbo large language models. Although subject to certain limitations, such as occasional inaccuracies in model outputs and reliance on well-formed input text, our approach offers a valuable contribution to the growing body of research on automating BPMN model generation. [ABSTRACT FROM AUTHOR]

Preparing future teachers to meet the demands of an increasingly dynamic world is a paramount challenge. Academic writing and critical thinking skills influence foreign language teachers' professional growth, shaping their students' cognitive development. This paper explores the integration of ChatGPT (generative pre-trained transformer) as a tool to augment future foreign language teachers' academic writing and critical thinking skills. Through engagement with ChatGPT, future foreign language teachers have the opportunity to interact with a large language model that can provide targeted feedback, prompt thought-provoking discussions, and assist in refining professionally oriented competencies. This study examines the impact of ChatGPT on skills development through a comprehensive analysis of empirical data, pedagogical frameworks, and student experiences. A qualitative study conducted with 35 students enrolled in a foreign languages master's program utilized focus group interviews and thematic analysis. The findings highlight ChatGPT's limitations in content generation while also recognizing its potential to reduce research labor and assist in formatting and referencing tasks. The study underscores the importance of informed usage of ChatGPT, with students and educators recognizing the necessity of verifying information and maintaining academic integrity. [ABSTRACT FROM AUTHOR]

Copyright of Revista de Ciencias Sociales (13159518) is the property of Revista de Ciencias Sociales de la Universidad del Zulia Venezuela and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The Common Vulnerabilities and Exposures (CVE) system is a widely used standard for identifying and tracking known vulnerabilities in software systems. The severity of these vulnerabilities must be determined in order to prioritize mitigation efforts. However, assigning severity to a vulnerability is a challenging task that requires careful analysis of its characteristics and potential impact. Considering the vast number of vulnerabilities identified every year, it is vital to automate the severity assignment, thereby reducing manual effort. This paper proposes a novel approach for predicting the severity of vulnerabilities based on their CVE description using GPT-2, a state-of-the-art language model. The CVSS severity values distribution imbalance is addressed using oversampling and contextual data augmentation techniques. This approach leverages the large-scale language modeling capabilities of GPT-2 to automatically extract relevant features from CVE descriptions and predict the severity level of the vulnerability. The model is evaluated on a test data set of 7,765 CVEs and achieves a high accuracy of 84.2% and an F 1 score of 0.82 in predicting the severity of the vulnerabilities on the test data. A comparative analysis of this approach was done against state-of-the-art methods, demonstrating the superior performance of the proposed approach. Based on the results, the proposed approach could be considered a valuable tool for quickly and accurately identifying high-severity vulnerabilities, facilitating more efficient and effective vulnerability management practices. Furthermore, this approach could be extended to other natural language processing tasks related to vulnerability analysis and management. [ABSTRACT FROM AUTHOR]

Recent developments in computerized scoring via semantic distance have provided automated assessments of verbal creativity. Here, we extend past work, applying computational linguistic approaches to characterize salient features of creative text. We hypothesize that, in addition to semantic diversity, the degree to which a story includes perceptual details, thus transporting the reader to another time and place, would be predictive of creativity. Additionally, we explore the use of generative language models to supplement human data collection and examine the extent to which machine‐generated stories can mimic human creativity. We collect 600 short stories from human participants and GPT‐3, subsequently randomized and assessed on their creative quality. Results indicate that the presence of perceptual details, in conjunction with semantic diversity, is highly predictive of creativity. These results were replicated in an independent sample of stories (n = 120) generated by GPT‐4. We do not observe a significant difference between human and AI‐generated stories in terms of creativity ratings, and we also observe positive correlations between human and AI assessments of creativity. Implications and future directions are discussed. [ABSTRACT FROM AUTHOR]

The Common Vulnerabilities and Exposures (CVE) system is a widely used standard for identifying and tracking known vulnerabilities in software systems. The severity of these vulnerabilities must be determined in order to prioritize mitigation efforts. However, assigning severity to a vulnerability is a challenging task that requires careful analysis of its characteristics and potential impact. Considering the vast number of vulnerabilities identified every year, it is vital to automate the severity assignment, thereby reducing manual effort. This paper proposes a novel approach for predicting the severity of vulnerabilities based on their CVE description using GPT-2, a state-of-the-art language model. The CVSS severity values distribution imbalance is addressed using oversampling and contextual data augmentation techniques. This approach leverages the large-scale language modeling capabilities of GPT-2 to automatically extract relevant features from CVE descriptions and predict the severity level of the vulnerability. The model is evaluated on a test data set of 7,765 CVEs and achieves a high accuracy of 84.2% and an F 1 score of 0.82 in predicting the severity of the vulnerabilities on the test data. A comparative analysis of this approach was done against state-of-the-art methods, demonstrating the superior performance of the proposed approach. Based on the results, the proposed approach could be considered a valuable tool for quickly and accurately identifying high-severity vulnerabilities, facilitating more efficient and effective vulnerability management practices. Furthermore, this approach could be extended to other natural language processing tasks related to vulnerability analysis and management. [ABSTRACT FROM AUTHOR]

As technology progresses, there has been an increasing interest in using Chatbot GPT (Generative Pre-trained Transformer) in education. Chatbot GPT, or ChatGPT, gained one million users within the first week of launching in November 2022 and had amassed over 100 million active users by February 2023. This type of artificial intelligence uses natural language processing to convert it into a user. This paper presents a comprehensive analysis and review of 34 articles published on ChatGPT and its potential impact on education by utilizing the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) methodology. This review analyzed various studies and articles to examine the strengths and limitations of GPT language models in education from 2018 to the present. The advantages of ChatGPT include its capacity to provide personalized and adaptive learning, instant feedback, and improved accessibility. However, there are potential drawbacks, such as the lack of emotional intelligence, the risk of overreliance on technology, and privacy concerns. This review suggests that ChatGPT has significant promise for education yet reinforces the necessity for further research and careful consideration of possible risks and limitations. Specifically, it pointed out potential invisible manipulations by instructing ChatGPT to answer educationrelated topics. The paper concludes by discussing the implications of ChatGPT for the future of education and emphasizing the need for further research in this field. [ABSTRACT FROM AUTHOR]

Digitalization of all life spheres is the reality of modern world development. The global digitization creates a powerful information environment. Its navigation requires serious tools for structuring, systematizing and processing information. Digital processes, as the most progressive, are in constant development. Artificial Intelligence (AI) is gaining popularity. The latest product that has created a lot of discussion is chatGPT (Generative Pre-trained Transformer) from OpenAI, which is an artificial intelligence chatbot that demonstrates the ability of digital devices to perform the tasks inherent to intelligent beings. The paper shows some issues of using ChatGPT for making a new course of Law English, a curriculum, a syllabus at the tertiary level removing concerns related to its application and utilization. The purpose of the manuscript is to describe a real case of working out a professional English course for university students applying ChatGPT. The methods of analysis, synthesis, case study, expert assessment were used. The results are as follows: a new course for Law English training, a curriculum, a syllabus and a textbook using content created by ChatGPT were readied. The conclusion is that, nowadays, everything can be taught by a teacher partly with an AI. ChatGPT can be used for a wide variety of educational purposes, including providing information, generating necessary texts, tasks, tests, and answering many questions. Most of the mentioned ChatGPT services are important educational elements in language teaching and learning, they can be used for professional English course development. [ABSTRACT FROM AUTHOR]

As the influence of Incremental Sheet Forming (ISF) grows in manufacturing sectors, so does the demand for precise and updated knowledge construction in this domain. In this research, we evaluate the capability of Large Language Models (LLMs) to capture domain-specific knowledge, using ISF as a case study. Recognizing common LLMs' limitations such as potential inaccuracies and outdated information reliance, we propose a comprehensive approach involving automated and adaptive knowledge extraction, enrichment, and integration into an ISF-specific dataset. We then fine-tune the LLMs for ISF-related text classification and prompt response tasks. Our results reveal a significant enhancement in LLMs' performance within the ISF domain, with a domain knowledge acquisition rate exceeding that of GPT-3.5 by 10.4%, achieved by the fine-tuned Alpaca-33B model. Additionally, we introduce a novel conversational prototype designed to refine the accuracy and relevance of LLMs in the ISF domain. Our findings will guide future efforts in downstream tasks such as ISF-domain knowledge graph construction and quality prediction. [ABSTRACT FROM AUTHOR]

As new AI technologies such as Large Language Models (LLM) quickly evolve, the need for enhancing general-purpose LLMs with physical knowledge to better serve the manufacturing community has been increasingly recognized. This paper presents a method that tailors GPT-3.5 with domain-specific knowledge for intelligent aircraft maintenance. Specifically, aircraft ontology is investigated to curate maintenance logs with encoded component hierarchical structure to fine-tune GPT-3.5. Experimental results demonstrate the effectiveness of the developed method in accurately identifying defective components and providing consistent maintenance action recommendations, outperforming general-purpose GPT-3.5 and GPT-4.0. The method can be adapted to other domains in manufacturing and beyond. [ABSTRACT FROM AUTHOR]

Recent advances in generative pre-trained transformer large language models have emphasised the potential risks of unfair use of artificial intelligence (AI) generated content in an academic environment and intensified efforts in searching for solutions to detect such content. The paper examines the general functionality of detection tools for AI-generated text and evaluates them based on accuracy and error type analysis. Specifically, the study seeks to answer research questions about whether existing detection tools can reliably differentiate between human-written text and ChatGPT-generated text, and whether machine translation and content obfuscation techniques affect the detection of AI-generated text. The research covers 12 publicly available tools and two commercial systems (Turnitin and PlagiarismCheck) that are widely used in the academic setting. The researchers conclude that the available detection tools are neither accurate nor reliable and have a main bias towards classifying the output as human-written rather than detecting AI-generated text. Furthermore, content obfuscation techniques significantly worsen the performance of tools. The study makes several significant contributions. First, it summarises up-to-date similar scientific and non-scientific efforts in the field. Second, it presents the result of one of the most comprehensive tests conducted so far, based on a rigorous research methodology, an original document set, and a broad coverage of tools. Third, it discusses the implications and drawbacks of using detection tools for AI-generated text in academic settings. [ABSTRACT FROM AUTHOR]

Effective evidence synthesis is important for the integration of scientific research into decision-making. However, fully depicting the vast mosaic of concepts and applications in environmental sciences and ecology often entails a substantial workload. New Artificial Intelligence (AI) tools present an attractive option for addressing this challenge but require sufficient validation to match the vigorous standards of a systematic review. This article demonstrates the use of generative AI in the selection of relevant literature as part of a systematic review on indicators of ecosystem condition. We highlight, through the development of an optimal prompt to communicate inclusion and exclusion criteria, the need to describe ecosystem condition as a multidimensional concept whilst also maintaining clarity on what does not meet the criteria of comprehensiveness. We show that, although not completely infallible, the GPT-3.5 model significantly outperforms traditional literature screening processes in terms of speed and efficiency whilst correctly selecting 83 % of relevant literature for review. Our study highlights the importance of precision in prompt design and the setting of query parameters for the AI model and opens the perspective for future work using language models to contextualize complex concepts in the environmental sciences. Future development of this methodology in tandem with the continued evolution of the accessibility and capacity of AI tools presents a great potential to improve evidence synthesis through gains in efficiency and possible scope. • Developments in generative AI present a useful tool for evidence synthesis. • GPT-3.5 is an effective method for screening literature on Ecosystem Condition. • Clarity of prompt and repetition of key terms is crucial for accurate responses. • The iterative prompt development allows to sharpen key research terms. • AI approach mitigates against persistent subjectivity in the review process. [ABSTRACT FROM AUTHOR]

In the age of information overload, the ability to distill essential content from extensive texts is invaluable. DeepExtract introduces an advanced framework for extractive summarization, utilizing the groundbreaking capabilities of GPT-4 along with innovative hierarchical positional encoding to redefine information extraction. This manuscript details the development of DeepExtract, which integrates semantic-driven techniques to analyze and summarize complex documents effectively. The framework is structured around a novel hierarchical tree construction that categorizes sentences and sections not just by their physical placement within a text, but by their contextual and thematic significance, leveraging dynamic embeddings generated by GPT-4. We introduce a multi-faceted scoring system that evaluates sentences based on coherence, relevance, and novelty, ensuring that summaries are not only concise but rich with essential content. Further, DeepExtract employs optimized semantic clustering to group thematic elements, which enhances the representativeness of the summaries. This paper demonstrates through comprehensive evaluations that DeepExtract significantly outperforms existing extractive summarization models in terms of accuracy and efficiency, making it a potent tool for academic, professional, and general use. We conclude with a discussion on the practical applications of DeepExtract in various domains, highlighting its adaptability and potential in navigating the vast expanses of digital text. [ABSTRACT FROM AUTHOR]

The vast volume of currently available unstructured text data, such as research papers, news, and technical report data, shows great potential for ecological research. However, manual processing of such data is labour-intensive, posing a significant challenge. In this study, we aimed to assess the application of three state-of-the-art prompt-based large language models (LLMs), GPT-3.5, GPT-4, and LLaMA-2-70B, to automate the identification, interpretation, extraction, and structuring of relevant ecological information from unstructured textual sources. We focused on species distribution data from two sources: news outlets and research papers. We assessed the LLMs for four key tasks: classification of documents with species distribution data, identification of regions where species are recorded, generation of geographical coordinates for these regions, and supply of results in a structured format. GPT-4 consistently outperformed the other models, demonstrating a high capacity to interpret textual data and extract relevant information, with the percentage of correct outputs often exceeding 90% (average accuracy across tasks: 87–100%). Its performance also depended on the data source type and task, with better results achieved with news reports, in the identification of regions with species reports and presentation of structured output. Its predecessor, GPT-3.5, exhibited slightly lower accuracy across all tasks and data sources (average accuracy across tasks: 81–97%), whereas LLaMA-2-70B showed the worst performance (37–73%). These results demonstrate the potential benefit of integrating prompt-based LLMs into ecological data assimilation workflows as essential tools to efficiently process large volumes of textual data. • Large Language Models (LLMs) are tested for extracting biodiversity data from text. • Tasks include identification of relevant sources, geocoding, and data structuring. • LLMs, especially GPT-4, show over 90% accuracy on multiple tasks. • LLMs promise effective data assimilation from unstructured text in ecology. [ABSTRACT FROM AUTHOR]

Although Indonesian is the fourth most frequently used language on the internet, the development of NLP in Indonesian has not been studied intensively. One form of NLP application classified as an NLG task is the Automatic Question Generation task. Generally, the task has proven well, using rule-based and cloze tests, but these approaches depend heavily on the defined rules. While this approach is suitable for automated question generation systems on a small scale, it can become less efficient as the scale of the system grows. Many NLG model architectures have recently proven to have significantly improved performance compared to previous architectures, such as generative pre-trained transformers, text-to-text transfer transformers, bidirectional autoregressive transformers, and many more. Previous studies on AQG in Indonesian were built on RNN-based architecture such as GRU, LSTM, and Transformer. The performance of models in previous studies is compared with state-of-the-art models, such as multilingual models mBART and mT5, and monolingual models such as IndoBART and IndoGPT. As a result, the fine-tuned IndoBART performed significantly higher than either BiGRU and BiLSTM on the SQuAD dataset. Fine-tuned IndoBART on most of the metrics also performed better on the TyDiQA dataset only, which has fewer population than the SQuAD dataset. [ABSTRACT FROM AUTHOR]

• We examined the applicability of the Divergent Association Task (DAT) in children. • The DAT was suitable to measure divergent thinking in elementary school students. • The DAT was evaluated with different language processing models in Chinese context. • Word2Vec performed better than the BERT and GPT models. The Divergent Association Task (DAT), published in July 2021, is a psychological test designed to measure an individual's divergent thinking. The test requires participants to name ten nouns that exhibit maximum dissimilarity from each other. The semantic distance between these nouns is then calculated to indicate the person's level of divergent thinking. In this study, we explored the applicability of the DAT for elementary school students in Chinese contexts, given that it was not initially designed for this specific population and was available only in English. We recruited a total of 348 students who were asked to complete three creativity tasks: the DAT, the Alternative Uses Task (AUT), and the Bridge-the-Associative-Gap Task (BAG). We examined the associations between DAT and the scores of the AUT and BAG tests. Moreover, we tested the accuracy of the DAT using varying numbers of nouns and different natural language processing models to calculate the semantic distance between nouns. Our findings supported the suitability of using the DAT to measure divergent thinking in elementary school students within Chinese contexts. We also found that using only eight nouns, instead of ten, could achieve a relatively high accuracy in measuring divergent thinking based on the DAT method. The language model of Word2Vec performed better than the BERT (Bidirectional Encoder Representations from Transformers) and GPT (Generative Pre-trained Transformer) models when calculating semantic distances between nouns. This study has methodological and practical implications. [ABSTRACT FROM AUTHOR]

The transformation of personal comfort models into a scalable solution has been hindered by their heavy reliance on occupant feedback across a wide range of environmental conditions, and their inability to adapt to changes in preferences. However, these challenges could be addressed by leveraging large-scale databases to build pre-trained models and using limited occupant interactions to select the most representative model, while simultaneously detecting potential changes in preferences. This paper introduces a novel transformer-based architecture (ComfortGPT), which employs pre-trained models built on the data derived from ECOBEE Donate Your Data program through a uniquely crafted cleaning process. The pre-trained models are derived through clustering the supervised learning models applied to each user's data, thereby capturing the diverse range of thermostat preferences. Due to their generative nature, these models can predict setpoints across a wide range of environmental conditions. In order to adapt to variations in occupant preferences, we adopted a transformer mechanism to encode user thermostat interactions based on time sequence and relevance. We applied an exhaustive search to identify the optimal hyperparameters and achieved R2 of 0.77 and Mean Absolute Error (MAE) of 0.65 °C. We also found that the pre-trained models represent a spectrum of setpoint/outdoor temperature relationships ranging from warm-likers to cool-likers and referred to them as comfort archetypes. These archetypes could be employed in energy simulations and design applications, facilitating more personalized and energy-efficient indoor environments. To support further research and practical applications, we provide a code repository and an interactive graphical interface at https://github.com/Building-Robotics-Lab/ComfortGPT. • Proposed a transformer-based architecture for setpoint prediction via pre-trained models. • Clustered pre-trained models to capture diverse thermostat preferences. • Encoded user interactions to adapt to occupant preferences. • Found a diverse spectrum of setpoint/outdoor temperature relationships. • Achieved predictive performance of R2 = 0.77 and MAE = 0.65 °C. [ABSTRACT FROM AUTHOR]