Background: Academic dishonesty (AD) and trustworthy assessment (TA) are fundamental issues in the context of an online assessment. However, little systematic work currently exists on how researchers have explored AD and TA issues in online assessment practice. Objectives: Hence, this research aimed at investigating the latest findings regarding AD forms, factors affecting AD and TA, and solutions to reduce AD and increase TA to maintain the quality of online assessment. Methods: We reviewed 52 articles in Scopus and Web of Science databases from January 2017 to April 2021 using the Preferred Reporting Items for Systematic Reviews and Meta‐Analyses model as a guideline to perform a systematic literature review that included three stages, namely planning, conducting, and reporting. Results and conclusions: Our review found that there were different forms of AD among students in online learning namely plagiarism, cheating, collusion, and using jockeys. Individual factors such as being lazy to learn, lack of ability, and poor awareness as well as situational factors including the influence of friends, the pressure of the courses, and ease of access to information were strongly associated with AD. A technology‐based approach such as using plagiarism‐checking software, multi‐artificial intelligence (AI) in a learning management system, computer adaptive tests, and online proctoring as well as pedagogical‐based approaches, such as implementing a research ethics course programme, and a re‐design assessment form such as oral‐based and dynamic assessment to reduce cheating behaviour and also sociocultural and sociotechnical adjustment related to the online assessment are reported to reduce AD and increase TA. Implications: Educators should adjust the design of online learning and assessment methods as soon as possible. The identified gaps point towards unexplored study on AI, machine learning, learning analytics tools, and related issues of AD and TA in K12 education could motivated future work in the field. Lay Description: What is already known about this topic: Online assessment has become a global trend in the last 10 years, especially since PISA (2015) and TIMSS (2019) were adopted, then, the increasing adoption of MOOCs and online learning due to the COVID‐19 pandemic.Academic dishonesty (AD) and trustworthy assessment (TA) are hot issues in online assessment practices that are interesting to study.Research findings on the associated factors of AD and efforts to increase TA in online assessments are still inconclusive.Contemporary research on AD and TA in online assessment lacks of comprehensive systematic review. What this paper adds: This paper systematically reviewed 5 years of empirical research (2017–2021) on AD and TA in online assessment in both K‐12 and higher education.This paper revealed the most recent trends developing in the literature regarding the factors contributing to the appearance of AD in online assessment.This paper revealed a recent report on efforts to improve TA in online assessment.This paper recommended several new pedagogical and technological approaches to reduce AD and increase TA in online assessment. Implications for practice: Education managers, professional associations, teachers, and lecturers need to reeducate the academic community about the risks of AD in online assessments.Teachers need to redesign the form of online assessment to achieve HOTs and reduce multiple‐choice questions. On the other hand, oral‐based, project‐based assessments, and student work portfolios are encouraged to be adopted.Online assessment using an adaptive and intelligent online assessment approach, randomization of delivery question systems, smart proctoring, AI for online assessment, and learning analytic technology is recommended to be studied massively.Research on AD and TA at the K‐12 and higher education levels needs to be intensified considering online learning programs in the era of the COVID‐19 pandemic. [ABSTRACT FROM AUTHOR]