Search

Language is an indispensable tool for human communication, and presently, the language that dominates the Internet is English. Language identification is the process of determining a predetermined language automatically from a given content (e.g., English, Malay, Danish, Estonian, Czech, Slovak, etc.). The ability to identify other languages in relation to English is highly desirable. It is the goal of this research to improve the method used to achieve this end. Three methods have been studied in this research are distance measurement, Boolean method, and the proposed method, namely, optimum profile. From the initial experiments, we have found that, distance measurement and Boolean method is not reliable in the European web page identification. Therefore, we propose optimum profile which is using N-grams frequency and N-grams position to do web page language identification. The result show that the proposed method gives the highest performance with accuracy 91.52%.

Current techniques of language identification are based on a method that assigns one or more textual documents into a set of predefined languages that are relevant to page contents. In this paper, we proposed an agent architecture that is used in criminal mobile devices identification systems. It is based on the usage of a software agent to process at least one document by using a dictionary that belong to a set of languages in order to determine the type of language features to be used in the text preprocessing module. Then, the agent will map at least one of the documents with the content of the dictionary in order to identify the languages used in the text by the language identification agent. Finally, the digital forensic agent will check the potential criminal short messages through the predefined keyword repository of corresponding language. Form our experiments, the agent architecture has been able to identify correctly the types of languages written in the short text messaging (SMS) system.