Search

NetFlow is a network protocol system that is used to represent an overall summary of computer network conversations. A NetFlow record can convert previously captured packet captures or obtain NetFlow session data in real time. This research examines the use of machine-learning techniques to identify anomalies in NetFlow records and classify malware behavior for further investigation. The intent is to identify low-cost solutions leveraging open-source software capable of deployment on computer hardware of currently in-use data networks. This work seeks to determine whether expert selection of features can improve machine-learning detection algorithm performance and evaluate the trade-offs associated with eliminating redundant or excessive numbers of features. We identify the Random Forest algorithm as the strongest single algorithm across three of four metrics, with our chosen NetFlow features cutting the testing and training times in half while incurring minor reductions in two metrics. The experiment demonstrates that the chosen NetFlow features are sufficiently discriminative to detect attacks with a success rate higher than 94%., NCWDG, Lieutenant, United States Navy, Approved for public release. Distribution is unlimited.

DNS hijacking represents a security threat to users because it enables bypassing existing DNS security measures. Several malware families exploit this by changing the client DNS configuration to point to a malicious DNS resolver. Following the assumption that users will never actively choose to use a resolver that is not well-known, our paper introduces the idea of detecting client-based DNS hijacking by classifying public resolvers based on whether they are well-known or not. Furthermore, we propose to use NetFlow-based features to classify a resolver as well-known or malicious. By characterizing and manually labelling the 405 resolvers seen in four weeks of NetFlow data from a national ISP, we show that classification of both well-known and malicious servers can be made with an AUROC of 0.85.

In this research, we use different supervised and unsupervised machine learning techniques to detect anomalies in NetFlow data. We aim to create a system for home or small-business use where the user is in control. We use WEKA for the machine learning models and feature selection. The UGR’16 dataset is used to train and test the models. We create three different models for each method where the model is trained on one day and tested on another. We find that supervised models perform better than unsupervised ones. Random Forest has the highest F1-score (0.9165) of the supervised models. However, Random Forest is statistically similar to 6 out of 8 different classifiers according to the Friedman and Nemenyi tests. One challenge with supervised methods is that there is a need for a third party to make sure the models are updated for new attacks. However, it seems feasible to create a system for network monitoring for home usage. Finally, we argue that the approach to research in machine learning should, in some cases, take a different direction. Instead of chasing the highest accuracy, we should look at which factors allow a user to work with a system. A set of rules is much easier to interpret than complex models. Especially if these models are statistically similar we could look at factors other than a single metric. After these results, we look at presenting warning messages to end-users. We want to motivate users to take action after they read a message. For this, we first create a theoretical framework based on behaviour, motivation and uncertainties. We want to find out if we can use uncertainties to group users, which allows us to create a semi-personal approach. Based on this theoretical framework, we create different versions of a warning message that focus on different uncertainties. Through 22 interviews, we asked the participants to rank the different versions and asked them questions about their stance on network security and preferences for warning messag, degree of Master of Science in Computer Science & Science Communication, Computer Science

In this research, we use different supervised and unsupervised machine learning techniques to detect anomalies in NetFlow data. We aim to create a system for home or small-business use where the user is in control. We use WEKA for the machine learning models and feature selection. The UGR’16 dataset is used to train and test the models. We create three different models for each method where the model is trained on one day and tested on another. We find that supervised models perform better than unsupervised ones. Random Forest has the highest F1-score (0.9165) of the supervised models. However, Random Forest is statistically similar to 6 out of 8 different classifiers according to the Friedman and Nemenyi tests. One challenge with supervised methods is that there is a need for a third party to make sure the models are updated for new attacks. However, it seems feasible to create a system for network monitoring for home usage. Finally, we argue that the approach to research in machine learning should, in some cases, take a different direction. Instead of chasing the highest accuracy, we should look at which factors allow a user to work with a system. A set of rules is much easier to interpret than complex models. Especially if these models are statistically similar we could look at factors other than a single metric. After these results, we look at presenting warning messages to end-users. We want to motivate users to take action after they read a message. For this, we first create a theoretical framework based on behaviour, motivation and uncertainties. We want to find out if we can use uncertainties to group users, which allows us to create a semi-personal approach. Based on this theoretical framework, we create different versions of a warning message that focus on different uncertainties. Through 22 interviews, we asked the participants to rank the different versions and asked them questions about their stance on network security and preferences for warning messag, degree of Master of Science in Computer Science & Science Communication, Computer Science

This article presents a part of our project focused on developing a network collector and analyzer. It is a Python application used to collect and analyze NetFlow version 9 messages. With this article, an underlying schema and code of the application are described.

This article presents a part of our project focused on developing a network collector and analyzer. It is a Python application used to collect and analyze NetFlow version 9 messages. With this article, an underlying schema and code of the application are described.

This article presents a part of our project focused on developing a network collector and analyzer. It is a Python application used to collect and analyze NetFlow version 9 messages. With this article, an underlying schema and code of the application are described.

This article presents a part of our project focused on developing a network collector and analyzer. It is a Python application used to collect and analyze NetFlow version 9 messages. With this article, an underlying schema and code of the application are described.

This article presents a part of our project focused on developing a network collector and analyzer. It is a Python application used to collect and analyze NetFlow version 9 messages. With this article, an underlying schema and code of the application are described.

This article presents a part of our project focused on developing a network collector and analyzer. It is a Python application used to collect and analyze NetFlow version 9 messages. With this article, an underlying schema and code of the application are described.

The amount of people and devices connected through the Internet has been growing at a rapid pace; as of June 2019 58,8% of the world’s population and billions of devices are joined by this vast network of information resources and services. Not every Internet user however has benign intentions. Cybercriminals use this technology for their own personal gain, by creating malicious software with the objective to compromise and even control the devices of unsuspecting victims. After devices have been infected with malicious software, they will be controlled by a central networking infrastructure, or Command and Control (C&C). Numerous studies have developed detection methods to find the commanding servers behind these attacks – which is important for locally implemented anti-virus software – but the infrastructures behind these attacks is only uncovered after these malicious servers have been taken down. We have collaborated with one of the largest Tier 1 Internet Service Providers and have collected ~4Tb of global NetFlow traffic consisting of daily connections made worldwide, giving us the possibility to analyze malicious infrastructures from a new perspective. This dataset allowed us to evaluate one of the most promising data sources to uncover and analyze adversarial C&C infrastructures; open source cyber threat intelligence. After having introduced a taxonomy to evaluate this intelligence, and having assessed the defensive advantages users will gain when adopting these information feeds, we came to the conclusion that – even though advertised differently – this intelligence only provides a small fragment of the total picture. For this reason we have decomposed Internet networking infrastructures into three categories in which Internet devices can be divided, computer clients, Internet of Things (or IoT ) and routers, and use a machine learning driven methodology to paint the threat landscape of the tactics, techniques and procedures adversaries employ withi, Computer Science | Data Science and Technology, Computer Science | Cyber Security

The amount of people and devices connected through the Internet has been growing at a rapid pace; as of June 2019 58,8% of the world’s population and billions of devices are joined by this vast network of information resources and services. Not every Internet user however has benign intentions. Cybercriminals use this technology for their own personal gain, by creating malicious software with the objective to compromise and even control the devices of unsuspecting victims. After devices have been infected with malicious software, they will be controlled by a central networking infrastructure, or Command and Control (C&C). Numerous studies have developed detection methods to find the commanding servers behind these attacks – which is important for locally implemented anti-virus software – but the infrastructures behind these attacks is only uncovered after these malicious servers have been taken down. We have collaborated with one of the largest Tier 1 Internet Service Providers and have collected ~4Tb of global NetFlow traffic consisting of daily connections made worldwide, giving us the possibility to analyze malicious infrastructures from a new perspective. This dataset allowed us to evaluate one of the most promising data sources to uncover and analyze adversarial C&C infrastructures; open source cyber threat intelligence. After having introduced a taxonomy to evaluate this intelligence, and having assessed the defensive advantages users will gain when adopting these information feeds, we came to the conclusion that – even though advertised differently – this intelligence only provides a small fragment of the total picture. For this reason we have decomposed Internet networking infrastructures into three categories in which Internet devices can be divided, computer clients, Internet of Things (or IoT ) and routers, and use a machine learning driven methodology to paint the threat landscape of the tactics, techniques and procedures adversaries employ withi, Computer Science | Data Science and Technology, Computer Science | Cyber Security

This project displays a novel method to find cryptomining using Netflow mesurements. This is achieved by investigating the differences betwen mining traffic and the rest of the traffic, and through the use of machine learning. This helps find miners with high confidence and using little resources.

Today, the Internet allows virtually anytime, anywhere access to a seemingly unlimited supply of information and services. Statistics such as the six-fold increase of U.S. online retail sales since 2000 illustrate its growing importance to the global economy, and fuel our demand for rapid, round-the-clock Internet provision. This growth has created a need for systems of control and management to regulate an increasingly complex infrastructure. Unfortunately, the prospect of making fast money from this burgeoning industry has also started to attract criminals. This has driven an increase in, and professionalization of, cyber-crime. As a result, a variety of methods have been designed with the intention of better protecting the Internet, its users and its underlying infrastructure from both accidental and malicious threats. Firewalls, which restrict network access, intrusion detection systems, which locate and prevent unauthorized access, and network monitors, which over see the correct functioning of network infrastructures, have all been developed in order to detect and avert potential problems. These systems can be broadly defined as either reactive or proactive. The reactive approach seeks to identify specific problem patterns. It uses models learnt from theory or practice to locate common dangers as they develop. The number of patterns applied grows as each new problem is encountered. Proactive methods work differently. They start defining an idealized model of the normal behavior of a given system. Any significant deviation from this model is assumed to be an aberrance caused by an external danger. However, this assumption may turn out to be incorrect, having actually not arisen from a disruption or a malicious act. Despite considerable improvements, the development of accurate proactive detection and classification methods is still an area of intense research. This is particularly true of methods fit for high speed networks. To cope with the huge amounts of dat

Today, the Internet allows virtually anytime, anywhere access to a seemingly unlimited supply of information and services. Statistics such as the six-fold increase of U.S. online retail sales since 2000 illustrate its growing importance to the global economy, and fuel our demand for rapid, round-the-clock Internet provision. This growth has created a need for systems of control and management to regulate an increasingly complex infrastructure. Unfortunately, the prospect of making fast money from this burgeoning industry has also started to attract criminals. This has driven an increase in, and professionalization of, cyber-crime. As a result, a variety of methods have been designed with the intention of better protecting the Internet, its users and its underlying infrastructure from both accidental and malicious threats. Firewalls, which restrict network access, intrusion detection systems, which locate and prevent unauthorized access, and network monitors, which over see the correct functioning of network infrastructures, have all been developed in order to detect and avert potential problems. These systems can be broadly defined as either reactive or proactive. The reactive approach seeks to identify specific problem patterns. It uses models learnt from theory or practice to locate common dangers as they develop. The number of patterns applied grows as each new problem is encountered. Proactive methods work differently. They start defining an idealized model of the normal behavior of a given system. Any significant deviation from this model is assumed to be an aberrance caused by an external danger. However, this assumption may turn out to be incorrect, having actually not arisen from a disruption or a malicious act. Despite considerable improvements, the development of accurate proactive detection and classification methods is still an area of intense research. This is particularly true of methods fit for high speed networks. To cope with the huge amounts of dat

[ES] La evaluación de algoritmos y técnicas para implementar sistemas de detección de intrusiones depende en gran medida de la existencia de conjuntos de datos (dataset) bien diseñados. En los últimos años, se ha realizado un gran esfuerzo para construir estos datasets. En este artículo se presenta un nuevo dataset que se construye con tráfico real y ataques actualizados. La principal ventaja de este conjunto de datos sobre los anteriores es su utilidad para la evaluación de IDSs que consideran la evolución a largo plazo y la periodicidad del tráfico. También permite entrenar y evaluar modelos que consideren las diferencias entre día/noche o entre días laborables/fines de semana.

[ES] La evaluación de algoritmos y técnicas para implementar sistemas de detección de intrusiones depende en gran medida de la existencia de conjuntos de datos (dataset) bien diseñados. En los últimos años, se ha realizado un gran esfuerzo para construir estos datasets. En este artículo se presenta un nuevo dataset que se construye con tráfico real y ataques actualizados. La principal ventaja de este conjunto de datos sobre los anteriores es su utilidad para la evaluación de IDSs que consideran la evolución a largo plazo y la periodicidad del tráfico. También permite entrenar y evaluar modelos que consideren las diferencias entre día/noche o entre días laborables/fines de semana.

[ES] La evaluación de algoritmos y técnicas para implementar sistemas de detección de intrusiones depende en gran medida de la existencia de conjuntos de datos (dataset) bien diseñados. En los últimos años, se ha realizado un gran esfuerzo para construir estos datasets. En este artículo se presenta un nuevo dataset que se construye con tráfico real y ataques actualizados. La principal ventaja de este conjunto de datos sobre los anteriores es su utilidad para la evaluación de IDSs que consideran la evolución a largo plazo y la periodicidad del tráfico. También permite entrenar y evaluar modelos que consideren las diferencias entre día/noche o entre días laborables/fines de semana.

[ES] La evaluación de algoritmos y técnicas para implementar sistemas de detección de intrusiones depende en gran medida de la existencia de conjuntos de datos (dataset) bien diseñados. En los últimos años, se ha realizado un gran esfuerzo para construir estos datasets. En este artículo se presenta un nuevo dataset que se construye con tráfico real y ataques actualizados. La principal ventaja de este conjunto de datos sobre los anteriores es su utilidad para la evaluación de IDSs que consideran la evolución a largo plazo y la periodicidad del tráfico. También permite entrenar y evaluar modelos que consideren las diferencias entre día/noche o entre días laborables/fines de semana.

[ES] La evaluación de algoritmos y técnicas para implementar sistemas de detección de intrusiones depende en gran medida de la existencia de conjuntos de datos (dataset) bien diseñados. En los últimos años, se ha realizado un gran esfuerzo para construir estos datasets. En este artículo se presenta un nuevo dataset que se construye con tráfico real y ataques actualizados. La principal ventaja de este conjunto de datos sobre los anteriores es su utilidad para la evaluación de IDSs que consideran la evolución a largo plazo y la periodicidad del tráfico. También permite entrenar y evaluar modelos que consideren las diferencias entre día/noche o entre días laborables/fines de semana.

This paper addresses the issue of detecting unwanted traffic in data networks, namely the detection of botnet networks. In this paper, we focused on a time behavioral analysis, more specifically said – lifespans of a simulated botnet network traffic, collected and discovered from NetFlow messages, and also of real botnet communication of a malware. As a method we chose survival analysis and for rigorous testing of differences Mantel–Cox test. Lifespans of those referred traffics are discovered and calculated by lifelines using Python language. Based on our research we have figured out a possibility to distinguish the individual lifespans of C&C communications that are identical to each other by using survival projection curves, although it occurred in a different time course., Tento dokument řeší problém detekce nežádoucích dat v datových sítích, zejména zjištění botnet sítě. Tento článek se zaměřuje na časovou behaviorální analýzy, konkrétně řečeno – životnost simulovaného botnet síťového přenosu, shromážděn pomocí NetFlow zpráv. Dále je také zaměřen na komunikaci skutečné botnet sítě. Jako metodu jsme zvolili analýzu přežití a testování rozdílů pomocí Mantel-Cox testu. Životnost těchto uvedených síťových provozů je zkoumána pomocí balíčku lifelines jazyka Python. Na základě našeho výzkumu jsme zjistili možnost rozlišovat jednotlivé životnosti C&C komunikace, které jsou navzájem shodné, a to pomocí projekce křivky přežití, i když k ní došlo v jiném časovém průběhu.

This paper addresses the issue of detecting unwanted traffic in data networks, namely the detection of botnet networks. In this paper, we focused on a time behavioral analysis, more specifically said – lifespans of a simulated botnet network traffic, collected and discovered from NetFlow messages, and also of real botnet communication of a malware. As a method we chose survival analysis and for rigorous testing of differences Mantel–Cox test. Lifespans of those referred traffics are discovered and calculated by lifelines using Python language. Based on our research we have figured out a possibility to distinguish the individual lifespans of C&C communications that are identical to each other by using survival projection curves, although it occurred in a different time course., Tento dokument řeší problém detekce nežádoucích dat v datových sítích, zejména zjištění botnet sítě. Tento článek se zaměřuje na časovou behaviorální analýzy, konkrétně řečeno – životnost simulovaného botnet síťového přenosu, shromážděn pomocí NetFlow zpráv. Dále je také zaměřen na komunikaci skutečné botnet sítě. Jako metodu jsme zvolili analýzu přežití a testování rozdílů pomocí Mantel-Cox testu. Životnost těchto uvedených síťových provozů je zkoumána pomocí balíčku lifelines jazyka Python. Na základě našeho výzkumu jsme zjistili možnost rozlišovat jednotlivé životnosti C&C komunikace, které jsou navzájem shodné, a to pomocí projekce křivky přežití, i když k ní došlo v jiném časovém průběhu.

Botnets are one of the most dangerous and serious cybersecurity threats since they are a major vector of large-scale attack campaigns such as phishing, distributed denial-of-service (DDoS) attacks, trojans, spams, etc. A large body of research has been accomplished on botnet detection, but recent security incidents show that there are still several challenges remaining to be addressed, such as the ability to develop detectors which can cope with new types of botnets. In this paper, we propose BotGM, a new approach to detect botnet activities based on behavioral analysis of network traffic flow. BotGM identifies network traffic behavior using graph-based mining techniques to detect botnets behaviors and model the dependencies among flows to traceback the root causes then. We applied BotGM on a publicly available large dataset of Botnet network flows, where it detects various botnet behaviors with a high accuracy without any prior knowledge of them.

Being able to model behavior described by a linear sequence of observations (such as log files) goes a long way towards better understanding the underlying processes. This improved understanding can be very helpful in a number of activities, ranging from software (reverse) engineering to network traffic analysis. The developments in this thesis were driven by specific goals in predicting (human) behaviors captured by a software appliance observing network traffic and user requests to specific resources. Its final contributions have exceeded the original goals of the project in two important ways: I present (1) a flexible learning algorithm for finite automata accompanied by theoretical underpinning and its implementation, a contribution towards better learning algorithms, and (2) applications of the algorithm to use-cases in computer networking and beyond. The central algorithm considered in the thesis is a blue-fringe state-merging automaton learning algorithm, conducting a greedy search over feasible solutions. Its key components are a heuristic to search for consistent merges and an evaluation metric to assess the quality of a merge by assigning scores to merges. I generalize this framework by making the heuristic components explicitly parametric. While state-merging algorithms were originally defined for probabilistic and non-probabilistic finite state machines and later used to derive algorithms for more extended models such as real-time automata, the work presented here extends the scope of the algorithms to a wide range of ad-hoc defined models as well as enables the user to implement modifications to the heuristic search process. These modifications help to account for domain knowledge and richer semantics of models with a regular language core. I provide an implementation and a Python interface of the flexible state-merging framework, including stream/online and interactive variants of the algorithm based on a C++ implementation of the blue-fringe greedy se

Botnets are one of the most dangerous and serious cybersecurity threats since they are a major vector of large-scale attack campaigns such as phishing, distributed denial-of-service (DDoS) attacks, trojans, spams, etc. A large body of research has been accomplished on botnet detection, but recent security incidents show that there are still several challenges remaining to be addressed, such as the ability to develop detectors which can cope with new types of botnets. In this paper, we propose BotGM, a new approach to detect botnet activities based on behavioral analysis of network traffic flow. BotGM identifies network traffic behavior using graph-based mining techniques to detect botnets behaviors and model the dependencies among flows to traceback the root causes then. We applied BotGM on a publicly available large dataset of Botnet network flows, where it detects various botnet behaviors with a high accuracy without any prior knowledge of them.

Being able to model behavior described by a linear sequence of observations (such as log files) goes a long way towards better understanding the underlying processes. This improved understanding can be very helpful in a number of activities, ranging from software (reverse) engineering to network traffic analysis. The developments in this thesis were driven by specific goals in predicting (human) behaviors captured by a software appliance observing network traffic and user requests to specific resources. Its final contributions have exceeded the original goals of the project in two important ways: I present (1) a flexible learning algorithm for finite automata accompanied by theoretical underpinning and its implementation, a contribution towards better learning algorithms, and (2) applications of the algorithm to use-cases in computer networking and beyond. The central algorithm considered in the thesis is a blue-fringe state-merging automaton learning algorithm, conducting a greedy search over feasible solutions. Its key components are a heuristic to search for consistent merges and an evaluation metric to assess the quality of a merge by assigning scores to merges. I generalize this framework by making the heuristic components explicitly parametric. While state-merging algorithms were originally defined for probabilistic and non-probabilistic finite state machines and later used to derive algorithms for more extended models such as real-time automata, the work presented here extends the scope of the algorithms to a wide range of ad-hoc defined models as well as enables the user to implement modifications to the heuristic search process. These modifications help to account for domain knowledge and richer semantics of models with a regular language core. I provide an implementation and a Python interface of the flexible state-merging framework, including stream/online and interactive variants of the algorithm based on a C++ implementation of the blue-fringe greedy se

This paper addresses the issue of detecting unwanted traffic in data networks, namely the detection of botnet networks. In this paper, we focused on a time behavioral analysis, more specifically said – lifespans of a simulated botnet network traffic, collected and discovered from NetFlow messages, and also of real botnet communication of a malware. As a method we chose survival analysis and for rigorous testing of differences Mantel–Cox test. Lifespans of those referred traffics are discovered and calculated by lifelines using Python language. Based on our research we have figured out a possibility to distinguish the individual lifespans of C&C communications that are identical to each other by using survival projection curves, although it occurred in a different time course., Tento dokument řeší problém detekce nežádoucích dat v datových sítích, zejména zjištění botnet sítě. Tento článek se zaměřuje na časovou behaviorální analýzy, konkrétně řečeno – životnost simulovaného botnet síťového přenosu, shromážděn pomocí NetFlow zpráv. Dále je také zaměřen na komunikaci skutečné botnet sítě. Jako metodu jsme zvolili analýzu přežití a testování rozdílů pomocí Mantel-Cox testu. Životnost těchto uvedených síťových provozů je zkoumána pomocí balíčku lifelines jazyka Python. Na základě našeho výzkumu jsme zjistili možnost rozlišovat jednotlivé životnosti C&C komunikace, které jsou navzájem shodné, a to pomocí projekce křivky přežití, i když k ní došlo v jiném časovém průběhu.

This paper addresses the issue of detecting unwanted traffic in data networks, namely the detection of botnet networks. In this paper, we focused on a time behavioral analysis, more specifically said – lifespans of a simulated botnet network traffic, collected and discovered from NetFlow messages, and also of real botnet communication of a malware. As a method we chose survival analysis and for rigorous testing of differences Mantel–Cox test. Lifespans of those referred traffics are discovered and calculated by lifelines using Python language. Based on our research we have figured out a possibility to distinguish the individual lifespans of C&C communications that are identical to each other by using survival projection curves, although it occurred in a different time course., Tento dokument řeší problém detekce nežádoucích dat v datových sítích, zejména zjištění botnet sítě. Tento článek se zaměřuje na časovou behaviorální analýzy, konkrétně řečeno – životnost simulovaného botnet síťového přenosu, shromážděn pomocí NetFlow zpráv. Dále je také zaměřen na komunikaci skutečné botnet sítě. Jako metodu jsme zvolili analýzu přežití a testování rozdílů pomocí Mantel-Cox testu. Životnost těchto uvedených síťových provozů je zkoumána pomocí balíčku lifelines jazyka Python. Na základě našeho výzkumu jsme zjistili možnost rozlišovat jednotlivé životnosti C&C komunikace, které jsou navzájem shodné, a to pomocí projekce křivky přežití, i když k ní došlo v jiném časovém průběhu.

Apache Spark’s capabilites offer new possibilities to make software systems more scalable and reliable. The framework can be used to improve old network visibility platforms. Previously, these systems used to be run in a single node, and used Deep Packet Inspection (DPI) techniques to classify the network flows. Deep Packet Inspection methods have a high computational cost so this limited the systems to a lower performance. Classifiers were forced to sample the input data in order to be able to process it in realtime, which caused important loss of information. This project makes use of Spark’s innovative features to create a distributed and fault tolerant platform that can analyse much more flows per second using Machine Learning to achieve a high precision and accuracy at a low computational cost.

This paper addresses the issue of detecting unwanted traffic in data networks, namely the detection of botnet networks. In this paper, we focused on a time behavioral analysis, more specifically said – lifespans of a simulated botnet network traffic, collected and discovered from NetFlow messages, and also of real botnet communication of a malware. As a method we chose survival analysis and for rigorous testing of differences Mantel–Cox test. Lifespans of those referred traffics are discovered and calculated by lifelines using Python language. Based on our research we have figured out a possibility to distinguish the individual lifespans of C&C communications that are identical to each other by using survival projection curves, although it occurred in a different time course., Tento dokument řeší problém detekce nežádoucích dat v datových sítích, zejména zjištění botnet sítě. Tento článek se zaměřuje na časovou behaviorální analýzy, konkrétně řečeno – životnost simulovaného botnet síťového přenosu, shromážděn pomocí NetFlow zpráv. Dále je také zaměřen na komunikaci skutečné botnet sítě. Jako metodu jsme zvolili analýzu přežití a testování rozdílů pomocí Mantel-Cox testu. Životnost těchto uvedených síťových provozů je zkoumána pomocí balíčku lifelines jazyka Python. Na základě našeho výzkumu jsme zjistili možnost rozlišovat jednotlivé životnosti C&C komunikace, které jsou navzájem shodné, a to pomocí projekce křivky přežití, i když k ní došlo v jiném časovém průběhu.

Tato práce popisuje propojení virtuální sítě s fyzickým zařízením a nastavení protokolu NetFlow. V praktické části je, navržena síť v simulačním nástroji GNS3, propojení s fyzickým zařízením od firmy MikroTik a monitorování datové sítě pomocí protokolu NetFlow., This work describes link between virtual network and physical device and setting protocol NetFlow. In the practical part is designed network in simulation tool GNS3, link with physical device from company MikroTik and track with protocol NetFlow., Fakulta elektrotechniky a informatiky, Cílem bakalářské práce bylo zachytávat a kontrolovat datový tok pomocí protokolu Netflow na navržené síťové infrastruktuře. Autor práce si připravil testovací síť, tvořenou grafickým simulátorem GNS3 a fyzickými prvky. Praktická část popisuje jednotlivé kroky vedoucí k vytvoření hybridní sítě. Zadané cíle byly dle vedoucí práce splněny. Student pracoval aktivně a samostatně. Student zodpověděl ze strany komise SZZ dotazy ke své práci. Komise v práci našla drobné nedostatky ve formální úpravě. Při hodnocení se komise přiklonila k vedoucí práce a bakalářskou práci ohodnotila známkou velmi dobře.

Traffic classification has always been a fundamental aspect regarding the identification of applications on the network that has allowed to apply different actions or services depending on their type, such as best-effort delivery or discarding the traffic in case of a malicious application. The cons

Настоящая статья посвящена исследованию нескольких различных методов сбора статистических данных, применительно к IP-сетям начиная с самых примитивных и заканчивая наиболее фукциональными – коммандная строка, SNMP, прокси-сервер, NetFlow, DPI. Все эти методы так или иначе применяются во всех современных сетях передачи данных. Были рассмотрены различия между методами и применимость каждого из них на практике., This article is devoted to the study of several different statistical techniques, applied to IP-based networks – from the most primitive and ending with the most functionality - the command line, SNMP, proxy server, NetFlow, DPI. All these methods are used anyway in all modern data networks. Have been considered the differences between the methods and applicability of each of them practically.

The task of network traffic monitoring has evolved drastically with the ever-increasing amount of data flowing in large scale networks. The automated analysis of this tremendous source of information often comes with using simpler models on aggregated data (e.g. IP flow records) due to time and space constraints. A step towards utilizing IP flow records more effectively are stream learning techniques. We propose a method to collect a limited yet relevant amount of data in order to learn a class of complex models, finite state machines, in real-time. These machines are used as communication profiles to fingerprint, identify or classify hosts and services and offer high detection rates while requiring less training data and thus being faster to compute than simple models., Accepted author manuscript, Cyber Security

FM köpte satellitkapacitet för närmare 20 miljoner kronor under 2014 för utbildning, övning och internationella operationer. Enligt HKV räcker dock inte upphandlad kapacitet för att täcka behoven ute hos förbanden. Syftet med denna studie är att ta reda på mer hur kapacitetsanvändningen ser ut vid en operativ stab utifrån deras arbete och därigenom se om det finns någon militär nytta att vinna kopplad till kapacitetsanvändning. Utgångspunkten för arbetet är insamlad information om datornätverkstrafiken i de olika nätverken. Detta kvantitativa data jämförs med stabens arbete utifrån kvantitativ och kvalitativ data i krigsdagbok, styrdokument och deltagande studie. Studien visar på att kapacitetsanvändningen ej är relaterad till stabens arbete utan snarare är kopplad till när personal är på plats och personalens internetanvändning. Studien pekar på att det kan finnas en potential att få mer militär nytta, dock måste mekanismer för prioritering i nätverket införas och vidare studier avseende trafiken genomföras., In 2014 the Swedish Armed Forces (SwAF) spent almost SEK 20 million on the procurement of satellite capacity for use during training, exercises and operations. However, according to SwAF Headquarters, the capacity procured did not meet unit demands. The aim of this study is to gain a better understanding of the capacity usage in an operational headquarters, based on the headquarters’ staff procedures and to see if there is any military utility to be gained in connection with this capacity. The starting point for the study is quantitative data about network usage. This quantitative data is then compared with staff work based on quantitative and qualitative data from war diaries, governing documentation and studies. The study shows that capacity usage is not related to staff work, but is instead linked to the presence of staff and their use of the Internet. The study also indicates that there could be potential for greater military utility of network capacity, but this will require the introduction of network priority mechanisms and further studies into user traffic.

The management of telecommunication networks is one of the critical activities made by network providers and their operators. New visualization tools allow the design, planning and administration and study of the time evolution of network behaviour in a user-friendly, intuitive and simple way. The aim of this project is to contribute to the design of the visualization tools aforementioned. This document includes, first of all, a study of the different existing graphical solutions. It classifies and compares its visualization models in order to present the main advantages and disadvantages of each one of them. Secondly, the document contains a study of the current technologies that could represent an alternative that comprises the positive aspects of each model. Finally, the document proposes a solution that presents the models studied and describes the different problems faced during its development. For the development of this project, the collection of valid data for their future visualization is necessary. For this reason, the solution proposes a data structure necessary for its development, as well as a storage system and a distribution engine. The development of a tool that copes with the creation and introduction of these periodical data was essential in order to offer the final-user an interactive feeling. Therefore, the final aim is the design and development of an application that fulfils the previously studied requisites and allows the management, edition and visualization of data regarding the previously gathered models through free, open-source and modern tools. Finally, this document includes a compendium of the difficulties found during the development of the application as well as the run-through scenarios tested and improvements implemented. The conclusion includes the guidelines for revision, expansion and future improvement of the application.

FM köpte satellitkapacitet för närmare 20 miljoner kronor under 2014 för utbildning, övning och internationella operationer. Enligt HKV räcker dock inte upphandlad kapacitet för att täcka behoven ute hos förbanden. Syftet med denna studie är att ta reda på mer hur kapacitetsanvändningen ser ut vid en operativ stab utifrån deras arbete och därigenom se om det finns någon militär nytta att vinna kopplad till kapacitetsanvändning. Utgångspunkten för arbetet är insamlad information om datornätverkstrafiken i de olika nätverken. Detta kvantitativa data jämförs med stabens arbete utifrån kvantitativ och kvalitativ data i krigsdagbok, styrdokument och deltagande studie. Studien visar på att kapacitetsanvändningen ej är relaterad till stabens arbete utan snarare är kopplad till när personal är på plats och personalens internetanvändning. Studien pekar på att det kan finnas en potential att få mer militär nytta, dock måste mekanismer för prioritering i nätverket införas och vidare studier avseende trafiken genomföras., In 2014 the Swedish Armed Forces (SwAF) spent almost SEK 20 million on the procurement of satellite capacity for use during training, exercises and operations. However, according to SwAF Headquarters, the capacity procured did not meet unit demands. The aim of this study is to gain a better understanding of the capacity usage in an operational headquarters, based on the headquarters’ staff procedures and to see if there is any military utility to be gained in connection with this capacity. The starting point for the study is quantitative data about network usage. This quantitative data is then compared with staff work based on quantitative and qualitative data from war diaries, governing documentation and studies. The study shows that capacity usage is not related to staff work, but is instead linked to the presence of staff and their use of the Internet. The study also indicates that there could be potential for greater military utility of network capacity, but this will require the introduction of network priority mechanisms and further studies into user traffic.

Treball on s'explica el state of the art de la estimació del delay per flow, es fa un análisis de la eficàcia del estimador MultiFlow en diferents escenaris i es proposa una metodología per millorar la eficàcia d'aquest estimador mitjançant la injecció de paquets SYN.

[ABSTRACT] In today's world, in which computing is tightly linked to IP networks, managing operational resources such as routers, switches, servers, and other network-related devices is becoming increasingly difficult. There are plenty of parameters to monitor, such as the use of bandwidth, the status of the links, the detection of bottlenecks, CPU/memory/disk usage, manage routing between devices, etc. The objective of this degree thesis is the study and deployment of the most popular monitoring tools, such as: Cacti, Nagios, Zabbix, The Dude and pmacct. This will allow us to evaluate and experiment with the different technologies for network monitoring, such as the information provided by routing protocols (BGP), data collection protocols (SNMP, NetFlow) or systems for storage of information (SQL). We will analyze the advantages and disadvantages from the results, the quality they offer, their requirements to see if they can be implemented in certain scenarios or not and their robustness, and how they can be incorporated on teaching activities (laboratory). For this study we set up a network scenario with 6 PCs connected together to simulate a small scale network. The study is complemented with the requirements and installation guide, a configuration tutorial for each of the studied tools and a presentation of the results obtained in the proposed scenario. Furthermore, the study will be complemented with the development and use of a MikroTik router in a conventional network monitored via SNMP and NetFlow in order to study the behavior of the device with monitoring tools (pmacct - NetFlow and The Dude - SNMP). It has been determined that the studied tools qualify, in terms of robustness to be incorporated on teaching duties in order to instruct students in the current monitoring and network management technologies., [RESUMEN] En el mundo actual, en el que la informática gira en torno al concepto de red, mantener en funcionamiento recursos tales como encaminadores (routers), conmutadores (switches), servidores, así como otros dispositivos que conforman la red, es cada vez más difícil. Hay gran cantidad de parámetros que es necesario monitorizar, como por ejemplo la utilización del ancho de banda, el estado de funcionamiento de los enlaces, la detección de cuellos de botella, uso de CPU/memoria/disco, tablas de encaminamiento, etc. El objetivo de este TFC es el estudio y despliegue de las herramientas de monitorización más populares como Cacti, Nagios, Zabbix, The Dude y pmacct. Esto nos va a permitir evaluar y experimentar con las diferentes tecnologías que ofrecen para el estudio de monitorización de redes, como la información proporcionada por protocolos de encaminamiento (BGP), protocolos de recolección de datos (SNMP, NetFlow) o sistemas de almacenamiento de la información (SQL). En este trabajo se analizan sus ventajas e inconvenientes a partir de los resultados, la calidad que ofrecen, sus requisitos para comprobar si pueden ser implementadas en determinados escenarios o no y su robustez de cara a incorporarlas en una práctica docente de Ingeniería Telemática. Para realizar el estudio se presenta un esquema de red formado por 6 PC conectados entre sí a fin de simular una red. El estudio viene complementado con los requisitos y guía de instalación, un manual de configuración para cada una de las herramientas estudiadas y una presentación de los resultados obtenidos en el escenario propuesto. Además, se complementará dicho estudio con la preparación y utilización de un router MikroTik en redes convencionales monitorizadas vía SNMP y NetFlow con tal de estudiar el comportamiento de dicho dispositivo y las herramientas de monitorización pmacct (mediante NetFlow) y The Dude (mediante SNMP). Se ha determinado que las herramientas estudiadas reúnen los requisitos de robustez como

Statistical summaries of IP traffic are at the heart of network operation and are used to recover aggregate information on subpopulations of flows. It is therefore of great importance to collect the most accurate and informative summaries given the router's resource constraints. A summarization algorithm, such as Cisco's sampled NetFlow, is applied to IP packet streams that consist of multiple interleaving IP flows. We develop sampling algorithms and unbiased estimators which address sources of inefficiency in current methods. First, we design tunable algorithms whereas currently a single parameter (the sampling rate) controls utilization of both memory and processing/access speed (which means that it has to be set according to the bottleneck resource). Second, we make a better use of the memory hierarchy, which involves exporting partial summaries to slower storage during the measurement period.

Statistical summaries of IP traffic are at the heart of network operation and are used to recover aggregate information on subpopulations of flows. It is therefore of great importance to collect the most accurate and informative summaries given the router's resource constraints. A summarization algorithm, such as Cisco's sampled NetFlow, is applied to IP packet streams that consist of multiple interleaving IP flows. We develop sampling algorithms and unbiased estimators which address sources of inefficiency in current methods. First, we design tunable algorithms whereas currently a single parameter (the sampling rate) controls utilization of both memory and processing/access speed (which means that it has to be set according to the bottleneck resource). Second, we make a better use of the memory hierarchy, which involves exporting partial summaries to slower storage during the measurement period.

Brute-force attacks are a prevalent phenomenon that is getting harderto successfully detect on a network level due to increasing volume and en-cryption of network traffic and growing ubiquity of high-speed networks.Although the research in this field advanced considerably, there still remainclasses of attacks that are undetectable. In this chapter, we present sev-eral methods for the detection of brute-force attacks based on the analysisof network flows. We discuss their strengths and shortcomings as well asshortcomings of flow-based methods in general. We also demonstrate thefragility of some methods by introducing detection evasion techniques.

Una de las principales dificultades con las que se encuentra todo operador de red a la hora de aplicar técnicas de ingeniería de tráfico radica en la obtención de la matriz de tráfico. Por definición, una matriz de tráfico describe el volumen de tráfico intercambiado entre los puntos de ingreso y egreso de una red durante un intervalo de tiempo dado. La forma directa de obtener la matriz de tráfico se basa en la realización de medidas a nivel de flujo. Por desgracia, no todas las redes cuentan con disp ositivos que posean tal capacidad. En la primera parte de este PFC se presenta un método simple y eficiente para la estimación de la matriz de tráfico de una red IP . Para ello se emplea la información de encaminamiento de la red y la carga de sus enlaces como entradas de un sistema de ecuaciones lineales . Dicho método , denominado tomogravedad, se aplica en las redes GÉANT y GRNET. En el caso de GÉANT, la información requerida se adqui ere a partir de dos aplicaciones existentes en su portal de monitorización: Cacti y Looking Glass. En GRNET se emplea la herramienta de monitorización Nemo , desarrollada por profesores de la EETAC. Utilizando estos datos a la entrada de un sistema compuesto por diferentes módulos desarrollados en R , Java y Matlab , se obtienen 2016 matrices para la red GÉANT y 268 para GRNET . Las matrices de tráfico han sido sometidas a un proceso de validación empleando nuevamente la carga de los enlaces. Los resultados obtenidos han sido satisfactorios, habiéndose conseguido , en el peor de los casos,unos valores de error cuadrático medio normalizado del 3,5% en GÉANT y 0,09% en GRNET. Logrado el objetivo anterior , las series temporales centran la atención de la segunda parte del PFC. En ella se plantea una metodología para el análisis y pre dicción de los volúmenes de tráfico estimados que se intercambian los puntos de ingreso y egreso de la red GÉANT durante el intervalo de tiempo en cuestión. Nuevamente, los resultados obtenidos han sido sati

Una de las principales dificultades con las que se encuentra todo operador de red a la hora de aplicar técnicas de ingeniería de tráfico radica en la obtención de la matriz de tráfico. Por definición, una matriz de tráfico describe el volumen de tráfico intercambiado entre los puntos de ingreso y egreso de una red durante un intervalo de tiempo dado. La forma directa de obtener la matriz de tráfico se basa en la realización de medidas a nivel de flujo. Por desgracia, no todas las redes cuentan con disp ositivos que posean tal capacidad. En la primera parte de este PFC se presenta un método simple y eficiente para la estimación de la matriz de tráfico de una red IP . Para ello se emplea la información de encaminamiento de la red y la carga de sus enlaces como entradas de un sistema de ecuaciones lineales . Dicho método , denominado tomogravedad, se aplica en las redes GÉANT y GRNET. En el caso de GÉANT, la información requerida se adqui ere a partir de dos aplicaciones existentes en su portal de monitorización: Cacti y Looking Glass. En GRNET se emplea la herramienta de monitorización Nemo , desarrollada por profesores de la EETAC. Utilizando estos datos a la entrada de un sistema compuesto por diferentes módulos desarrollados en R , Java y Matlab , se obtienen 2016 matrices para la red GÉANT y 268 para GRNET . Las matrices de tráfico han sido sometidas a un proceso de validación empleando nuevamente la carga de los enlaces. Los resultados obtenidos han sido satisfactorios, habiéndose conseguido , en el peor de los casos,unos valores de error cuadrático medio normalizado del 3,5% en GÉANT y 0,09% en GRNET. Logrado el objetivo anterior , las series temporales centran la atención de la segunda parte del PFC. En ella se plantea una metodología para el análisis y pre dicción de los volúmenes de tráfico estimados que se intercambian los puntos de ingreso y egreso de la red GÉANT durante el intervalo de tiempo en cuestión. Nuevamente, los resultados obtenidos han sido sati

La monitorización y predicción del tráfico en redes de ordenadores es de vital importancia a la hora de evaluar el rendimiento y el servicio que ofrecen. Permite analizar el uso que se le está dando al sistema, obteniendo información del origen, destino y tipo de tráfico transmitido, así como si son suficientes los recursos de red disponibles. Con esto se consigue mejorar el funcionamiento de la red, detectar posibles anomalías y sus causas e implementar soluciones a la hora de optimizar los recursos disponibles. El objetivo de este TFC es el estudio y despliegue de la herramienta de distribución libre pmacct. Ésta nos va a permitir evaluar y experimentar con las diferentes tecnologías que ofrecen para el estudio de monitorización de redes, como protocolos de encaminamiento (BGP), protocolos de recolección de datos (NetFlow) o sistemas de almacenamiento de la información (SQL). Se van a analizar las diferentes soluciones presentadas, determinando sus posibles limitaciones y deficiencias con el fin de comprobar si es una herramienta suficientemente robusta de cara a incorporarla a tareas docentes. Para realizar el estudio con el máximo detalle, se presenta un esquema de red creado mediante un software de virtualización dónde se representará un entorno de red lo más realista posible en términos de, número de dispositivos, distribución de los mismos y protocolos desplegados tanto de encaminamiento como de monitorización. Se asentarán las bases para futuros proyectos sobre esta herramienta, presentando diferentes ejemplos de configuración definiendo sus ventajas e inconvenientes, presentación de resultados obtenidos y descripción de los mismos. Se ha conseguido determinar, que el software pmacct es una buena solución de cara a implementarlo en tareas docentes con el fin de estudiar técnicas relacionadas con la monitorización del tráfico., English: The monitoring and prediction of the traffic over computer networks is of vital importance in order to evaluate the performance and the service offered. This allows analyzing the usage of the system obtaining information about the origin, destination and type of transmitted traffic. With this, a better network performance can be achieved, by detecting possible faults and it causes and implementing solutions to optimize the available resources. The goal of this degree thesis is the study and deployment of the free distribution tool pmacct. This tool allows us to evaluate and experiment with different technologies related to network monitoring, as well as routing protocols (BGP), data collection protocols (NetFlow) or data storing systems (SQL). This document describes and analyzes different solutions, defining its pointing out their limitations and deficiencies in order to test if it is a sufficientrobust tool with the purpose of using it in teaching tasks. In order to perform the study with the maximum detail lever, we present a network testbed created with virtualization software. The scenario is as realistic as possible on terms of number of devices, their distribution and the protocols used as much for routing and for monitoring. This thesis can be the basis of new studies about the aforementioned tools, showing up different examples of configuration, defining their advantages and disadvantages, showing the obtained results and their description. We conclude that pmacct is a good solution to deploy and implement teaching activities in order to study techniques related with traffic and routing monitoring.

Legitimate web browser redirection is often used to take users to web pages that have moved or to help users find the correct website when they have entered the web address incorrectly. Unfortunately, computer network attackers can use web browser redirection to manage malware-serving hosts and conceal their activity. An analysis of network flow records yields heuristics for flow size, flow duration, and inter-flow duration that indicate flows where web browser redirection is likely to have occurred. Results show that flows matching these redirection heuristics are indeed several times more likely to communicate with Internet hosts that have exhibited a history of malicious behavior. A network security administrator can thus filter large sets of network flow records to reveal flows most likely to contain web browser redirection. This capability reduces the sample space when looking for evidence of malicious activity targeting web browsers and contributes more generally to the expanding field of flow-based application recognition.

Brute-force attacks are a prevalent phenomenon that is getting harderto successfully detect on a network level due to increasing volume and en-cryption of network traffic and growing ubiquity of high-speed networks.Although the research in this field advanced considerably, there still remainclasses of attacks that are undetectable. In this chapter, we present sev-eral methods for the detection of brute-force attacks based on the analysisof network flows. We discuss their strengths and shortcomings as well asshortcomings of flow-based methods in general. We also demonstrate thefragility of some methods by introducing detection evasion techniques.

Brute-force attacks are a prevalent phenomenon that is getting harderto successfully detect on a network level due to increasing volume and en-cryption of network traffic and growing ubiquity of high-speed networks.Although the research in this field advanced considerably, there still remainclasses of attacks that are undetectable. In this chapter, we present sev-eral methods for the detection of brute-force attacks based on the analysisof network flows. We discuss their strengths and shortcomings as well asshortcomings of flow-based methods in general. We also demonstrate thefragility of some methods by introducing detection evasion techniques.