Abstract |
Network traffic monitoring is the basis for a multitude of systems, such as intrusion
detection, network forensics, and traffic classification systems, which support
the robust, efficient, and secure operation of modern computer networks. However,
building efficient network monitoring systems has become a challenging task.
Emerging network monitoring applications become more demanding in terms of
memory and CPU resources, due to the increasingly complex analysis operations
they need to perform on the monitored traffic. Moreover, many network monitoring
applications need to analyze the captured traffic at higher protocol layers. This
need for reconstructing high-level entities results in increased application complexity
and reduced performance. At the same time, the volume of traffic that should be
analyzed in today’s network links increases significantly. This leads to a growing
demand for more resources to monitor the network traffic at line speeds, while it
is very likely that the deployed monitoring systems will become overloaded. Even
worse, attackers are able to intentionally overload a network monitoring system
to impede its correct operation and pass malicious activities over the network undetected,
as the existing systems do not provide protection against such attacks.
Therefore, there is an increasing need for building efficient and robust network
monitoring systems that will provide intelligent overload control mechanisms, will
be able to defend against sophisticated attacks, and will utilize recent advances in
the available commodity hardware.
In this dissertation we address the above issues, and we propose new techniques
and frameworks to improve the performance, accuracy, and robustness of
network monitoring systems when processing high volumes of traffic using commodity
hardware. Our thesis is that we need to enrich the lower layers of a network
monitoring system with intelligence based on flow-level information from the
transport layer, in order to build efficient network monitoring systems under heavy
load. First, we show that rearranging the captured packet stream based on source
and destination port numbers can lead to significant performance benefits due to improved memory access locality. We implement this technique, which we call
as locality buffering, within a popular packet capture library, and we show its performance
improvements in common network monitoring applications. To improve
the accuracy of an overloaded Network-level Intrusion Detection System (NIDS),
we suggest to focus on the first few bytes of each connection, a technique we call
as selective packet discarding. Our evaluation shows that this approach can significantly
improve the effectiveness of a NIDS under extreme load. To defend against
overload attacks, we propose selective packet paging: a technique based on a twolayer
memory management system to prevent packet loss, and on a randomized
detection approach to find and isolate packets attacking the network monitoring
system. To fill the semantic gap we identified between monitoring applications,
which need to analyze network traffic at higher protocol layers, and monitoring
libraries, which deliver just raw IP packets, we present the design, implementation,
and evaluation of the Stream capture library (Scap): a new multicore-aware framework
for stream-oriented network traffic monitoring. Scap captures and delivers to
user-level programs reassembled transport-layer streams, allowing for a wide variety
of performance optimizations, such as hardware-assisted stream truncation,
prioritized packet loss, and flexible stream reassembly. Finally, we show that
our ideas can be applied in other problems of network monitoring systems as well,
such as long-term network traffic recording and reducing the detection latency of
an energy-efficient NIDS. To build more efficient and secure network monitoring
systems, all these techniques we propose rely on the fact that monitoring applications
are actually interested in a stream-oriented analysis.
|