Introduction

Internet traffic monitoring and analysis entails monitoring the Internet network links and understanding their behavior. This course will cover the techniques and tools being developed for Internet traffic monitoring and analysis. Active and passive monitoring techniques will be studied. In this course, students will get to develop algorithms and a prototype system for capturing packets and analyzing them for various purposes.

Instructor:

Prof. James Won-Ki Hong (PIRL337) : 279-2244, jwkhong(@)postech.ac.kr

Lectures:

Tue. & Thu. 11:00-12:15 (PIRL 222)

Pre-requisites:

A course on computer or telecommunication networks is required. A course on network management is recommended.

Required Textbook:

There will not be a textbook for this course. Lecture materials and research papers will be used for the course.

Recommended Books:

• Alberto Leon-Garcia, Communication Networks: Fundamental Concepts and Key Architectures, McGraw-Hill, ISBN: 0070228396, 2003.
• Richard Stevens, TCP/IP Illustrated, Volume 1: The Protocols, Addison-Wesley, ISBN: 0-201-63346-9, 1994.
• Douglas E. Comer, Computer Networks and Internets, Prentice Hall, ISBN 0-13-599010-6, 1997.
• D. Comer, Internetworking with TCP/IP, Vol I: Principles, Protocols, and Architecture, Second edition, Prentice-Hall, Englewood Cliffs, NJ, ISBN 0-13-468505-9 1991.
• D. Comer and D. Stevens, Internetworking with TCP/IP, Vol II: Design, Implementation, and Internals, Prentice-Hall, Englewood Cliffs, NJ, ISBN 0-13-472242-6 1991.
• William Stallings, Data and Computer Communications, Fifth Edition, Prentice Hall, ISBN 0-02-415425-3, 1997.
• William Stallings, SNMP, SNMPv2, SNMPv3 and RMON 1 and 2, Third Edition, Addison-Wesley, 1999.

Suggested Reference Journals and Conferences:

• John Wiley & Sons, International Journal of Network Management, ISSN 1055-7148.
• IEEE, IEEE Transactions on Network and Service Management
• Springer, Journal of Network and Systems Management, ISSN 1064-7570.
• IEEE/ACM, IEEE/ACM Transactions on Networking
• IEEE Communications Society, IEEE Network, ISSN 0890-8044.
• IEEE Communications Society, IEEE Communications Magazine, ISSN 0163-6084.
• Passive and Active Measurement Workshop (PAM): 2016, 2006, 2005, 2004, 2003, 2002, 2001, 2000
• Internet Measurement Conference (IMC)
• International Workshop on Traffic Monitoring & Analysis (TMA 2016)
• IEEE Network Softwarization (NetSoft)

Evaluation:

• Evaluation on each student will be done based on the following:
  1. Assignments - 40%
  2. Term Project - 50%
  3. Class Participation - 10%
• Note: the above evaluation scheme may change slightly during the course.

Term Project:

There will be a major term project (worth 50% of the final mark) on developing a traffic monitoring and analysis system. The topics will be discussed in class. Students will be asked to prepare, submit and present materials (Word & Powerpoint) related to the project throughout the course.
1. Project Proposals (due Oct. 15)
2. Requirements Document (due Oct. 29)
3. Detailed Design Document (due Nov. 12)
4. Demo & Implementation Document (due Dec. 15, 2-4pm)
5. Project Research Paper for a conference (due Dec. 20)

Assignments:

• There will be a few assignments (worth 40% of the final mark). You should submit your assignment materials to the course repository in Dropbox.
  • Assignment 1 (1%) (Out: Sept. 3, Due: Sept. 8 in class)
  • Assignment 2 (5%) (Out: Sept. 8, Due: Sept. 22 in class)
  • Assignment 3 (5%) (Out: Sept. 17, Due: Sept. 28 Midnight)
  • Assignment 4 (3%) (Out: Oct. 1, Due: Oct. 6 in class)
  • Assignment 5 (5%) (Out: Oct. 1, Due: Oct. 8 in class) Paper choices
  • Assignment 6 (5%) (Out: Oct. 15, Due: Oct. 22 in class) Paper choices
  • Assignment 7 (5%) (Out: Oct. 22, Due: Nov. 3 in class) Paper choices
  • Assignment 8 (5%) (Out: Nov. 10, Due: Nov. 17 in class) Paper choices
  • Assignment 9 (6%) (Out: Nov. 17, Due: Nov. 24 in class) Paper choices
  • Assignment 10 (6%) (Out: Nov. 24, Due: Dec. 1 in class) Paper choices

• Note: the above assignment schedule may change slightly during the course.

• Late assignments may be handed in, but there will be a penalty of 20% of the mark for assignments turned in less than one day late, and an additional penalty of 10% for each day thereafter.

• Cheating Policy -- Cheating will not be tolerated in this course. Students are encouraged discuss things related to courses and assignments but the materials handed in must be his/her own. The maximum penalty for the first offense is for the assignment in question. For subsequent offenses may result in an automatic failure of the course and possibly other academic punishments.

Class Participation:

Students are strongly encouraged to attend all lectures and to participate in discussions during lectures. 10% of the final mark is assigned for good and active class participation.

Research Papers:

• Surveys
  1. "A survey of network flow applications," Bingdong Li, Jeff Springer, George Bebis and Mehmet Hadi Gunes, Journal of Network and Computer Applications 36 (2013) 567–581
  2. "Flow Monitoring Explained: From Packet Capture to Data Analysis With NetFlow and IPFIX," Rick Hofstede, Pavel Celeda, Brian Trammell, Idilio Drago, Ramin Sadre, Anna Sperotto, and Aiko Pras, IEEE COMMUNICATION SURVEYS & TUTORIALS, VOL. 16, NO. 4, FOURTH QUARTER 2014
  3. "A Survey on Internet Traffic Identification," Callado, Carlos Kamienski, Geza Szabo, Balazs Gero, Judith Kelner, Stenio Fernandes, Djamel Sadok, IEEE Communications Surveys & Tutorials, Vol. 11, No. 3. (2009), pp. 37-52.
  4. "A Survey of Techniques for Internet Traffic Classification Using Machine Learning," T. Naguyen and G. Armitage, IEEE Communications Surveys and Tutorials, 2008
  5. "A Taxonomy of DDoS attacks and DDoS defense Mechanisms," Jelena Mirkovic and Peter Reiher. ACM Computer Communication Review, Volume 34 Issue 2, April 2004, pp. 39-53.
  6. "Anomaly detection: A survey," V. Chandola, A. Banerjee and V. Kumar, ACM Computing Surveys, Vol. 41, Issue 3, July 2009, pp. 15:1-15:58.

• Traffic Measurement and Analysis
  1. "The hadoop distributed file system," Shvachko, Konstantin, et al, IEEE Mass Storage Systems and Technologies (MSST), 2010
  2. "Research of Hadoop-based data flow management system," Zhi, Qiu, Zhao-wen LIN, and Ma Yan, The Journal of China Universities of Posts and Telecommunications 18 (2011): 164-168
  3. "Toward scalable internet traffic measurement and analysis with hadoop," Lee, Yeonhee, and Youngseok Lee, ACM SIGCOMM Computer Communication Review 43.1 (2013): 5-13
  4. "A Small-time Scale Netflow-based Anomaly Traffic Detecting Method Using MapReduce," Jin-Song, Wang, et al, International Journal of Security and Its Applications 8.2 (2014): 231-242
  5. "Seven Years and One Day: Sketching the Evolution of Internet Traffic," P. Borgnat, G. Dewael, K. Fukuda, P. Abry, and K. Cho, IEEE INFOCOM 2009, Rio de Janeiro, Brazil, April, 2009, pp.711--719.
  6. "On the stability of the information carried by traffic flow features at the packet level," A. Este, F. Gringoli, and L. Salgarelli. SIGCOMM Comput. Commun. Rev., 39(3):13.18, 2009.
  7. "Unveiling core network-wide communication patterns through application traffic activity graph decomposition," Y. Jin, E. Sharafuddin, and Z.-L. Zhang, ACM SIGMETRICS, 2009.
  8. "On Dominant Characteristics of Residential Broadband Internet Traffic," G. Maierm, A. Feldmann, V. Paxson and M. Allman, ACM SIGCOMM IMC, 2009
  9. "Profiling the End Host," T. Karagiannis and K. Papagiannaki, N. Taft and M. Faloutsos, Pasive and Active Measurement, 2007
  10. "Network Traffic Characteristics of Data Centers in the Wild," T. Benson, A. Akella and D. Maltz, ACM IMC, 2010.
  11. "Characterizing the Global Impact of the P2P Overlay on the AS-level Underlay," A. Rasti, R. Rejaie and W. Willinger, Passive and Active Measurement, Zurich, Switzerland, Apr. 7-9, 2010.
  12. "Youtube traffic characterization: a view from the edge," P. Gill, M. Arlitt, Z. Li and A. Mahanti, ACM IMC, 2007.

• Network Performance
  1. "Pingmesh: A Large-Scale System for Data Center Network Latency Measurement and Analysis", Chuanxiong Guo, et al, SIGCOMM 2015.
  2. "Large-scale measurements of wireless network behavior" Sanjit Biswas, et al, SIGCOMM 2015.
  3. "BackFi: High Throughput WiFi Backscatter" Dinesh Bharadia, et al, SIGCOMM 2015.
  4. "Packet-Level Telemetry in Large Datacenter Networks" Yibo Zhu, et al, SIGCOMM 2015.
  5. "Planck: Millisecond-scale Monitoring and Control for Commodity Networks" Jeff Rasley, et al, SIGCOMM 2014.
  6. "Characterizing user behavior and network performance in a public wireless LAN," A. Balachandran, G. Voelker, P. Bahl, P. Rangan, ACM SIGMETRICS, 2002.
  7. "Netgauge: A Network Performance Measurement Framework," T. Hoefler, T. Mehlan, A. Lumsdaine and W. Rehm, Proceedings of High Performance Computing and Communications (HPCC), Sep. 2007, pp.659--671.
  8. "WiMAX Performance Evaluation," P. Mach, R. Bestak, Sixth International Conference on Networking (ICN'07), Apr. 22-28, 2007, pp.17--20.
  9. "Mobile WiMAX systems: performance and evolution", F. Wang, A. Ghosh, C. Sankaran, P. Fleming, F. Hsieh, and S. Benes, IEEE Communications Magazine, Vol.46, Issue. 10, Oct. 2008, pp.41--49.
  10. "Best-case WiBro performance for a single flow," S. Woo, K. Jang, S. Kim, S. Cho, J. Lee, Y. Lee, S. Moon, ACM Workshop on Mobile Internet through Cellular Networks: Operations, Challenges, and Solutions (MICNET), October 2009, Beijing, China.
  11. "Evaluation of VoIP Quality over WiBro," M. Han, Y. Lee, S. Moon, K. Jang, D. Lee, Passive and Active Measurement Conference (PAM), April 2008.
  12. "Performance Impact of Large File Transfer on Web Proxy Caching: A Case Study in a High Bandwidth Campus Network Environment," H. Kim, D. Lee, K. Chon, B. Jang, T. Kwon, and Y. Choi, Journal of Communications and Networks, Volume 12, Number 1, Feb. 2010.

• Application Traffic Monitoring and Identification
  1. "Harvesting unique characteristics in packet sequences for effective application classification," Yuan, Zhenlong, Yibo Xue, and Yingfei Dong, IEEE Communications and Network Security (CNS), 2013
  2. "Automatic traffic signature extraction based on Smith-waterman algorithm for traffic classification," Feng, Xuepeng, et al, IEEE Broadband Network and Multimedia Technology (IC-BNMT), 2010
  3. "Automatic firewall rules generator for anomaly detection systems with Apriori algorithm," Saboori, Ehsan, Shafigh Parsazad, and Yasaman Sanatkhani, IEEE Advanced Computer Theory and Engineering (ICACTE), 2010
  4. "NetCube: a comprehensive network traffic analysis model based on multidimensional OLAP data cube," Park, Daihee, et al, International Journal of Network Management 23.2 (2013): 101-118
  5. "Traffic identification engine: an open platform for traffic classification," de Donato, Walter, Antonio Pescapé, and Alberto Dainotti, Network, IEEE 28.2 (2014): 56-64
  6. "A multilevel taxonomy and requirements for an optimal traffic-classification model," Khalife, Jawad, Amjad Hajjar, and Jesus Diaz-Verdejo, International Journal of Network Management 24.2 (2014): 101-120
  7. "Real-time traffic classification based on statistical tests for matching signatures with packet length distributions," Neto, Miguel, et al, IEEE Local & Metropolitan Area Networks (LANMAN), 2013
  8. "Ip mining: Extracting knowledge from the dynamics of the internet addressing space," Casas, Pedro, Pierdomenico Fiadino, and Arian Bar, IEEE Teletraffic Congress (ITC), 2013
  9. "Internet Traffic Classification Demystified: On the Sources of the Discriminative Power," Y. Lim, H. Kim, J. Jeong, C. Kim, T. Kwon, and Y. Choi, ACM SIGCOMM CoNEXT, Philadelphia, PA, Dec. 2010.
  10. "Early application identification," L. Bernaille, R. Teixeira, and K. Salamatian, ACM CoNEXT, 2006.
  11. "Traffic classification through simple statistical fingerprinting," M. Crotti, M. Dusi, F. Gringoli, and L. Salgarelli, SIGCOMM Comput. Commun. Rev., 37(1):5.16, 2007.
  12. "Graph-based p2p traffic classification at the internet backbone," M. Iliofotou, H.-c. Kim, M. Faloutsos, M. Mitzenmacher, P. Pappu, and G. Varghese, IEEE INFOCOM, 2009.
  13. "Comparing traffic classifiers," L. Salgarelli, F. Gringoli, and T. Karagiannis, SIGCOMM Comput. Commun. Rev., 37(3):65.68, 2007.
  14. "BLINC: Multilevel Traffic Classification in the Dark," T. Karagiannis, K. Papagiannaki, and M. Faloutsos, ACM SIGCOMM, Philadelphia, PA, August 2005.
  15. "PortLoad: taking the best of two worlds in traffic classification," Giuseppe Aceto, Alberto Dainotti, Walter de Donato, Antonio Pescape, IEEE INFOCOM, 2010
  16. "Lightweight, payload-based traffic classification: An experimental evaluation," M. Morandi O. Baldini A. Monclus P. Risso, F. Baldi, ICC08, May 2008.

• Social Network & Mobile Traffic
  1. "Understanding Mobile Traffic Patterns of Large Scale Cellular Towers," Huandong Wang, et. al, ACM IMC 2015, Tokyo, Japan, Oct. 2015.
  2. "Identifying Traffic Differentiation in Mobile Networks," Arash Kakhki Molavi, et. al, ACM IMC 2015 Tokyo, Japan, Oct. 2015
  3. "Tracking the Evolution and Diversity in Network Usage of Smartphones," Kensuke Fukuda, et. al, ACM IMC 2015, Tokyo, Japan, Oct. 2015.
  4. "Characterizing Smartphone Usage Patterns from Millions of Android Use," Huoran Li, et. al, ACM IMC 2015 Tokyo, Japan, Oct. 2015
  5. "Peeking Beneath the Hood of Uber," Le Chen, et. al, ACM IMC 2015, Tokyo, Japan, Oct. 2015.
  6. "An Analysis of Social Network-Based Sybil Defenses," Bimal Viswanath, Ansley Post, Krishna P. Gummadi, Alan Mislove, ACM SIGCOMM, 2010.
  7. "You are who you know: Inferring user profiles in online social networks," A. Mislove, B. Viswanath, K. P. Gummadi, and P. Druschel. In Proc. WSDM10, New York, NY, Feb 2010.
  8. "User Interactions in Social Networks and their Implications," C. Wilson, B. Boe, A. Sala, K. P. N. Puttaswamy, and B. Y. Zhao. In Proc. Eurosys09, Nuremberg, Germany, Apr 2009.
  9. "Comparison of Online Social Relations in Terms of Volume vs. Interaction: A Case Study of Cyworld", H. Chun, H. Kwak, Y. Eom, Y. Ahn, S. Moon, H. Jeong, ACM SIGCOMM IMC 2008.
  10. "Analysis of topological characteristics of huge online social networking services," Y.-Y. Ahn, WWW 07, New York, NY, USA, 2007.
  11. "Statistical properties of community structure in large social and information networks," J. Leskovec, WWW08, New York, NY, USA, 2008.
  12. "A First Look at Traffic on Smartphones, H. Falaki, D. Lymberopoulos, R. Mahajan, S. Kandula, and D. Estrin, ACM Internet Measurement Conference, Melbourne, Australia, Nov. 1-3, 2010.
  13. "A First Look at Mobile Hand-held Device Traffic," G. Maier, F. Schneider, and A. Feldmann, Passive and Active Measurement, Zurich, Switzerland, Apr. 7-9, 2010.
  14. "A Comparative Study of Handheld and Non-Handheld Traffic in Campus WiFi Networks," A. Gember, A. Anand, and A. Akella, Passive and Active Measurement, Atlanta, USA, Mar. 20-22, 2011.
  15. "Measurement Analysis of Mobile Data Networks," Young J. Won, B.C. Park, S.C. Hong, K.B. Jung, H.T. Ju, and James W. Hong, Passive and Active Measurement Conference (PAM 2007), Louvain-la-neuve, Belgium, Apr. 5-6, 2007, pp. 223-227.

• Network Security
  1. "Neither Snow Nor Rain Nor MITM... An Empirical Analysis of Email Delivery," Zakir Durumeric, et. al, ACM IMC 2015, Tokyo, Japan, Oct. 2015
  2. "Resilience of Deployed TCP to Blind Off-Path Attacks," Matthew Luckie, et. al, ACM IMC 2015, Tokyo, Japan, Oct. 2015
  3. "The Dark Menace: Characterizing Network-based Attacks in the Cloud," Rui Miao, et. al, ACM IMC 2015, Tokyo, Japan, Oct. 2015
  4. "Security of the Internet of Things: perspectives and challenges," Jing, Qi, et al, Wireless Networks 20.8 (2014): 2481-2501
  5. "Encrypted traffic classification based on an improved clustering algorithm," Zhang, Meng, et al, Trustworthy Computing and Services. Springer Berlin Heidelberg, 2013. 124-131
  6. "An investigation on identifying ssl traffic," McCarthy, Curtis, IEEE Computational Intelligence for Security and Defense Applications (CISDA), 2011
  7. "A Hybrid Approach for Accurate BT Traffic Identification," Zhang, Ru Hui, et al, Advanced Materials Research. Vol. 108. 2010
  8. *"Snort Lightweight Intrusion Detection for Networks," M. Roesch, Proceedings of LISA '99: 13th Systems Administration Conference Seattle, Washington, USA, November 1999, pp. 229-238.
  9. *"Diagnosing network-wide traffic anomalies," A. Lakhina, M. Crovella, and C. Diot, ACM SIGCOMM 2004, pp. 219-230.
  10. *"A Taxonomy of DDoS attacks and DDoS defense Mechanisms," Jelena Mirkovic and Peter Reiher. ACM Computer Communication Review, Volume 34 Issue 2, April 2004, pp. 39-53.
  11. *"Botnet Detection Based on Network Behavior", W. Strayer, David Lapsely, Robert Walsh and Carl Livadas, Advances in Information Security, 2008, Volume 36, 1-24.
  12. "DDoS attack detection method using cluster analysis," K. Lee, J. Kim, K. Kwon, Y. Han and S. Kim, Expert Systems with Applications, Volume 34, Issue 3, April 2008, Pages 1659-1665.
  13. *Modeling and Automated Containment of Worms," IEEE TRANSACTIONS ON DEPENDABLE AND SECURE COMPUTING, Vol. 5, No. 2, APRIL-JUNE 2008, pp. 71-86.
  14. *BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection," G. Gu, R. Perdisci, J. Zhang and W. Lee, 17th USENIX Security Symposium, 2008, pp. 139-154.
  15. *"Anomaly-based network intrusion detection: Techniques, systems and challenges", P. Garcia-Tedodro, J. Diaz-Verdejo, G.Macia-Fernandez and E. Vazquez, Computers & Security, Vol. 28, 2009, pp. 18-28.
  16. "Anomaly detection: A survey," V. Chandola, A. Banerjee and V. Kumar, ACM Computing Surveys, Vol. 41, Issue 3, July 2009, pp. 15:1-15:58.
  17. "BotGrep: Finding P2P Bots with Structured Graph Analysis," S. Nagaraja, P. Mittal, C. Hong, M. Caesar, and N. Borisov, USENIX Security 2010, Washington DC, USA, August 2010, pp. 1-16.
  18. *Top 10 Computer Viruses and Worms, ABC News, Sept. 2009.

Topics Covered:

The following is a tentative list of lecture topics for the course.

1. Overview of Computer Networking
2. Networking Protocols
   • Data Link Layer
   • Network Layer
   • Transport Layer
   • Application Layer
3. Introduction to Traffic Monitoring and Analysis
4. Traffic Monitoring Metrics and Approaches
5. Active Monitoring Techniques
6. Passive Monitoring Techniques
7. Standards Activities
8. PCAP
9. Wireshark Tutorial
10. Monitoring and Analysis Tools
11. Network Neutrality
12. DPNM Research Activities in Traffic Monitoring and Analysis (WebTrafMon & NG-Mon)
13. Project Presentations