Big Data Analytics Architecture for Cybersecurity Applications
Abstract
The technological and social changes in the cur- rent information age pose new challenges for security analysts. Novel strategies and security solutions are sought to improve security operations concerning the detection and analysis of security threats and attacks. Security analysts address security challenges by analyzing large amounts of data from server logs, communication equipment, security solutions, and blogs related to information security in different structured and unstructured formats. In this paper, we examine the application of big data to support some security activities and conceptual models to generate knowledge that can be used for the decision making or automation of security response action. Concretely, we present a massive data processing methodology and introduce a big data architecture devised for cybersecurity applications. This architecture identifies anomalous behavior patterns and trends to anticipate cybersecurity attacks characterized as relatively random, spontaneous, and out of the ordinary.
Downloads
References
IBM. AI for cybersecurity. [Online]. Available: https://www.ibm.com/security/artificial-intelligence [Accessed: Nov.25, 2020].
Kaspersky. New IoT-malware grew three-fold in H1 2018. [Online]. Availablet: https://www.kaspersky.com/ [Accessed: Nov.25, 2020].
Microsoft. Enhancing Cybersecurity with Big Data: Challenges and Opportunities. [Online]. Availablet: https://query.prod.cms.rt.microsoft.com [Accessed: Nov.25, 2020].
SK., Kamaruddin and V. Ravi, “Credit Card Fraud Detection using Big Data Analytics: Use of PSOAANN based One-Class Classi- fication,” In Proceedings of the International Conference on Informatics and Analytics (ICIA-16). ACM, New York, NY, USA, Article 33 , 8 pages, 2016.
FBI. Audit of the Federal Bureau of Investigation’s Cyber Threat Priori- tization . [Online]. Available: https://oig.justice.gov/reports/2016/ [Accessed: Nov.25, 2020].
Kaspersky. DDoS attacks in Q4 2016. [Online]. Available: https://securelist.com/ddos-attacks-in-q4-2016/77412/ [Accessed: Nov.25, 2020].
P. Las Casas, V. Santos Dias, W. Meira Jr, and D. Guedes, “A Big Data Architecture for Security Data and Its Application to Phishing Characterization,” pp.36-41, 2016
NIST. Data Science. [Online]. Available: https://www.nist.gov/programs-projects/data-science [Accessed: Nov.25, 2020].
H. Rasheed, A. Hadi and M. Khader, “Threat Hunting Using GRR Rapid Response,” International Conference on New Trends in Computing Sciences (ICTCS), Amman, 2017, pp. 155–160.
NIST. Big Data Public Working Group. [Online]. Availablet: https://www.nist.gov/el/cyber-physical-systems/big-data-pwg [Accessed: Nov.25, 2020].
U. Fayyad, G. Piatetsky-Shapiro and P. Smyth, “From data mining to knowledge discovery in databases,” AI magazine, 17(3), pp. 37–37, 1996.
R. Alguliyev and Y. Imamverdiyev, “Big data: Big Promises for Information Security,” IEEE 8th International Conference on Application of Information and Communication Technologies (AICT), Astana, 2014, pp. 1–4.
S.R. Bandre, and J.N Nandimath, “Design consideration of Network Intrusion detection system using Hadoop and GPGPU,” 2015 International Conference on Pervasive Computing (ICPC), Pune, pp. 1– 6.
Bayer, Ulrich, P. Comparetti, C. Hlauschek, Ch. Kruegel, and E. Kirda, “Scalable, behavior-based malware clustering,” In NDSS, vol. 9, pp. 8–11. 2009.
J. Bin, M, Yan, H. Xiaohong, L, Zhaowen and S. Yi, “A Novel Real-Time DDoS Attack Detection Mechanism Based on MDRA Algorithm in Big Data,” Mathematical Problems in Engineering. 2016. pp. 1–10.
Z. Chen, H. Zhang, W.G. Hatcher, J. Nguyen and W. Yu, “A streaming-based network monitoring and threat detection system,” IEEE 14th International Conference on Software Engineering Research, Management and Applications (SERA), Towson, MD, 2016, pp. 31–37.
Cloudera. Cloudera cybersecurity. [Online]. Available: https://www.cloudera.com/ [Accessed: Nov.10, 2020].
A. Dauda, S. Mclean, A. Almehmadi and K. El-Khatib, “Big Data Analytics Architecture for Security Intelligence,” Proceedings of the 11th International Conference on Security of Information and Networks, 2018.
L. Fetjah, K. Benzidane, H.E. Alloussi, O.E Warrak, S. Jai-Andaloussi and A. Sekkaki, “Toward a Big Data Architecture for Security Events Analytic,” IEEE 3rd International Conference on Cyber Security and Cloud Computing (CSCloud), Beijing, 2016, pp. 190–197.
R. Fontugne, J. Mazel and K. Fukuda, “Hashdoop: A MapReduce framework for network anomaly detection,” IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Toronto, ON, 2014, pp. 494–499.
Hadoop. Apache Hadoop. [Online]. Available: https://hadoop.apache.org/ [Accessed: Nov.10, 2020].
C. Hsieh and T. Chan, “Detection DDoS attacks based on neural- network using Apache Spark,” International Conference on Applied System Innovation (ICASI), Okinawa, 2016, pp. 1–4.
Hortonworks. Ciberseguridad de los macrodatos. [Online]. Available: https://es.hortonworks.com/ [Accessed: Nov.10, 2020].
G.P.Gupta and M. Kulariya, “A Framework for Fast and Efficient Cyber Security Network Intrusion Detection Using Apache Spark,” Procedia Computer Science.
IBM. Watson and Cybersecurity: The Big Data challenge. [Online]. Available: https://www.ibm.com/blogs/think [Accessed: Nov.10, 2020].
IBM. Cognitive Cybersecurity Intelligence (CCSI) Group. [Online]. Available at: https://researcher.watson.ibm.com/researcher [Accessed: Nov.10, 2020].
IEEE. IEEE Special Interest Group (SIG). [Online]. Available: http://computing.northumbria.ac.uk/staff/FGPD3/sig-bdcsp/ [Accessed: Nov.10, 2020].
ITU. Study Group 17. [Online]. Available: https://www.itu.int/en/ITUT/about/groups/Pages/sg17.aspx [Accessed: Nov.10, 2020].
Z. Jia, C. Shen, X. Yi, Y. Chen, T. Yu and X.Guan, “Big- data analysis of multi-source logs for anomaly detection on network- based system,” 13th IEEE Conference on Automation Science and Engineering (CASE), 2017.
Lighari, S. N., and Hussain, D. M. A. (2017). Testing of algorithms for anomaly detection in Big Data using apache spark. 2017 9th Inter- national Conference on Computational Intelligence and Communication Networks (CICN).
H.C. Manjunatha and R.Mohanasundaram, “BRNADS: Big data real-time node anomaly detection in social networks,” 2nd International Conference on Inventive Systems and Control (ICISC), 2018.
S. Marchal, X. Jiang, R. State, R and T. Engel, “A Big Data Architecture for Large Scale Security Monitoring,” In Proceedings of the IEEE International Congress on Big Data (BIGDATACONGRESS ’14). IEEE Computer Society, Washington, DC, USA, 2014, pp. 56–63.
This article is published by LAJC under a Creative Commons Attribution-Non-Commercial-Share-Alike 4.0 International License. This means that non-exclusive copyright is transferred to the National Polytechnic School. The Author (s) give their consent to the Editorial Committee to publish the article in the issue that best suits the interests of this Journal. Find out more in our Copyright Notice.
Disclaimer
LAJC in no event shall be liable for any direct, indirect, incidental, punitive, or consequential copyright infringement claims related to articles that have been submitted for evaluation, or published in any issue of this journal. Find out more in our Disclaimer Notice.
