enhanced security for hadoop distributed file system … · hadoop is an open source framework used...

ENHANCED SECURITY FORHADOOP DISTRIBUTED FILE

SYSTEM BY USING DNACRYPTOGRAPHY

Balaraju.J 1 ,Dr.P.V.R.D. Prasada Rao 2

Rajeev Gandhi Memorial College of EngineeringTechnology.,Nandyal & Research scholar1

Professor 2

School of Computer ScienceEngineering,

KLEF University, Guntur, India.

July 19, 2018

Abstract

Many business organizations and researchers use bigdata for regular storage but, that may not have thefundamental aspects particularly from security and privacyperspective. Enormous quantity of data is generating dayby day from multiple sources in different formats. Thethree most important challenges are, how to store, processand secure big data. Hadoop is capable of handling thecommodity hardware and it is efficient for storage andprocessing of data but it has certain security issues. Thispaper proposes Hadoop based data centre which stores bigdata. The proposed security mechanism uses DNACryptography by creating new Secure-HDNA Node andthis mechanism includes collecting metadata of each userfrom Name Node and updating for every 8 seconds. TheSecure-HDNA Node is an interface between users and

1

International Journal of Pure and Applied MathematicsVolume 120 No. 6 2018, 8127-8142ISSN: 1314-3395 (on-line version)url: http://www.acadpubl.eu/hub/Special Issue http://www.acadpubl.eu/hub/

8127

Name Node for providing authentication by storing userinformation in an encrypted form and metadata of users.The proposed method may provide optimal solution forimproving security and performance by eliminating thedisadvantages of existing HDFS security mechanisms.

Keywords: Big Data, Hadoop, HDFS, DNACryptography, Secure-HDNANODE

1 Introduction

Bigdata [1] refers to the huge volume of complex data beinggenerated from different sources like sensors, social networks,retail, logistics and financial databases and internet of things. Thetraditional computing technologies are not capable for analyzingand handling the big data because the data includes structured,semi structured and un-structured data. In Real time, Big Data isnot just a process for storing petabytes and Exabytes of data in acloud but also useful for improving cloud data centers in businessorganizations. Softwares like Hadoop[2], NoSQL[3] databases andcloud computing are available to deal with big data. Big data isgenerally characterized by 3 V’s are Volume, Velocity and Variety[4]. The characteristics make it difficult for storing and processingbig data using traditional data processing application software.Hadoop is an open source framework used for distributed datastorage and parallel processing on big data sets. One is massivedata storage called as Hadoop Distributed File System (HDFS) [5]and another one is faster processing techniques called as MapReduce [6]. Hadoop stores the data as it is without encryption toimprove efficiency.

2 Hadoop Distributed File System

HDFS is a distributed data storage and distributed parallelprocessing system for handling the big data. The input data isfirst split, in such a way that it fits the size of the block in DataNodes [7]. By default each block has its unique id; presently thedefault block size is 128MB, which was 64MB previous version.HDFS does not waste the memory when data size is less than 128

2

International Journal of Pure and Applied Mathematics Special Issue

8128

MB and it releases the free memory for other data. The storedblock is replicated by two for securing the data so that if oneblock is lost another block of data is available. And all the blockscommunicate periodically with their Data Node, which in turncommunicate periodically in every 8 seconds with its Name Node.

2.1 HDFS Security Mechanisms

In any distributed system, the communication between two nodesis only established when there will be trusted communicationbetween two nodes, through a secured distributed network onauthentication. Communication over unsecured distributednetwork will lead to leakage of information to untrustedenvironment.

a.In order to accomplish the authentication between user andName Node for accessing data blocks from Data Node,Kerberos[8] authentication mechanism is used for improving thesecurity. The association between client and namenode is reachedby using remote procedure call in HDFS. User want to access datafrom Data Node, they must use Ticket Granting Ticket or ServiceTicket to authenticate by Name Node. The TGT and ST can berenewed while Kerberos is rehabilitated after long running of jobs.The newly renewed TGT,ST are strewn for all tasks. K.D.Cissues the Service Ticket to the Kerberos by using TGT, once therequest is arrived from the task. The aim of this ways and meansis to create endorsement between user and Name Node forretrieving data.

b. In order to screen and secure profound data like personaldetails, account numbers, passwords to upturn the haven in HDFSBull’s Eye[9] security mechanism is proposed. This exemplaryoutlooks all the sensitive info in 360◦and find whether allwarehoused data is secured without any menace. It consents onlyaccredited users to realm their own information of the users in aproper way. This feature increases security in Hadoop basestratum which provides security to the sensitive data in HDFSround the clock and to all the applied nodes. By applying thistactic to the available nodes in the rack1, data storage in the

3


8129

appropriate data block is verified along with the securityverification which allows the data storage in the required blocksdata blocks only by particular user. This acts as a bridge to fillthe gap in between the original data and the replicated one.Whenever the patron wants to secure the replicated data thenaccording to the bull eye method the relation between two racks isverified and then the replicated information stored in another rackis also provided with security. Once the relation is verified, theread or write privileges are activated for the user for the storage ofdata in data blocks. This algorithm is applicable for all varietiesof data ranging from terabytes to multi petabytes. Manycompanies like Data guise’s DGsecure and Amazon ElasticMapReduce used this approaches. DG secure Company gave aData driven and Governance arrangements and furthermoreincludes security for Hadoop in the cloud.

c. In HDFS, if the Name Node (NN) is lost entire Hadoop systemwill fail. The Name Node is Master Node which vittles metadataand user cannot admittance their data without Name Node. Witha specific end goal to upsurge security in information attainablequality is achieved by including Secondary Name Node. Anadditional Name Node besides the existing Name Node isprovided for avoiding Hadoop crash so that the data is not lost.These two Name Nodes can be configured with in the same clusterwhich is nothing but NN Security Enhancement (NNSE)[10], andit uses algorithm called Bull eye. Hadoop Administrator canaccess both the nodes among which, one will be acting as masternode and other one is slave node. Equal rights are not providedfor both the slave and master nodes. In the absence of masternode Admin cannot access data from slave node without approvalfrom NNSE and it may condense complex recovery process. Thismechanism reduces the crashes and secondary name node sendsthe heartbeats for every 8 seconds to the Master Node in order toacquire the meta data. By using this imperative setup user has asecure route for accessing information continuously from thehadoop cluster. In future, security can be enhanced by providingvibrant configuration that ensures the availability of data in asecured manner by performing the replication of name nodeswhich is done by NNSE in HDFS clusters.

4


8130

3 Problem Definition

Integrating Kerberos protocol with the HDFS Frame work is quitea tedious task. Hadoop Distributed systems need a more efficientdesign on Granting and Revoking capabilities. The above threeHDFS security mechanisms works independently and NNSEsecurity mechanism is a copy of Name Node. Kerberos protocol isnot proprietary of HDFS and bull’s eye algorithm is used forsecuring sensitive data in hadoop cluster. The proposedSecure-HDNANode can be used to overcome the drawbacks ofexisting HDFS security mechanisms by combining the features ofAuthentication and securing metadata. This will give a partialsubstitution to the Kerberos and Name Node Security Enhancemethodologies.

4 DNA Based Big Data Security in

HDFS

Nowadays, many researchers are using DNA [11] security conceptto improve the security of handling the big data by the popularHadoop Distributed File System. Biological DNA orDeoxyribonucleic Acid is the molecule that contains the geneticinformation and functioning of all living organisms and viruses.Genetic information is encoded as a sequence of nucleotides(Guanine, Adenine, Thymine and Cytosine). A DNA sequence[12] consists of four 4 letters of alphabet A, C, G, and T. Eachletter is related to a nucleotide and represents binary numbers likeA with 00, C with 01, G with 10 and T with 11. DNA Computingis a bimolecular computation and it makes use of biologicalmethods for performing massively parallel distributedcomputations.

DNA based cryptography [13] techniques are the most interestingarea for the Big data security, because of its complex structureand no direct connection between DNA and information security[14]. The application of DNA cryptography is in its initial stage ofbig data security.

5


8131

5 Method Of Securing Big Data In

Cloud Using DNA

Suyel Namsudra, Pinki Roy et al [15] proposed Efficient SecureDNA based data Access control model for cloud environmentbased on data sizes shared by data owners. This method also usesa long 512-bit secret key which is based on the users attributes ordata decryption key for data storing process, reducing data accesstime and the users can pay fewer amounts for utilizing cost forusing cloud services. This technology also having several serviceslike system setup, user registration, user authorization, datastorage which encrypts and decrypts the entire data by usingcomplex DNA crypto systems which leads to decrease inperformance. For this purpose the cloud server maintainsCSP TAB table and data access. This mechanism tries to providestrong security, provides a scalable and efficient data storageprocess on the cloud server. This method concentrates on storage,access and securing big data in cloud and mainly focus is done onoriginal data rather than on the Meta data.

The proposed methodology includes DNA crypto system whichfocuses on secure authentication and securing Meta data bycreating an additional node called Secure-HDNA Node in Hadoopwhich leads to increase in the performance.

6 Proposed System

As shown in Fig 6(a). The Cloud Data center is configured withHadoop by adding an additional Secure-HDNA Node for thepurpose of user authentication and securing metadata. Hadoophas been using two separate security mechanisms, one forauthentication (Kerberos) another for metadata (NNSE) security.Performance of Hadoop based cloud data centers is not up to themark due to the usage of two separate security mechanisms. Toovercome the existing disadvantages Hadoop requires its ownsecurity protocols. The proposed mechanism paves way for thebetter security and improves performance.

6


8132

Data Center by Using HADOOP Secure- HDNANode

Fig.6 (a). HDFS Proposed System

The main goal of proposed scheme is Secure Authentication andSecuring the Meta Data by eliminating the disadvantages of existingHDFS security mechanisms.

6.1 Authentication with Secure-HDNA Node

Secure-HDNANode provides Secure Authentication between usersand HDFS, for storing data and accessing data in two phases.

6.1.1 User Registration and Authentication Phase

Every new user must send request to Secure-HDNANode byproviding email id. Secure-HDNANode accepts request from theuser and the corresponding emaild will be converted into DNAsequence for hiding the email id. The DNA characters will beassigned with sequential numbers starting from A to 0, C to 1, Gto 3 and T to 3 in continuous manner in order to obtain decimalnumber. The obtained decimal number is multiplied with

7


8133

increasing powers of 4 and then added. The obtained number isthe unique key for the first individual member of the mail id.

Similarly unique keys for the rest of the members in mail id canbe obtained. All the obtained unique keys are combined with aseparator (-). The obtained unique key will be maintained by theSecure-HDNANode in common table that is constructed for thepurpose of all users.

6.1.2 Unique Key Generation Process Using DNACrypto Systems

Table-1. Unique Key Generation Process

Secure-HDNANode sends the generated unique key to the user forfurther registration process, which includes name, date of birth,mobile number, address and other credentials. All these credentialswill be updated in the table that is being maintained by the Secure-HDNANode based on the unique key that is sent to the user forthe completion of registration process.

8


8134

Fig.6 (b). User Authentication Process

Once the registration process is completed the user sends the mailid and corresponding unique key to the NameNode. Now theNameNode requests the Secure-HDNANode for the verification ofthe mail id and corresponding unique key that is sent by the user.Secure-HDNANode receives the request from NameNode andchecks for the availability of the mail id and corresponding uniquekey in the table. The Secure-HDNANode acknowledges theNameNode whether the user is authorized or not.

NameNode grants the permission to the user to send the data toit, if the user is authorized. Now there is a trusted communicationchannel between user, NameNode and Secure-HDNANode. Theuser sends the data to the NameNode and it distributes the dataamong the data blocks available in Data Nodes.

9


8135

6.2 Securing Metadata

Fig.6(c). Metadata Storage in Secure-HDNA Node

Once the distribution of data is over the NameNode maintainsmetadata of the users and the communication between theNameNode and corresponding DataNodes occurs for every 8seconds. The Secure-HDNANode sends unique key and mail id toNameNode requesting for users Meta data. NameNode verifies theunique key and sends the corresponding Meta data toSecure-HDNANode and it will update the metadata in centraltable in every 8 seconds. The users can access their data fromDataNodes by acquiring metadata from central table ofSecure-HDNANode even in the absence of NameNode andSecondary Name Node. This can be used as an optimal solutionto the permanent crashing of Hadoop.

7 Security Technologies For Securing

Hadoop

In this we will use the various commercial and open sourcetechnologies that are available to address the various securityaspects of big data Hadoop. Cloud Data Center configured withHadoop 2.x and above single node technology (NameNode).

Apache Sentry [16] an open source project by Cloudera is anauthorization module for Hadoop that offers the granular,role-based authorization required to provide precise levels ofaccess to the right users and applications and supports forrole-based authorization, fine-grained authorization, and

10


8136

multi-tenant administration [17] and developing with extraSecure-HDNANode with JAVA or Python 3.0 and its maintaincentral table for users credentials are stores in NOSQL Databases.

Apache Knox [18] Gateway is a system that provides a singlepoint of authentication and access for various Hadoop services ina cluster. It provides a perimeter security solution for Hadoop.The second advantage is it supports various authentication andtoken verification scenarios. It manages security across multipleclusters and versions of Hadoop. It also provides SSO solutions,and allows integrating other identity management solutions suchas LDAP, Active Directory (AD), and SAML based SSO andother SSO systems [19].

Project Rhino [20] provides an integrated end-to-end datasecurity solution to the Hadoop ecosystem. It provides a tokenbased authentication and SSO solution. It offers Hadoop cryptocodec framework and crypto codec implementation to provideblock level encryption for the data stored in Hadoop.

8 Comparative Study of Security

Performance

Fig.8 (a): HDFS with External Security Mechanisms.

Figure 8(a) shows HDFS system with external security

11


8137

mechanisms like Kerberos and NNSE. It provides minimalsecurity but computational time is high, because forauthentication purpose Kerberos requires KDC. It is difficult toaccess data for an user in the absence of NNSE and Name Node.

Fig.8 (b): HDFS Integrated with Secure-HDNANode

Figure 8 (b) shows HDFS integrated with Secure-HDNA Nodewhich is used for both authentication and meta data securitypurpose which leads to less computational time and securityperformance is enhanced.

Fig 8(c)Comparative Representation of Nodes and ComputationalTime for Authentication

12


8138

9 Conclusion and Future Work

Hadoop Distributed File System is different from normalDistributed System. Kerberos protocol is developed based on thenormal distributed system; HDFS uses Kerberos as one of thesecurity mechanisms. Since Kerberos protocol is not proprietaryof HDFS and Hadoop may crash in the absence of NameNode andNNSE, the performance is not up to the mark. To overcome this,Hadoop needs its own protocol (Secure-HDNANode) which servesthe purpose of authentication and metadata security. The visionof this proposed system is to reduce hadoop crashes whichultimately improve the performance and Data Security. Thefuture work of this system is to encrypt the metadata by usingcomplex DNA crypto system, so that it is not easy for the hackersto hack data, even though metadata is hacked from central tableof Secure-HDNANode. By implementing all the above methods ,HDFS provides better security of Bigdata and performance eventhough this is applied for Hadoop based cloud computing.

References

[1] YojnaArora, Dinesh Goyal Big Data: A Review of AnalyticsMethods Techniques”, International Conference on EmergingTrends in Computing and Communication Technologies(ICETCCT), 978-1-5090-5256-1116/$31.00 2016 IEEE.

[2] Dhole Poonam B, GunjalBaisa L, ”Survey Paper onTraditional Hadoop and Pipelined Map Reduce”, InternationalJournal of Computational Engineering Research Vol 03,Issue12.

[3] Jing Han, Haihong E, Guan Le, Jian Du,”Survey onNoSQL Database”,International Conference on PervasiveComputing and Applications (ICPCA),2011 978-1-4577-0208-2/11/26.002011IEEE.

13


8139

[4] Stephen Kaisler, Frank Armour, J.Alberto Espinosa and WolliamMoney ”Big Data: Issues and Challenges Moving Forward,” HawaiiInternational Conference on System Sciences 46th, pp-995-1003,2013.

[5] Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoopdistributed file system. In: Proceedings of the 2010 IEEE 26thSymposium on Mass Storage Systems and Technologies (MSST),pp. 110 (2010).

[6] J. Dean and S. Ghemawat, Mapreduce: Simplified data processingon large clusters, Commun. of ACM, vol. 51, no. 1, pp. 107-113,2008.

[7] Ms. VibhavariChavan, Prof. Rajesh. N. Phursule,” Survey PaperOn Big Data, ”(IJCSIT) International Journal of Computer Scienceand Information Technologies, Vol. 5 (6) , 2014,7932-7939.

[8] Al-Janabi, Rasheed, M.A.-S., ”Public-Key Cryptography EnabledKerberos Authentication”, IEEE, Developments in E-systemsEngineering (DeSE), 2011.

[9] B. Saraladevia, N. Pazhanirajaa, P. Victer Paula, M.S. SaleemBashab, P. Dhavachelvanc ”Big Data and Hadoop-A Study inSecurity Perspective” International Symposium on Big Data andCloud Computing (ISBCC’15) doi:10.1016/j.procs.2015.04.091 [10].

[10] Duygu SLQDQF THUL G , Ramazan THUL, Seref SDJLURJOX,”A Survey on Security and Privacy Issues in Big Data” , thInternational Conference for Internet Technology and SecuredTransactions (ICITST-2015), th International Conference forInternet Technology and Secured Transactions (ICITST-2015).

[11] M.R. Abbasy, A.A. Manaf, M.A. Shahidan, Data hiding methodbased on DNA basic characteristics, in: E. Ariwa, E.E. Qawasmeh(Eds.), Digital Enterprise and information Systems, Springer,Berlin Heidelberg, 2011, pp. 5362.

[12] He PA, Wang J. , ”Characteristic sequences for DNA primarysequence.” , J Chem Inf Comput Sci. 2002 Sep-Oct;42(5):1080-5.

14


8140

[13] Ashish Gehani ,Thomas LaBean , John Reif, ”DNA-basedCryptography”,Molecular Computing (Head Festschrift),LNCS 2950, pp.167188, 2004. c Springer-Verlag Berlin Heidelberg2004.

[14] Guangzhao Cui , Limin Qin, Yanfeng Wang , Xuncai Zhang,”Information Security Technology Based on DNA Computing” 1-4244-1035-5/07/25.00.2007IEEE.

[15] Suyel Namasudra , Pinki Roy , PandiVijayakumar,SivaramanAudithan, Balamurugan Balusamy ,”Time Efficient Secure DNA Based Access Control Model forCloud Computing Environment” Future Generation ComputerSystems (2017), http://dx.doi.org /10.1016 /j.future.2017.01.017.

[16] Vivekanand; Vidyavath i, B. M. ”Security Challenges in Big Data:Review”, International Journal of Advanced Research in ComputerScience . Jul/Aug2015, Vol. 6 Issue 6, p199-201. 3p

[17] Hassan Takabi , James B.D. Joshi, Gail-Joon Ahn, ”Security andPrivacy Challenges in Cloud Computing Environments” , IEEESecurity Privacy ( Volume: 8, Issue: 6, Nov.-Dec. 2010 )

[18] Masoumeh Rezaei Jam, Leili Mohammad Khanli, MortezaSargolzaei Javan, A survey on security of Hadoop, InternationaleConference on Computer and Knowledge Engineering (ICCKE),IEEE 2014, DOI: 10.1109/ICCKE.2014.6993455.

[19] Andreas Pashalidis and Chris J. Mitchell, ”A Taxonomy of SingleSign-On Systems”, R. Safavi-Naini and J. Seberry (Eds.): ACISP2003, LNCS 2727, pp. 249264, 2003. Berlin Heidelberg 2003.

[20] Issa Khalil, Zuochao Dou, Abdallah Khreishah , ”TPM-BasedAuthentication Mechanism for Apache Hadoop” , InternationalConference on Security and Privacy in Communication Networkspp 105-122 , 2015

15


8141

enhanced security for hadoop distributed file system … · hadoop is an open source framework used...

Documents