Apache Hadoop is an open source big data processing platform. It has its own eco-system products to support various needs. Different big data products/platforms can integrate Hadoop and NoSQL into one platform so it provides better performance and a single source of truth. Let us have a look at how NoSQL and Hadoop can work together for big data challenges.
Sometime people often get confused that Hadoop is a database as it has a storage system associated. But let us clearly understand that Apache Hadoop is not at all a database.
Apache Hadoop is an open source big data platform consists of the following main components.
- HDFS: A file system known as Hadoop Distributed File System (HDFS)
- MapReduce: A distributed programming framework known as MapReduce
- Hadoop Common: It contains the libraries and utilities to support associated Hadoop modules.
- Hadoop YARN: This is called ‘Yet Another Resource Negotiator’. To je v bistvu platforma za upravljanje virov za upravljanje računalniških virov in naloge urnika.
Hadoop ima tudi drugo vrsto programskih paketov za podporo komponente ekosistemskih. Okvir podpira obdelavo podatkov intenzivne porazdeljene aplikacije. To omogoča aplikacijam, da delujejo v porazdeljenem okolju, sestavljena iz tisoč vozlišč in petabajtov podatkov. Vozlišča so neodvisni računalniki, znan tudi kot nizki stroški surovin strojne opreme. Hadoop grozd pomeni skupino računskih enot (v bistvu stroji) teče v splošnem okolju z Hadoop porazdeljenega datotečnega sistema (HDFS) podpreti luščenje.
Temeljna oblika cilj Hadoop je premagati okvare strojne opreme. Ker so strojne napake zelo pogoste in bi okvir lahko samodejno premagati. The Hadoop eco-system achieves this goal by all the modules.
The key features of Hadoop platform are a distributed storage and a distributed processing framework. The distributed storage (HDFS) splits large files into small blocks (default is 64MB) and distribute it across the clustered nodes. The distributed processing framework also known as ‘MapReduce’ supports very efficient parallel processing. The key feature of MapReduce is that, it ships the code (which will do the processing) to the node where the data resides. This is also called ‘data locality’, where the data remains in its original location and the code comes to it for doing the processing. This is indeed a revolution in parallel processing domain.
The main components of Hadoop (HDFS and MapReduce) are derived from Google’s File System (GFS) in Googlov MapReduce. Poleg zgoraj navedenih sestavin, Hadoop je sestavljen iz številnih sorodnih projektov, kot so Apache panj, Apache HBase, in Apache Pig etc.
On the other hand, NoSQL (razlagati tako, da ne samo "SQL") je ne-relacijski sistem za upravljanje podatkovnih baz. To je ugotovila, da se ne upoštevanje relacijski model baze podatkov. podatkovne baze NoSQL ne temeljijo predvsem na mizah.
Tehnologija NoSQL podatkovne baze zagotavlja učinkovit mehanizem za shranjevanje in pridobivanje podatkov, vendar to ni podobno kot relacijski model. Glavni cilji pri dizajnu podatkovnih baz NoSQL so preproste zasnove, horizontalni luščenje in boljšo dostopnost. Ime interpretira kot "ne samo SQL", zato podpira nekaj SQL, kot so poizvedbe jezike, kot HQL itd. podatkovne baze NoSQL se uporabljajo predvsem v velikih podatkov in analitičnih aplikacij.
So, Na kratko lahko definiramo Hadoop in NoSQL takole
- Hadoop: Distributed okvir računalništvo.
- NoSQL: Non-relacijske baze podatkov.
Kako lahko Hadoop in NoSQL sodelovati?
From the above discussion, je jasno, da je Hadoop in NoSQL ni ista stvar, vendar sta oba povezana s podatki intenzivno izračuna. Okvir Hadoop se večinoma uporablja za obdelavo velike količine podatkov (znan tudi kot velik podatkov) in NoSQL je zasnovan za učinkovito shranjevanje in pridobivanje velikih količin podatkov. Torej, vedno obstaja možnost, da se NoSQL kot del izvajanja Hadoop. V večini primerov, obdelani podatki iz Hadoop sistem je shranjena v bazi podatkov NoSQL. Vendar pa lahko vedno neodvisne primerov uporabe, ki morda ne potrebujejo podporo obeh platform. For example, Če potrebujemo le vzporedno obdelavo velikih podatkov in shranjevanje v HDF, then may be Hadoop alone is sufficient. Similarly, only for storing and retrieval of unstructured data, any NoSQL database and its associated query language can meet the requirement.
So the integration of NoSQL with Hadoop is always a preferred environment for large scale parallel processing and real time data access. Different Hadoop based products provide the integration of Hadoop and NoSQL in one platform. And this ‘in-Hadoop’ NoSQL database provides real time, operational analytics capabilities. Hadoop products including Apache Hadoop is the best fit for business critical production deployments. These products do not require any additional administrative tasks for the NoSQL data. The integrated platform (NoSQL and Apache Hadoop) supports high performance, extreme scalability, high availability, snapshots, disaster recovery, integrated security and many more, suitable for any production ready operational analytics.
So we can conclude that Apache Hadoop and NoSQL are not the same technology platform, but they are always recommended as an integrated environment suitable for big data solutions.
