Houve um monte de hype em torno Hadoop por um longo tempo. Esta campanha publicitária era esperado porque Hadoop é percebida uma ferramenta de processamento de dados extremamente eficiente grande. Mas chegou a hora de olhar para alguns frio, fatos concretos. É o momento em que a campanha publicitária morre lentamente para baixo e empresas começam a olhar para o Retorno sobre o investimento,,en,Uma pesquisa,,en,realizado pela Gartner,,en,parece mostrar que muitas empresas não estão a planear investir em Hadoop, porque eles não têm as habilidades para usá-lo ou ele ainda é considerado uma ferramenta hostil pelo usuário,,en,Há outras razões também,,en,há um outro grupo de pessoas que estão otimistas sobre as perspectivas de Hadoop,,en,Se a situação parece confuso,,en,é porque a atitude para Hadoop está entrando na fase de desilusão da de hype e que é natural,,en,Este é o momento quando as empresas começam a ficar realista,,en,Este é o momento em que as empresas vão avaliar objetivamente e usar Hadoop,,en,Contrastando opiniões sobre adoção do Hadoop,,en,Como dito acima,,en (ROI). A survey, conducted by Gartner, seems to show that a lot of companies are not planning to invest in Hadoop because they do not have the skills to use it or it is still deemed a user-unfriendly tool. There are other reasons as well. However, there is another group of people who are bullish about the prospects of Hadoop. If the situation seems confusing, it is because the attitude towards Hadoop is entering the phase of disillusionment from that of hype and that is natural. This is the time when businesses start to get realistic. This is the time when companies will objectively evaluate and use Hadoop.
Contrasting opinions on Hadoop adoption
As stated above, as empresas estão divididas em dois grupos quando se trata de adoção do Hadoop,,en,um grupo compreende as empresas que são relutantes,,en,hesitante ou prudente quando se trata de adoção do Hadoop eo segundo grupo compreende empresas que acreditam Hadoop vai dar bom ROI,,en,A atitude do primeiro grupo se reflete na pesquisa Gartner,,en,Dada a seguir são as conclusões mais importantes da pesquisa,,en,Note-se que as conclusões do inquérito foram lançado em maio,,en,os resultados são muito atualizados,,en,O público-alvo levantamento abrangeu as empresas pequenas e médias e executivos de nível C,,en,dos entrevistados não pretende investir em Hadoop no futuro,,en,Somente,,en,dos entrevistados têm planos de investir em Hadoop nos próximos dois anos,,en,dos entrevistados estão implantando ou experimentando com Hadoop,,en: one group comprises enterprises that are reluctant, hesitant or circumspect when it comes to Hadoop adoption and the second group comprises enterprises that believe Hadoop is going to give good ROI. The attitude of the former group is reflected in the Gartner survey. Given below are the salient findings from the survey. Note that the survey findings were released in May, 2015. So, the results are pretty updated. The survey target audience comprised small and medium sized companies and C-level executives.
- 54% of the respondents do not plan to invest in Hadoop in the future.
- Just 18% of the respondents have plans to invest in Hadoop in the next two years.
- 26% of the respondents are deploying or experimenting with Hadoop.
- Empresas que estavam relutantes ou hesitante com a adoção do Hadoop citada escassez de competências e user-unfriendliness como razões para não pensar sobre Hadoop,,en,De acordo com a Merv Adrian,,en,vice-presidente do Gartner,,en,Com essa grande incidência de organizações sem planos ou já em sua jornada Hadoop,,en,demanda futura por Hadoop parece bastante anêmica longo de pelo menos a próxima,,en,Além disso,,en,a falta de planos de curto prazo para adoção do Hadoop sugere que,,en,apesar do entusiasmo de continuar para o fenômeno Big Data,,en,demanda por Hadoop especificamente não está acelerando,,en,As principais razões para tal uma resposta negativa para Hadoop são dadas abaixo,,en,Falta de habilidades Hadoop é uma restrição importante,,en,Empresas alegam que o seu pessoal não é capaz de usar Hadoop,,en,em sua forma original,,en.
According to Merv Adrian (vice president at Gartner), “With such large incidence of organizations with no plans or already on their Hadoop journey, future demand for Hadoop looks fairly anemic over at least the next 24 meses”. Moreover, the lack of near-term plans for Hadoop adoption suggests that, despite continuing enthusiasm for the big data phenomenon, demand for Hadoop specifically is not accelerating”. The main reasons for such a negative response to Hadoop are given below.
- Lack of Hadoop skills is an important constraint. Enterprises claim that their staff is not capable of using Hadoop. Hadoop, in its original form, has been largely confined to an exclusive group who could use it productively. Though a number of third-party tools are coming up to facilitate the use of Hadoop, even they are not easy to use. The main complaint against Hadoop and the third-party tools are they require new skills to be learnt which means additional investment. Existing skills cannot be used. Though training is available for these tools, experts believe that it will take another 2 to 3 years for these programs to gain credence.
- For many enterprises, Hadoop is not a priority. They think that Hadoop is overkill for the business problems it is supposed to solve. It is like deploying a missile to kill a group of flies. Also, the cost of Hadoop adoption is more than the benefit derived by solving the business problems enterprises are facing.
The second group is optimistic and confident about Hadoop adoption. There are companies that have started using Hadoop for their mainstream business and are reaping benefits. The main feature that is being used is real-time data processing. For example, companies can prevent fraud by analyzing data on a real-time basis. Companies are able to provide better products by analyzing on a real-time basis data feedback from their customers. They are receiving data from website-usage, video, Internet banking, social media and various other sources. In a survey conducted by TechValidate, 96% of the respondents are running several use cases on a single Hadoop cluster and among them, 20% are deploying nearly 50 use cases on a single Hadoop cluster. The survey revealed that 73% of the respondents are deploying their products and services and 59% are benefiting from reduced costs. Most of the above companies are MapR customers. According to Bryon Dover, a big data engineer with the Rubicon Project, “MapR gives me the reliability to process 3 trillion transactions a month with 99.999% uptime.”
What to make of the above findings?
The two extreme attitudes towards Hadoop adoption can be generally confusing but in the context of business cycle, this stage represents just another phase: that of enterprises leaving the hype stage and entering the evaluation stage. When there is hype, everything seems rosy but when there is evaluation, the disadvantages also come out. So, businesses are finding out how to best use Hadoop to solve their business problems or whether, Hadoop is at all required.
Hadoop needs to get over its exclusivity for sure because it is considered a difficult tool to use, available only to people who are specialized. There is no good front-end that makes it easy for people to process and analyze data. Also, there is a need to learn Hadoop and that needs additional investment. Third-party tools that claim to make Hadoop easy to use are not exactly living up to their claims. So, the whole offering needs modifications and it is going to take time. Basically, Hadoop needs to prove that it is easy to use.
Enterprises need to realize that one of the best uses of Hadoop is when you are processing data real time. That is where the second group of customers is reaping benefits. Batch processing is not where you should be focusing. Real time processing can enable you prevent frauds and offer customized products and services to your customers. Hadoop is not meant for static data. The image below shows that real-time usage is the biggest consideration for Hadoop.
In this context, it is pertinent to mention Apache Spark which has been doing a stellar job analyzing big data real time. It gives you a unified and comprehensive framework that helps you to manage huge data sets from variety of sources in real-time basis. The biggest advantage with Spark is that you can rapidly write applications in Python, Java or Scala and has more than 80 high-level operators. It also supports SQL queries, machine learning, streaming data and processing of graph data, other than Map and Reduce operations. Em poucas palavras, it can prove to be an effective real-time processing application.
Real time usage of Hadoop
Image1: Real time usage of Hadoop
Any adoption of a new technology takes time. Hype and adoption are different things. It is quite possible that a percentage of the 57% of the respondents of the Gartner survey who did not plan to invest in Hadoop may do so after some time as Hadoop enters the mainstream production stage of many companies and its benefits start showing. This is especially after new technologies of products start to make Hadoop more usable. The SQL on Hadoop, por exemplo, may be just the starting point of making Hadoop more accessible to a wider community.
Summary
The indifference towards Hadoop does not make it an unproductive tool. It is only that businesses are still unfamiliar with its ways. As MapR customers will corroborate, you need to identify how to best use Hadoop for solving your business problems. Using it for real-time data processing in the mainstream production appears to be the way to go. Similarly, there are other benefits too that still seem undiscovered. Of course, much around Hadoop and its ecosystem needs to change. It needs to be more accessible to anyone who wants to use it. The slow adoption rate of Hadoop could turn out into an acceptable proposition after 2 to 3 years.
