One of my 5 predictions for 2019 is about Hadoop. Basically, I do expect that a lot of projects won’t take Hadoop as a full-blown solution anymore. Why is that? Basically, one of the most exciting news in 2018 was the merger between Hortonworks and Cloudera. The two main competitors now joining forces? How can this happen? Basically, I do believe that a lot of that didn’t come out of a strength of the two and that they somehow started to “love” each other but rather out of economical calculations. Now, it isn’t a competition between Hortonworks or Cloudera anymore (even before the merger), it is rather Hadoop vs. new solutions. These solutions are highly diversified – Apache Spark is one of the top competitors to it. But there are also other platforms such as Apache Kafka and some NoSQL databases such as MongoDB, plus TensorFlow emerging. One would now argue that all of that is included in a Cloudera
As 2016 is around the corner, the question is what this year will bring for Big Data. Here are my top assumptions for the year to come: The growth for relational databases will slow down, as more companies will evaluate Hadoop as an alternative to classic rdbms The Hadoop stack will get more complicated, as more and more projects are added. It will almost take a team to understand what each of these projects does Spark will lead the market for handling data. It will change the entire ecosystem again. Cloud vendors will add more and more capability to their solutions to deal with the increasing demand for workloads in the cloud We will see a dramatic increase of successful use-cases with Hadoop, as the first projects come to a successful end What do you think about my predictions? Do you agree or disagree?