Errol and I had the idea of doing a “Data Hike” back in 2020. However, Covid Restrictions didn’t allow us to meet. So, both of us ended up hiking on our own. Errol the beautiful lakes of Sweden, me the beautiful alpine lakes in Austria. However, things get easier now and so we finally made it last weekend!
Our hike took us to the beautiful and scenic area of Semmering in Lower Austria. This place is not just known for its beautiful scenery, but also for an area where the Austrian emperors often went to. The hike took us through forests, alms and even a small summit (Großer Pfaff). We touched different topics around data and one’s rather remote to it.
The Data Mesh Hike
One of the key topics we talked about was the “Data Mesh”. This trend started in 2019 by ((LINK)) is getting popular fast. Both Errol and I had our ideas of it and how to approach it. We were both in-line with the idea that it isn’t just a trend but something that will ease up all our lives and how we deal with data. We talked about some companies we both know that are already thinking into that direction. Also, we highlighted the need for more know-how on this topic, since it is a new and emerging topic.
The Datalake
We didn’t really pass by a lake during our hike – only at a swamp. This was a “signature” for what the datalakes eventually had become: data swamps. We discussed how this appeared and why it ended like this; we argued that most likely it was because companies focused on implementing a complex technology and fully ignored the fact that a good governance is necessary. All tech geeks discussed the fancy technology around the Datalake – e.g. Spark, Hive, Kafka, … but nobody wanted to “pitch” for governance. Our final thoughts were: the datalake is death. However, we both agreed that it is only about the legacy technology, not the concept of it.
Heads up in the Cloud
Once we reached the top of “Großer Paff”, we had a marvelous view over the area. This led us to another topic: The Cloud. In the Cloud, you reach another level of maturity by using Microservices and alike. It isn’t necessary anymore to build a big Datalake. It is already existing and available. Using services such as S3 is easy and straight forward. There is no need for a complex HDFS-based solution.
It’s the business, stupid!
After a 4 and a half hours hike, we reached our car and rewarded ourselves with nice Topfenknödel. I made my pitch for “Kasnocken” but it isn’t served in this area of Austria. When we asked ourselves, why we did this hike, the answer was simple: Data needs to serve a business need. There is no need to go for complex technology, keep it simple (with Microservices) and deliver value!
If you are also interested in joining a hike with us, hit me up! One thing is clear to us: this was just the beginning of a series of Data Hikes (or maybe even Data Ski’s?)