A few weeks ago we announced what we are working on at Stackable: A distribution for Open Source Big Data software that you can use on-premises and the cloud. In this post I’ll explain where we are coming from and why we are doing this.
2008 – Finding our niche
For me it all started in 2008 when I discovered Apache HBase. For those who don’t know what HBase is: It’s a “database” of sorts for Big Data use-cases modeled after the Google BigTable paper. I fell in love with HBase itself, the community and the Open Source movement.
From there I began to consult on Big Data projects, became a committer for HBase, Apache Hive and others. In 2016 my friends and colleagues Sönke Liebau and Lars George and I started a company called OpenCore with the same focus: Big Data consultancy.
We soon learned that we’re filling a niche with a strong demand. We focused on very short, very focused but high-impact projects. Come in, deploy, fix, train and leave but always be ready to help when needed. Our goal has always been to enable our customers to help themselves. This allowed us - as a very small team - to see and help a lot of different clients all over the world. It also meant that we cooperated with a lot of other consulting companies because our business models complement each other perfectly.
2019 - Cloudera & Hortonworks: How we became a community
This brings us to 2019 when Cloudera & Hortonworks completed their merger -and announced a huge change for their users: All products and services will only be available for paying customers. They also announced a price raise and a new cloud strategy. They also announced a price raise and a new cloud strategy. These changes came into effect at the beginning of 2021but right after the announcement were been contacted by a lot of our customers who were asking about alternatives. It became clear to us that there’s a need for action here! To cut a long story short: In December 2019 we ran the first of several Meet-ups with around 30 companies from Germany and got talking. The goal: Exchange, Knowledge transfer and establishing an Open Source community which still exists today.
2020 – How Stackable grew out of the community
These repeated meetings with like-minded people was (and still is) very valuable for everyone involved and has shown multiple challenges that everyone was facing:
- Flexibility: Technology moves fast so there is a strong desire to less strict solutions. People would like to update to new versions as soon as they are released and update components independently in a way that fits into running operations instead of multi-month projects.
- Security: Securing clusters is hard and complexity grows with the number of components. On top of this, in current architectures security is quite often implemented only as an afterthought, not right from the start. Our community desires a holistic security concept from Authentication, Authorization, Auditing to Governance etc.
- Infrastructure as Code: Talking about infrastructure our customers have the same overwhelming desire for simplicity and flexibility. They’d like to define their infrastructure as Code and not using a graphical interface.
- Price: Of course pricing plays a major role especially considering the changes announced by Cloudera. Pricing should reflect the flexibility and modularity requirements of the users.
2021 – The year we want to take off with Stackable
All of this feedback is what the Stackable platformis built on: Infrastructure-as-Code, Security-as-Code, modularity and flexibility with an easy integration into existing systems. We’re also building an Open-Source solution which has been proven to be a successful model. By founding Stackable in September 2020 Sönke and I have laid the foundations which we’re now building upon with the support of the community that brought us all these ideas and the initiative for Stackable. Thank you everyone involved!
We want to grow this community even further. If you want to be a part of it sign up for our Newsletter in order to not miss any news around Stackable. You’ll also soon learn when and how you can test our platform yourselves!