SAP is betting the company on their next generation HANA technology and it is interesting to watch as HANA evolves. Gartner analysts Massimo Pezzini and Daniel Sholler recently published a report entitled SAP Throws Down the Next-Generation Architecture Gauntlet With HANA, which is definitely worth a read. Some of their key findings which I found interesting were:
- “SAP application infrastructure and architecture strategy, dubbed HANA Architecture, combines cloud and in-memory technologies in a progressive, and potentially disruptive, vision.”
- “The HANA Architecture is a work in progress, and will undergo several significant changes before it is completed.”
- “SAP’s vision will force megavendor competitors to respond by clearly and openly articulating their cloud and in-memory computing strategies.”
One thing that I still find missing around the whole discussion of the in-memory approach of HANA is why it is possible. While it is great to hold all the data in memory so as to be able to quickly access it, the key concept that is making this possible is the fact they store data by columns rather than by rows. Storing data by columns is a concept that has been around for a long time but was never widely implemented because traditional databases were OLTP and hence were optimized for writing data. Today’s analytical databases (OLAP) need to be optimized for reading data and hence the column store makes sense. Optimizing a database for read access and storing data by column allows you to significantly compress the data. When the data is compressed, it becomes much easier to pull it into memory – basically, you don’t need as much memory. This allows you to actually have a much more affordable in-memory system or if you have a lot of money, to have an in-memory system that holds a large amount of data. Once the data is in memory, then your analytical engine can run much faster and presto, you have a speed of thought analytics/OLAP system.
Companies like Teradata and Vertica are also following strongly with the column store. While Vertica made a lot of noise about the column store and was acquired by HP, Teradata has not been saying as much. I have heard from people at Teradata like Jeff Carter and Rich Charucki that they have been able to engineer the column store technology into the existing Teradata engine so the user can choose to store data either row-oriented or column-oriented depending on how they want to access it. This is a great concept and I look forward to hearing more in the future. In the meantime, read the Gartner report on SAP HANA and consider how this will cause the industry to change.