Lambda-in-a-Box: Merging Apache Spark & HBase into an Open-Source Database
This presentation will describe the architecture and implementation of Splice Machine V2.0, an open-source database that combines the benefits of modern lambda architectures with the full expressiveness of ANSI-SQL.
The system is powered by a sharded key-value store for fast short reads and writes, and short range scans (Apache HBase) and an in-memory, cluster data flow engine for analytics (Apache Spark). Like typical lambda architectures, it employs separate compute engines for different workloads - some call this an HTAP database (Hybrid Transactional and Analytical Platform).
It differs from most other clustered SQL systems such as Impala, SparkSQL, and Hive because it combines analytical processing with a distributed Multi-Value Concurrency Method that provides fine-grained concurrency which is required to power real-time applications.
This talk will highlight the Splice Machine storage representation, transaction engine, cost-based optimizer, and present the detailed execution of operational queries on HBase, and the detailed execution of analytical queries on Spark.
We will also compare the process of executing queries on Splice Machine versus other HTAP systems such as Apache Phoenix and Apache Trafodian, and discuss future items on the roadmap for development, including new row-based and column-based storage encodings.
As Vice President of Customer Solutions, Eric adds more than 20 years of enterprise software consulting and professional services management experience to the Splice Machine team.
Previously, Eric was Senior Director, Global Customer Services at Model N, a leading provider of revenue management solutions that can be deployed on-premise or in the cloud. Eric played a key role in building the organization from an early stage startup to a successful public company. Prior to Model N, Eric was one of the original employees at Blue Martini Software, where he served as Director, Professional Services and Analytic Services.
Prior to that, he held executive roles at Lumin Corporation, Om Records and Colorscape Communications. In addition to his extensive professional services management experience, Eric brings deep industry knowledge in the Life Sciences, High Tech, Telecom, Retail and Digital Media sectors. Eric holds dual BAs in Music and Religious Studies from Naropa University.