Skip to main content

DSI faculty co-director Michael Franklin was one of a large group of researchers and open source developers who received the 2022 ACM SIGMOD Systems Award for their work on the popular Apache Spark system. The awards committee cited Spark as “an innovative, widely-used, open-source, unified data processing system encompassing relational, streaming, and machine-learning workloads”.

Franklin was an author of the original Spark paper in 2010 and contributed to the design of many of its main components, including the core processing engine, SQL interface, graph processing engine, and its libraries for simplifying large scale machine learning.

Spark, which was originally developed as a research project at UC Berkeley’s AMPLab, is a testament to the real-world impact that academic computer science research can have, especially in the context of a vibrant open-source community. According to a Databricks, Inc. blog post on the award, some recent data points regarding Apache Spark are:

  • It was downloaded 45 million times last month, in PyPI and Maven Central alone. This represents a 90% year-over-year growth in downloads.
  • It is used in at least 204 countries and regions.
  • It is ranked #1 in top paying technologies in Stack Overflow’s 2021 developer survey.