Saturday, December 3, 2022

Facebook, Uber, Twitter and Alibaba form Presto Foundation to Tackle Distributed Data Processing at Scale

By ET Bureau - September 24, 2019 2 Mins Read

The Linux Foundation, the nonprofit organization enabling mass innovation through open source, announced that Presto, the widely adopted interactive data query engine, will be hosted under the Linux Foundation. The newly-established Presto Foundation will have an open and neutral governance model that will enable Presto to scale and diversify its community. Founding members include Facebook, Alibaba, Twitter, and Uber, who will use the foundation to enable contributors to drive the future direction of the project, helping Presto become the fastest and most reliable SQL engine for massively distributed data processing.

Presto was developed at Facebook in 2012 as a high-performance distributed SQL query engine for large scale data analytics. Presto’s architecture allows users to query a variety of data sources such as Hadoop, S3, Alluxio, MySQL, PostgreSQL, Kafka, MongoDB and move at scale and speed. It solves the problem of having to choose between having fast analytics that uses an expensive commercial solution or using a slow but “free” solution that requires excessive hardware.

“Presto has been designed for high-performance exabyte-scale data processing on a large number of machines. Its flexible design allows processing data from a wide variety of data sources. From day one Presto has been designed with efficiency, scalability, and reliability in mind, and it has been improved over the years to take on additional use cases at Facebook, such as batch and other application-specific interactive use cases,” said Nezih Yigitbasi, Engineering Manager of Presto at Facebook.

“At Facebook alone, over a thousand employees use Presto, running several million queries and processing petabytes of data per day. After creating Presto we open-sourced it to see if other companies were having the same issues and wanted to collaborate. It turns out many other companies were interested and so under The Linux Foundation, we believe the project can engage others and grow the community for the benefit of all,” said Kathy Kam, Head of Open Source at Facebook.



AUTHOR

ET Bureau

The Enterprise talk Bureau has five well-trained writers and journalists, well versed in B2B enterprise technology industry, and constantly in touch with industry leaders for the latest trends, opinions, and other inputs- to bring you the best and latest in the domain.

Subscribe To Newsletter

*By clicking on the Submit button, you are agreeing with the Privacy Policy with Enterprise Talks.*