Shattering the Ephemeral Storage Cost Barrier for Data-Intensive Serverless Workflows
Serverless computing enables developers to deploy applications as work ows of functions that invoke one another, with cloud providers handling autoscaling and routing. However, serverless platforms lack e cient mechanisms for cross-function data transfers, which hinders the performance of data-intensive applications. Current solutions rely on intermediary services like AWS S3 or ElastiCache(EC), leading to signi cant cost ine ciencies-storage costs can account for 24-99% of the total execution bill. Zipline addresses this challenge with a fast, API-compatible data communication method enabling direct function-to-function transfers. Zipline bu ers data in the sender function's memory and transmits only the references to the dynamically selected receiver, which pulls the data directly from the sender's memory. While eliminating the need for intermediary services, it also integrates seamlessly with existing autoscaling infrastructure, preserving function invocation semantics while signi cantly reducing costs and latency. In a vHive/Knative prototype on AWS EC2, Zipline achieves 2-5× lower costs & 1.3-3.4× faster execution times compared to S3. Against EC, Zipline cuts costs by 17-772× while improving performance by 2-5%. Zipline demonstrates a cost-e ective and high-performance solution for data-intensive serverless applications. CCS Concepts • Computer systems organization → Cloud computing.
3721465.3721866.pdf
Main Document
openaccess
CC BY
699.03 KB
Adobe PDF
3a71137195525ca9587e05d374ccb934