Lydia Chen: Dependability Management for Datacenters: A Machine Learning Perspective

TU Delft, Netherlands

The practice of collecting big performance data has changed how infrastructure providers model and manage the system in the past decade. There is a methodology shift from domain-knowledge based white-box models, e.g., queueing and simulation, to black-box machine learning models. At the same time, there is an ever-increasing number of applications based on (deep) machine learning models. In this talk, I will first show how such a game change affects the dependability management for major infrastructure providers, e.g., IBM. In the second part of the talk, I will focus on recent techniques, e.g., differential approximation, scheduling, and parallel tuning, that aim to strike an optimal tradeoff between the latency and accuracy for deep learning applications at a minimum cost.

Evgenia Smirni: Serverless Machine Learning Serving for Scalable Workflows

College of William & Mary, Virginia, US

Serverless computing is a new pay-per-use cloud service paradigm that automates resource scaling for stateless functions and can potentially facilitate bursty machine learning serving. Batching is critical for latency performance and cost-effectiveness of machine learning inference, but unfortunately it is not supported by existing serverless platforms due to their stateless design. Initial experiments show that without batching, machine learning serving cannot reap the benefits of serverless computing. In this talk, I will describe BATCH, a framework for supporting efficient machine learning serving on serverless platforms. BATCH uses an optimizer to provide inference tail latency guarantees and cost optimization and to enable adaptive batching support. We prototype BATCH atop of AWS Lambda and popular machine learning inference systems. The evaluation verifies the accuracy of the analytic optimizer illustrates its use for performance and cost advantages over the state-of-the-art.

Paul Brebner: Scaling Open Source Big Data Cloud Applications is Easy/Hard

Instaclustr, California, US

In the last decade, the development of modern horizontally scalable open-source Big Data technologies such as Apache Cassandra (for data storage), and Apache Kafka (for data streaming) enabled cost-effective, highly scalable, reliable, low-latency applications, and made these technologies increasingly ubiquitous. To enable reliable horizontal scalability, both Cassandra and Kafka utilize partitioning (for concurrency) and replication (for reliability and availability) across clustered servers. But building scalable applications isn’t as easy as just throwing more servers at the clusters, and unexpected speed humps are common. Consequently, you also need to understand the performance impact of partitions, replication, and clusters; monitor the correct metrics to have an end-to-end view of applications and clusters; conduct careful benchmarking, and scale and tune iteratively to take into account performance insights and optimizations. In this presentation, I will explore some of the performance goals, challenges, solutions, and results I discovered over the last 5 years building multiple realistic demonstration applications. The examples will include trade-offs with elastic Cassandra auto-scaling, scaling a Cassandra and Kafka anomaly detection application to 19 Billion checks per day, and building low-latency streaming data pipelines using Kafka Connect for multiple heterogeneous source and sink systems.