The workshop will take place on the 20th of April, 2021. Our program will include Keynotes by Marc Brooker and Guillaume Pierre. Here is our preliminary program schedule; note that the times are shown in Central European Time (Rennes, UTC+2); you can visit our Program page for a table with other time zones:

16:15 Welcome

Keynote Talks

16:30 Keynote 1 “Container orchestration in geo-distributed fog computing platforms” by Guillaume Pierre

17:00 Keynote 2 "AWS Lambda’s performance journey" by Marc Brooker

17:30 BREAK

Session 1: Applications and Machine Learning

17:40 Richard Bieringa, Abijith Radhakrishnan, Tavneet Singh, Sophie Vos, Jesse Donkervliet and Alexandru Iosup. An Empirical Evaluation of the Performance of Video Conferencing Systems.

17:55 Dheeraj Chahal and Mayank Mishra. Performance and Cost Comparison of Cloud Services for Deep Learning Workload.

18:10 Malte S. Kurz. Distributed Double Machine Learning with a Serverless Architecture.

18:25 Tim Hegeman, Matthijs Jansen, Alexandru Iosup and Animesh Trivedi. GradeML: Towards Holistic Performance Analysis for Machine Learning Workflows.

18:40 BREAK

Session 2: Performance in the Cloud

18:50 Luuk Klaver, Thijs van der Knaap, Johan van der Geest, Edwin Harmsma, Bram van der Waaij and Paolo Pileggi. Towards Independent Run-Time Cloud Monitoring.

19:05 Giulia Guidi, Marquita Ellis, Aydin Buluc, Katherine Yelick and David Culler. 10 Years Later: Cloud Computing is Closing the Performance Gap.

19:20 Yuxuan Zhao, Dmitry Duplyakin, Robert Ricci and Alexandru Uta. Cloud Performance Variability Prediction.

19:35 BREAK

19:45 Social Session with members of the SPEC RG Cloud group; open to all ICPE attendees. Bring your own drink/snacks!

20:15 END

The HotCloudPerf workshop proposes a meeting venue for academics and practitioners, from experts to trainees, in the field of cloud computing performance. The workshop aims to engage this community, and to lead to the development of new methodological aspects for gaining deeper understanding not only of cloud performance, but also of cloud operation and behavior, through diverse quantitative evaluation tools, including benchmarks, metrics, and workload generators. The workshop focuses on novel cloud properties such as elasticity, performance isolation, dependability, and other non-functional system properties, in addition to classical performance-related metrics such as response time, throughput, scalability, and efficiency.

Registration is now open and FREE! https://icpe2021.spec.org/participate/registration

The HotCloudPerf workshop is technically sponsored by the Standard Performance Evaluation Corporation (SPEC)’s Research Group (RG), and is organized annually by the RG Cloud Group. HotCloudPerf has emerged from the series of yearly meetings organized by the RG Cloud Group, since 2013. The RG Cloud Group group is taking a broad approach, relevant for both academia and industry, to cloud benchmarking, quantitative evaluation, and experimental analysis.

Past workshop editions: 2020, 2019 and 2018.

“Benchmarking in the Cloud”

Articles focusing on this topic are particularly encouraged for HotCloudPerf-2021.

1. Empirical performance studies in cloud computing environments, applications, and systems, including observation, measurement, and surveys.

2. Comparative performance studies and benchmarking of cloud environments, applications, and systems.

3. Performance analysis using modeling and queueing theory for cloud environments, applications, and systems.

4. Simulation-based studies for all aspects of cloud computing performance.

5. Tuning and auto-tuning of systems operating in cloud environments, e.g., auto-scaling of resources and auto-tiering of data, optimized resource deployment.

6. Software patterns and architectures for engineering cloud performance, e.g., serverless.

7. Experience with and analysis of performance of cloud deployment models, including IaaS/PaaS/SaaS/FaaS.

8. End-to-end performance engineering for pipelines and workflows in cloud environments, or of applications with non-trivial SLAs.

9. Tools for monitoring and studying cloud computing performance.

10. General and specific methods and methodologies for understanding and engineering cloud performance.

11. Serverless computing platforms and microservices in cloud datacenters.

January 20, 2021 Abstract due (informative; extended deadline)

January 25, 2021 Papers due (extended deadline)

February 11, 2021 Author Notification

February 22, 2021 Camera-ready deadline (ACM's firm camera ready deadline is February 24th)

April 20, 2021 Workshop Day

1. Full-papers (6 pages), plus 1 page for references

2. Short-papers (3 pages), plus 1 page for references

3. Talk only (1-2 pages, not included in the proceedings).

The format of the submissions should follow the ACM format of the companion conference, ICPE.

All presented papers will have a good amount of time allocated for Q&A plus feedback. In addition, presentation session will be wrapped-up by a 10-15 min discussion. All materials presented during the workshop will be made available, in widely used formats (i.e., pdf).

Please submit via EasyChair.

You can find our full Call For Papers (CFP) here.

Session 1: Applications and Machine Learning

• Richard Bieringa, Abijith Radhakrishnan, Tavneet Singh, Sophie Vos, Jesse Donkervliet and Alexandru Iosup. An Empirical Evaluation of the Performance of Video Conferencing Systems.

• Dheeraj Chahal and Mayank Mishra. Performance and Cost Comparison of Cloud Services for Deep Learning Workload.

• Malte S. Kurz. Distributed Double Machine Learning with a Serverless Architecture.

• Tim Hegeman, Matthijs Jansen, Alexandru Iosup and Animesh Trivedi. GradeML: Towards Holistic Performance Analysis for Machine Learning Workflows.

Session 2: Performance in the Cloud

• Luuk Klaver, Thijs van der Knaap, Johan van der Geest, Edwin Harmsma, Bram van der Waaij and Paolo Pileggi. Towards Independent Run-Time Cloud Monitoring.

• Giulia Guidi, Marquita Ellis, Aydin Buluc, Katherine Yelick and David Culler. 10 Years Later: Cloud Computing is Closing the Performance Gap.

• Yuxuan Zhao, Dmitry Duplyakin, Robert Ricci and Alexandru Uta. Cloud Performance Variability Prediction.