The convergence of HPC, Big Data, and Deep Learning is becoming the next game-changing opportunity. Modern HPC systems and Cloud Computing platforms have been fueled with the advances in multi-/many-core architectures, Remote Direct Memory Access (RDMA) enabled high-speed networks, persistent memory (PMEM), and NVMe-SSDs. However, many Big Data systems and libraries (such as Hadoop, Spark, Flink, Memcached) have not embraced such technologies fully. Recent studies have shown that default designs of these components can not efficiently leverage the advanced features on modern clusters with RDMA, PMEM, and NVMe-SSD. In this tutorial, we will provide an in-depth overview of the architectures, programming models, features, and performance characteristics of RDMA networks, PMEM, and NVMe-SSD. We will examine the challenges in re-/co-designing communication and I/O components of Big Data systems and libraries with these emerging technologies. We will provide benchmark-level studies and system-level (like Hadoop/Spark/TensorFlow/Memcached) case studies to discuss how to efficiently use these new technologies for real applications.

Computer architecture research has largely employed detailed full-system simulation with real-world workloads, however, the very large simulation times taken by this methodology has started to prohibit good design space exploration. Our ongoing research has come up with successful techniques to characterize benchmarks, and synthesize or clone benchmarks into miniaturized code sequences with approximately the same performance and power behavior as the original workload. This tutorial will present the proxy generation methodology, proxies for SPEC CPU 2017 benchmarks, and proxies for Cassandra, MongoDB, and MySQL. It will also present SimPoints for SPEC CPU 2017 and their pinballs. The use of miniaturized proxies for reproducible research will be examined.

There have been many notable advances in compiler and runtime technologies to efficiently map applications on to multiple generations of parallel hardware, ranging from symmetric multiprocessors to distributed-memory multiprocessors, and from homogeneous to heterogeneous computing with accelerators. However, for the most part, compilers and runtime systems still have little knowledge of the characteristics of the applications that they process. The benefits of profile-directed optimization, as well as a number of predictive techniques (including recent approaches based on deep learning), have only begun to scratch the surface of how compilers and runtime systems can adapt to the characteristics of the applications that they process. Domain-specific programming systems are notable in their use of domain characteristics by compiler and runtime systems, but they typically account for small fractions of entire workloads.
In this talk, we make a case for using more comprehensive workload characterizations to drive future advances in compilers and runtime systems. We will draw from recent research experiences with 1) the MAESTRO analytical cost model that leverages compiler techniques to efficiently explore a large space of mappings of DNNs on spatial accelerators, 2) the use of data-aware code transformations to optimize data analytics kernels such as Word2Vec, and 3) the use of the Habanero-C runtime system to efficiently execute HPC applications that require large volumes of small irregular asynchronous data transfers. We envision a future in which workload characterizations are regularly used as first-class inputs in the design and implementation of compiler and runtime systems. Modern workloads like DNNs provide a great starting point for such a direction because their characteristics are well known ahead of time. However, we expect workload-aware compiler and runtime techniques to also be beneficial for more general-purpose programs in the future.

The fields of computer and information science and engineering (CISE) are central to nearly all of society’s needs, opportunities, and challenges. The US National Science Foundation (NSF) was created 70 years ago with a broad mission to promote the progress of science and to catalyze societal and economic benefits. NSF, largely through its CISE directorate which has an annual budget of more than $1B, accounts for over 85% of federally-funded, academic, fundamental computer science research in the US. My talk will give an overview of NSF/CISE research, education, diversity and inclusion efforts, and research infrastructure programs. I will relate them to the technical and societal trends and topics that will impact their future trajectory. I will particularly highlight opportunity areas most in need of the engagement and insights from IISWC researchers going forward.