Remote persistent memory access API - The Second Approach

Tomasz Gromadzki, Intel, Corp.; Jan Michalski, Intel, Corp. Tomasz Gromadzki is a software architect at Intel Corporation’s Memory & Storage Product Group. His focus is on remote persistent memory access, which includes proper integration of persistent memory with other (networking) technologies as well as optimal persistent memory replication procedures and algorithms. Before joining Intel in 2018, for over 20 years, Tomasz designed, implemented, and deployed a variety of communication solutions for power distribution, industrial, and mining automation systems. He holds a Master of Science in Computer Science from the Gdansk University of Technology, Poland. Jan Michalski is a software engineer in Intel Corporation’s Memory & Storage Products Group. His focus is on remote persistent memory access, which includes proper integration of persistent memory with other technologies, as well as looking for optimal persistent-memory replication procedures and algorithms. He holds a master's degree in computer engineering from the Gdańsk University of Technology, Poland, where he studied system software engineering. PMDK team started working on combining RDMA with Persistent Memory many years ago because we've recognized the potential this combination had. Our initial approach, as seen in librpmem, was very conservative and focused on one use case: synchronous replication of data for libpmemobj pools. The result is that librpmem is great at that one specific workload, but we've since discovered that the APIs exposed by the library can be limiting for other things. That's the primary reason for why we've started on the path towards a new library, partially based on librpmem's implementation, that will have a generic enough interface to be useful for a wide variety of other workloads. A new API has been defined, a prototype has been created and is now reviewing with potential users. The presentation is going to demonstrate API itself but also reveals the major facts behind key design decisions.