Outgrowing your node? Zero stress scaling with cuPyNumeric.

🔊 Recorded at PyCon DE & PyData 2025, April 24, 2025 https://2025.pycon.de/program/HPGEKH/ 🎓 cuPyNumeric enables zero-code-change scaling of NumPy applications from laptops to supercomputers, transforming how researchers approach distributed scientific computing. Speakers: Bo Dong Description: In this presentation, Bo Dong from NVIDIA's CUDA team introduces cuPyNumeric, a scalable implementation of NumPy designed for distributed computing environments. The talk addresses the challenges researchers face when scaling NumPy applications from workstations to supercomputers, particularly the complexity of traditional MPI-based distributed programming. cuPyNumeric enables researchers to run the same NumPy code across multiple nodes and GPUs with minimal code modifications, eliminating the need for complex MPI implementations. The framework builds on the Legate runtime, a task-based HPC system that handles data partitioning and communication automatically. Dong demonstrates the practical impact through a case study at the SLAC National Accelerator Laboratory, where cuPyNumeric transformed the analysis of X-ray diffraction data. The implementation reduced processing time from over 10 hours to approximately 5 minutes, enabling real-time analysis of weak signals that were previously undetectable. The framework currently supports Linux environments and WSL2, with both CPU and GPU execution capabilities, though GPU optimization remains the primary focus. While the technology is open source under Apache 2 license, current development prioritizes NVIDIA GPU architectures. ⭐️ About PyCon DE & PyData: The PyCon DE & PyData conference unite the Python, AI, and data science communities, offering a unique platform for collaboration and innovation. The PyCon DE & PyData 2025 conference, provided an exceptional experience, fostering deeper connections within the Python community while showcasing advancements in AI and data science. Attendees enjoyed a diverse and engaging program, solidifying the event as a highlight for Python and AI enthusiasts nationwide. Follow us: • LinkedIn: https://www.linkedin.com/company/28908640/ • X: https://www.x.com/pyconde Links: • Conference website: http://pycon.de • Other sessions: https://2025.pycon.de/talks/ The conference is organized by • Python Softwareverband e.V.: http://pysv.org • NumFOCUS Inc.: http://numfocus.org • Pioneers Hub gemeinnützige GmbH: http://pioneershub.org If you enjoyed this session, please like, comment, and subscribe to our channel for more insightful talks and discussions. Share this video with your network to spread the knowledge! Hashtags: #Python #PyConDE #PyData #OpenSource #AI #DataScience #MachineLearning #SoftwareDevelopment #LLMs #Community Acknowledgements: Special thanks to all the volunteers and sponsors who made this event possible. About: Python Softwareverband e.V.: PySV is a non-profit that promotes the use and development of Python in Germany through events, education, and advocacy, fostering an open Python community. NumFOCUS Inc. supports open-source scientific computing by providing financial and logistical support to key projects like NumPy and Jupyter, promoting sustainable development and collaboration. Pioneers Hub gemeinnützige GmbH: is a non-profit fostering innovation in AI and tech by connecting experts and promoting knowledge exchange through events and collaborative initiatives.