Is Spark Still Relevant?

Eric Dill, Director of Data Science Platform at DTN and member of the conda-forge core team, joins James Bourbeau and Hugo Bowne-Anderson for a discussion comparing Dask, Spark, and RAPIDS for data science use cases. 00:00 We're live with Hugo, James, and Eric! 00:30 How did Eric get involved with Python and data science? 06:30 Data science: More than a technical concept 8:30 What is scalable computing? 11:30 What do businesses care about in the scalable computing ecosystem? 14:20 Spark is the new IBM 15:30 Dask companies enter the scene 17:30 Eric walks us through a schematic for the data orchestration ecosystem 27:00 Dask and where Coiled fits into the ecosystem 31:10 Business risk and picking "best-of-breed" tools 34:10 Ecosystem comparison: Spark, RAPIDS, and Dask 40:00 Dask and SQL 46:40 Interfaces: SQL vs DataFrame 54:00 Ecosystem as of January 2021 56:00 Data visualization and dashboarding 58:20 Bringing businesses into the data science revolution 1:01:06 Wrapping up! Thanks for watching. You can try out Coiled Cloud for free today here: http://cloud.coiled.io/