In October, it was announced that the Pandas API was being integrated with Spark. This was particularly exciting news for a Pandas-baby like myself, whose first exposure to data analytics were Pandas-based notebook tutorials. Spark 3.2 has been out for several months now and a curiosity has been building inside me – how easy it is to …

How Easy It Is to Re-use Old Pandas Code in Spark 3.2? Read More »