[scottmcdot]

These days I usually prefer to use sqldf. It's so powerful how it let's you reference existing dataframes within the SQL command. It's like spark.sql in Python and temporary views but much more simple.