Databricks Cloud handles the metadata, launching and provisioning a Spark Cluster, and makes it easy for that cluster to process an organization's data stored in Amazon's S3 service.

It will allow any developer to launch their own serverless cluster through a universal API that works anywhere. Databricks cofounder’s next act: Shining a Ray on serverless autoscaling | ZDNET X ...

Databricks, the commercial company created from the open source Apache Spark project, announced the release of a free Community Edition today aimed at teaching people how to use Spark — and as ...