Databricks

Tonic supports the running of Spark jobs via Databricks on AWS.  To run Tonic on Databricks on Azure please reach out to [email protected]
Supported versions of Databricks
Tonic supports Spark 2.4.x and Spark 3+.  However, Spark 2.4.2 is not supported.  Any version of Databricks running one of the aforementioned Spark versions should be compatible, however, Tonic has only been tested against Databricks v7.4.
Supported providers
Tonic supports the following data providers:
Source Provider
Output Provider
Parquet
Parquet
CSV
Parquet
Avro
Avro
JSON
JSON
Delta
Delta
Supported table types
Databricks supports both MANAGED and EXTERNAL tables.
MANAGED tables store all of their data within Databricks.
EXTERNAL tables store their data on a separate file system (often S3). 
Tonic can read from both table types. When writing output data, Tonic only writes to EXTERNAL tables.
​

Redshift-specific configuration options

Initial cluster setup

Last modified 19d ago

Copy link

Contents

Supported versions of Databricks

Supported providers

Supported table types