System requirements for Databricks

If you want to use a version or provider that Structural does not support, contact Tonic.ai.

Supported versions of Databricks

The following table lists the supported versions of Databricks, along with the corresponding version of Spark, based on this information in the Databricks documentation.

Databricks version	Spark version
14.3 LTS	3.5.0
13.3 LTS	3.4.1
12.2 LTS	3.3.2
11.3 LTS	3.3.0
10.4 LTS	3.2.1
9.1 LTS	3.1.2

Supported providers

Structural supports the following data providers:

Source Provider	Output Provider
Parquet	Parquet
CSV	Parquet
Avro	Avro
JSON	JSON
ORC	ORC
Delta	Delta

Supported table types

Databricks supports both MANAGED and EXTERNAL tables.

• MANAGED tables store all of their data within Databricks.

• EXTERNAL tables store their data on a separate file system (often Amazon S3).

Structural can read from both table types. When it writes output data, Structural only writes to EXTERNAL tables.

Must have clustering columns

Your Databricks tables must have clustering columns.