Skip to contents

Readers and writers

Functions for reading and writing SparkDataFrames (both batch and streaming) in Delta format.

dlt_write() write.delta() dlt_read() read.delta()
Write and read SparkDataFrame in delta format
dlt_write_stream() dlt_read_stream()
Write and read streaming SparkDataFrame in delta format

Delta table

Functions for loading, managing and updating DeltaTable objects.

DeltaTable-class
S4 class that represents DeltaTable
dlt_for_name()
Load DeltaTable from metastore
dlt_for_path()
Load DeltaTable from a path
dlt_alias()
Set alias for this DeltaTable
dlt_delete()
Delete data from this DataTable
dlt_update()
Update data in this DataTable
dlt_to_df()
Get SparkDataFrame corresponding to DeltaTable
dlt_show()
Print rows of the corresponding SparkDataFrame
dlt_history()
Get commit history for this DeltaTable
dlt_is_delta_table()
Check if path corresponds to a DeltaTable
dlt_convert_to_delta()
Create a DeltaTable from the given parquet table.
dlt_generate_manifest()
Generate manifest files for this DeltaTable
dlt_upgrade_table_protocol()
Upgrade protocol version of this DeltaTable
dlt_vacuum()
Vacuum this DeltaTable
dlt_restore_to_timestamp()
Restore the DeltaTable to an older version of the table specified by a timestamp.
dlt_restore_to_version()
Restore the DeltaTable to an older version of the table specified by version number.
dlt_detail()
Get the details of a Delta table such as the format, name, and size.

Delta merge builder

Functions for merging DeltaTable target with SparkDataFrame source.

DeltaMergeBuilder-class
S4 class that represents pending merge operation
dlt_merge()
Merge data from source with this DataTable, based on condition
dlt_execute(<DeltaMergeBuilder>)
Execute merge operation on this builder
dlt_when_matched_delete()
Delete row from the table
dlt_when_matched_update()
Update rows for table using set rules
dlt_when_matched_update_all()
Update all columns of the matched table
dlt_when_not_matched_insert()
Insert a new row based the set of rules
dlt_when_not_matched_insert_all()
Insert a new row if not source not matched

Delta table builder

Functions for building new DeltaTable objects

DeltaTableBuilder-class
S4 class that represents pending table build operation
dlt_create() dlt_create_if_not_exists() dlt_replace() dlt_create_or_replace()
Initalize a new DeltaTableBuilder
dlt_execute(<DeltaTableBuilder>)
Execute build operation on this builder
dlt_location()
Specify data storage location for the build table.
dlt_table_name()
Specify name of the build table.
dlt_add_column()
Add column to the build table
dlt_add_columns()
Add columns to the build table
dlt_comment()
Add comment describing the build table.
dlt_partitioned_by()
Specify partitioning columns
dlt_property()
Set property for the build table

Delta table optimize builder

Compose and execute OPTIMIZE.

DeltaOptimizeBuilder-class
S4 class that represents pending optimize operation
dlt_execute_compaction()
Compact the small files in selected partitions.
dlt_execute_z_order_by()
Z-Order the data in selected partitions using the given columns.
dlt_optimize()
Optimize the data layout of the table.
dlt_where()
Optimize the data layout of the table.

Generics

dlt_execute()
Execute pending operation on a builder