Function reference • dlt

Readers and writers

Functions for reading and writing SparkDataFrames (both batch and streaming) in Delta format.

dlt_write() write.delta() dlt_read() read.delta(): Write and read SparkDataFrame in delta format

dlt_write_stream() dlt_read_stream(): Write and read streaming SparkDataFrame in delta format

Delta table

Functions for loading, managing and updating DeltaTable objects.

DeltaTable-class: S4 class that represents DeltaTable

dlt_for_name(): Load DeltaTable from metastore

dlt_for_path(): Load DeltaTable from a path

dlt_alias(): Set alias for this DeltaTable

dlt_delete(): Delete data from this DataTable

dlt_update(): Update data in this DataTable

dlt_to_df(): Get SparkDataFrame corresponding to DeltaTable

dlt_show(): Print rows of the corresponding SparkDataFrame

dlt_history(): Get commit history for this DeltaTable

dlt_is_delta_table(): Check if path corresponds to a DeltaTable

dlt_convert_to_delta(): Create a DeltaTable from the given parquet table.

dlt_generate_manifest(): Generate manifest files for this DeltaTable

dlt_upgrade_table_protocol(): Upgrade protocol version of this DeltaTable

dlt_vacuum(): Vacuum this DeltaTable

dlt_restore_to_timestamp(): Restore the DeltaTable to an older version of the table specified by a timestamp.

dlt_restore_to_version(): Restore the DeltaTable to an older version of the table specified by version number.

dlt_detail(): Get the details of a Delta table such as the format, name, and size.

Delta merge builder

Functions for merging DeltaTable target with SparkDataFrame source.

DeltaMergeBuilder-class: S4 class that represents pending merge operation

dlt_merge(): Merge data from source with this DataTable, based on condition

dlt_execute(<DeltaMergeBuilder>): Execute merge operation on this builder

dlt_when_matched_delete(): Delete row from the table

dlt_when_matched_update(): Update rows for table using set rules

dlt_when_matched_update_all(): Update all columns of the matched table

dlt_when_not_matched_insert(): Insert a new row based the set of rules

dlt_when_not_matched_insert_all(): Insert a new row if not source not matched

Delta table builder

Functions for building new DeltaTable objects

DeltaTableBuilder-class: S4 class that represents pending table build operation

dlt_create() dlt_create_if_not_exists() dlt_replace() dlt_create_or_replace(): Initalize a new DeltaTableBuilder

dlt_execute(<DeltaTableBuilder>): Execute build operation on this builder

dlt_location(): Specify data storage location for the build table.

dlt_table_name(): Specify name of the build table.

dlt_add_column(): Add column to the build table

dlt_add_columns(): Add columns to the build table

dlt_comment(): Add comment describing the build table.

dlt_partitioned_by(): Specify partitioning columns

dlt_property(): Set property for the build table

Delta table optimize builder

Compose and execute OPTIMIZE.

DeltaOptimizeBuilder-class: S4 class that represents pending optimize operation

dlt_execute_compaction(): Compact the small files in selected partitions.

dlt_execute_z_order_by(): Z-Order the data in selected partitions using the given columns.

dlt_optimize(): Optimize the data layout of the table.

dlt_where(): Optimize the data layout of the table.

Generics

dlt_execute(): Execute pending operation on a builder