Skip to content

Add get getUpdatedPartitions and optimizeUpdatedPartitions #84

@zeotuan

Description

@zeotuan

add an API similar to what was implemented in mrpowers-io/levi#23 to get recently updated partitions
Also add an additional API to perform optimize on delta table using result from the above mentioned API since everyone will most likely perform similar operation to translate them to optimize condition

def getUpdatedPartitions(path: String, startTime: Option[Instant], endTime: Option[Instant]): Array[Map[String, String]]

def optimizeUpdatedPartition(path: String, startTime: Option[Instant], endTime: Option[Instant], zOrderCols: Option[Seq[String]] = None): DataFrame

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions