Skip to content

Spark 3.2 new functions #234

@eruizalo

Description

@eruizalo

Look into the following functions if they already are in doric or we have to add them:

  • def typedlit[T : TypeTag](literal: T): Column
  • def count_distinct(expr: Column, exprs: Column*): Column
  • def product(e: Column): Column
  • def sum_distinct(e: Column): Column ✅ covered in Feat/#60 implemented aggregate functions #123
  • def lag(e: Column, offset: Int, defaultValue: Any, ignoreNulls: Boolean): Column
  • def lead(e: Column, offset: Int, defaultValue: Any, ignoreNulls: Boolean): Column
  • def bitwise_not(e: Column): Column ⁉️ WIP in Non-aggregate functions #68
  • def shiftleft(e: Column, numBits: Int): Column ✅ covered in feat: [+] #66 math functions #223
  • def shiftright(e: Column, numBits: Int): Column ✅ covered in feat: [+] #66 math functions #223
  • def shiftrightunsigned(e: Column, numBits: Int): Column ✅ covered in feat: [+] #66 math functions #223
  • def sentences(string: Column, language: Column, country: Column): Column
  • def sentences(string: Column): Column
  • def next_day(date: Column, dayOfWeek: Column): Column
  • def session_window(timeColumn: Column, gapDuration: String): Column
  • def session_window(timeColumn: Column, gapDuration: Column): Column
  • def call_udf(udfName: String, cols: Column*): Column

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions