All except simplification: Option 1

[Jolanrensen]

Fixes #761

This PR rewrites except to disallow pointing to nested columns. The columns selection DSL should only be allowed to "select" columns, not restructure it; we have move, remove, etc. for that.

When a user now calls, say

df.select { all() except userData.age }

they will receive an IllegalArgumentException:
Cannot exclude the nested columns '[userData/age]' from the column set '[userData/{age, name}, otherCol1, otherCol2]' with the except-functions. Use the `DataFrame<*>.remove { }` operation instead.
because userData.age is not directly selected by all(), only its parent, userdata, is.

TODO:

• Documentation website

---

Some internal struggle: It seems like except was used internally in a couple places and it relied on the old behavior.

Rewriting this now pushes me to this horrible notation:

Before:

df.groupBy { allExcept(columnSetWithNestedCols) }...

After:

df.groupBy {
    (this as DataFrame<*>)
        .remove(columnSetWithNestedCols)
        .columns()
        .toColumnSet()
}...

(something like df.remove {}.groupBy {} wouldn't not work, because you still want the columns inside the group)

Having to write something like this makes me rethink what we should do about this. Clearly there's a need for selecting the top-level columns of a version of the current dataframe where a (set of) column(s) and their parents are removed. Though, it seems out of the scope for except (and the 0.15 version of except even breaks in some cases). We could:

• Fix the bugs in except and keep it as is
  • (there are likely quite a few more we don't know about yet)
  • The ones we know can be fixed: All except fix: Option 2 #1038
• introduce a new function that has the old behavior with a good name
• Create a way to access the original DF in the selection dsl and promote the idea that you can use functions like remove on it. Probably also needs a df.toColumnSet() function.
• Merge this as is and just block except with nested columns
  • (We only use the concept 3 times after all)
• Something else

@zaleslaw @koperagen @nikitinas @belovrv WDYT?

[zaleslaw]

I believe that we should split our internallyUsedExcept and newOneExcept in our publicAPI

[Jolanrensen]

As discussed on Slack, we might actually need the ability to select column groups with some column removed, especially in groupBy, so it may be best to revert this, fix the NPE, and revert/stabilize #1030. Let's revisit it later

[Jolanrensen]

Let's go with Option 2