drop duplicate row from table

[kenn719]

Hi, do you have plan to include the feature to drop duplicate row from a table based on different column ? Example. Keep the first occurrence or last occurrence similar to python pandas ?

[lwhite1]

myTable.dropDuplicateRows() considers all the columns in the table.

…

On Fri, Aug 6, 2021 at 9:14 AM kenn719 ***@***.***> wrote: Hi, do you have plan to include the feature to drop duplicate row from a table based on different column ? Example. Keep the first occurrence or last occurrence similar to python pandas ? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#964>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2FPAUCEFPXNCKVTZWKWD3T3PN33ANCNFSM5BV4RBSQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[kenn719]

Hi
Thanks for the reply
But did the tablesaw come with the feature to only select certain column as the index for dropping the duplication rows?

[lwhite1]

no it doesn't. It would be nice to have, but for now you'd have to write your own method for that

…

On Fri, Aug 6, 2021 at 11:40 AM kenn719 ***@***.***> wrote: Hi Thanks for the reply But did the tablesaw come with the feature to only select certain column as the index for dropping the duplication rows? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#964 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2FPAUJCHMC6S4R2EDV4CTT3P66BANCNFSM5BV4RBSQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[kenn719]

Do you have the plan to include this feature in future ?

[lwhite1]

not currently. but now that there is an open issue for it, i will consider it in the future.

…

On Fri, Aug 6, 2021 at 12:16 PM kenn719 ***@***.***> wrote: Do you have the plan to include this feature in future ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#964 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2FPATI6XR77TDRZ5VKOMTT3QDEHANCNFSM5BV4RBSQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

[lwhite1]

Giving it a little more thought, i'm not sure it fits many use-cases. You are basically saying:

"Give me one random row for each unique combination of these column values Keep the other columns, but I don't care what values are in them."

It's basically a (non)random selection. You can do this really easily using Table.splitOn(my_four_column names). Roughly:

    Table c = myTable.emptyCopy();
    for (TableSlice slice : myTable.splitOn("a", "b", "c", "d")) {
        c.append(slice.first(1));
    }

Not super efficient that way, but easy enough to write.