Skip to content

[ENH] Merge data allows matching by multiple pairs of columns #3919

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ajdapretnar merged 11 commits into from
Jul 30, 2019
Merged

[ENH] Merge data allows matching by multiple pairs of columns #3919

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
c12ba34
OWMergeData: Allow merging by multiple columns
janezd Jun 28, 2019
9852019
OWMergeData: Tests for uniqueness of combinations, not single columns
janezd Jul 1, 2019
969f043
OWMergeData: Fix report, other minor fixes
janezd Jul 1, 2019
1572677
OWMergData: Increase test coverage
janezd Jul 12, 2019
81b8db0
OWMergData: Fix reloading schema settings
janezd Jul 12, 2019
1fd20ba
OWMergeData: Sync combos if the attribute with the same name exists
janezd Jul 24, 2019
ba38e73
OWMergeData: In searching for the best match, don't assume metas are …
janezd Jul 24, 2019
4f6ac7d
OWMergeData: Improve gui, add tooltips
janezd Jul 24, 2019
30c9e67
OWMergeData: Improve test coverage
janezd Jul 24, 2019
2bc5b69
OWMergeData: Fix ids and duplicated columns in outer join
janezd Jul 27, 2019
690c9d1
Table: Properly iitialize missing values (empty strings, not nans for…
janezd Jul 30, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions Orange/data/table.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,8 @@ def from_table(cls, domain, source, row_indices=...):

global _conversion_cache

def get_columns(row_indices, src_cols, n_rows, dtype=np.float64, is_sparse=False):
def get_columns(row_indices, src_cols, n_rows, dtype=np.float64,
is_sparse=False, variables=[]):
if not len(src_cols):
if is_sparse:
return sp.csr_matrix((n_rows, 0), dtype=source.X.dtype)
Expand Down Expand Up @@ -331,7 +332,7 @@ def get_columns(row_indices, src_cols, n_rows, dtype=np.float64, is_sparse=False
shared_cache = _conversion_cache
for i, col in enumerate(src_cols):
if col is None:
a[:, i] = Unknown
a[:, i] = variables[i].Unknown
elif not isinstance(col, Integral):
if isinstance(col, SharedComputeValue):
if (id(col.compute_shared), id(source)) not in shared_cache:
Expand Down Expand Up @@ -394,19 +395,22 @@ def get_columns(row_indices, src_cols, n_rows, dtype=np.float64, is_sparse=False
self.domain = domain
conversion = domain.get_conversion(source.domain)
self.X = get_columns(row_indices, conversion.attributes, n_rows,
is_sparse=conversion.sparse_X)
is_sparse=conversion.sparse_X,
variables=domain.attributes)
if self.X.ndim == 1:
self.X = self.X.reshape(-1, len(self.domain.attributes))

self.Y = get_columns(row_indices, conversion.class_vars, n_rows,
is_sparse=conversion.sparse_Y)
is_sparse=conversion.sparse_Y,
variables=domain.class_vars)

dtype = np.float64
if any(isinstance(var, StringVariable) for var in domain.metas):
dtype = np.object
self.metas = get_columns(row_indices, conversion.metas,
n_rows, dtype,
is_sparse=conversion.sparse_metas)
is_sparse=conversion.sparse_metas,
variables=domain.metas)
if self.metas.ndim == 1:
self.metas = self.metas.reshape(-1, len(self.domain.metas))
if source.has_weights():
Expand Down
Loading