Skip to content

Conversation

@pwalsh
Copy link
Contributor

@pwalsh pwalsh commented Sep 17, 2021

TODO:

  • don't do some stuff when from_key == to_key['name']
  • support other options on the to_key field spec? Current just name and type
  • docs in PROCESSORS.md

@coveralls
Copy link

coveralls commented Sep 17, 2021

Pull Request Test Coverage Report for Build 561

  • 28 of 29 (96.55%) changed or added relevant lines in 2 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.2%) to 85.081%

Changes Missing Coverage Covered Lines Changed/Added Lines %
dataflows/processors/unwind.py 27 28 96.43%
Totals Coverage Status
Change from base Build 559: 0.2%
Covered Lines: 2207
Relevant Lines: 2594

💛 - Coveralls

Copy link
Member

@akariv akariv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a few comments - also don't forget to update PROCESSORS.md


def func(package):
matcher = ResourceMatcher(resources, package.pkg)
yield package.pkg
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before yielding here, you can process the datapackage itself and remove the from_key field if source_delete and from_key != to_key.

from dataflows.helpers.resource_matcher import ResourceMatcher


def unwind(from_key, to_key, transformer=None, resources=None, source_delete=True):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should also add here the 'to_key' field (with its type and other options)
(or - if from_key == to_key - update the field's type accordingly)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants