-
Notifications
You must be signed in to change notification settings - Fork 32
Open
Description
In order to submit an issue, please ensure you can check the following. Thanks!
- Declare which version of Python you are using (
python --version) - Declare which operating system you are using
tableschema-sql==0.10.4
tabulate==0.8.2
tabulator==1.19.0
dataflows==0.0.32
datapackage==1.5.1
datapackage-pipelines==2.0.0
when I run a pipeline including the following steps, excluding the string_to_int processor, I get a cast error at the end of the pipeline because the value 100000-15297008027871520 is not an integer. It looks like the values from the csv are strings, and I suspect the sum operation does += resulting in concatenation.
- run: load
parameters:
from: 'http://datastore.openspending.org/b9d2af843f3a7ca223eea07fb608e62a/estimates-of-provincial-expenditure-2018-19-uploaded-2019-05-01t1328/final/data/epre-2018-19.csv'
cache: true
# because otherwise string "addition" happens which looks like
# '7871520' + '9700802' + '-152' + '100000' = '100000-15297008027871520'
- run: string_to_int
parameters:
column_name: 'value'
- run: filter
parameters:
resources: epre-2018-19
in:
- finyear: '2018'
- run: delete_fields
parameters:
resources: epre-2018-19
fields:
- voteno
- econ5
- sprogno
- subprogramme
- run: join
parameters:
source:
name: epre-2018-19
key:
- "province"
- "department"
- "progno"
- "programme"
- "fg1"
- "fg2"
- "econ1"
- "econ2"
- "econ3"
- "econ4"
- "finyear"
- "budget_phase"
delete: yes
target:
name: epre-2018-19
key: null
fields:
"province": {}
"department": {}
"progno": {}
"programme": {}
"fg1": {}
"fg2": {}
"econ1": {}
"econ2": {}
"econ3": {}
"econ4": {}
"finyear": {}
"budget_phase": {}
"value":
aggregate: sum
full: true| ERROR :ROW: {'Amount Kind': 'Total', 'Budget Phase': 'Main appropriation', 'Department': 'Education', 'Econ1': 'Capital', 'Econ2': 'Payments for capital assets', 'Econ3': 'Machinery and equipment', 'Econ4': 'Other machinery and equipment', 'Financial Year': 2018, 'FunctionGroup1': 'Learning and culture', 'FunctionGroup2': 'Basic education', 'Government': 'Eastern Cape', 'Programme': 'Public Ordinary School Education', 'Programme Number': 2, 'Value': '100000-15297008027871520'}
Metadata
Metadata
Assignees
Labels
No labels
