Skip to content

Commit 9683479

Browse files
AmandaJ-WAmandaJ-W
authored
Update pull_data_kobotoolbox.md
1 parent a0ebf7f commit 9683479

File tree

1 file changed

+108
-35
lines changed

1 file changed

+108
-35
lines changed

‎source/pull_data_kobotoolbox.md‎

Lines changed: 108 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,37 +1,110 @@
1-
# Pull Data Functionality in KoboToolbox
1+
# Pulling data from an external CSV
22
**Last updated:** <a href="https://github.com/kobotoolbox/docs/blob/202f8e0e134d0695913bf6c5d5b52449c5e61e5d/source/pull_data_kobotoolbox.md" class="reference">18 Aug 2025</a>
33

4-
This is best done on xls version of the form.
5-
6-
- In the survey side of the form add a calculate field to your survey.
7-
- Give that field a name that you want
8-
- Then in its calculation column, call the pulldata() function, indicating
9-
which field to pull from which row of which .csv file. This can be achieved
10-
by writing as follows
11-
`pulldata(‘nameofcsv’, ‘columnheadingtopulldatafrom’, ‘columncheckformatchingTEXT’, ‘TEXTtoCheckfor’`
12-
13-
![image](/images/pull_data_kobotoolbox/xls.png)
14-
15-
- Note your CSV needs to have at-least two columns and ensure that the
16-
columntocheckformatchingTEXT is always the first column from the left
17-
- TexttoCheckfor can also be referenced from an earlier field question by
18-
using `${Question}` as an example above
19-
- Once you have finished updating the xls you will need to upload your form
20-
from xls (do not edit it on the formbuilder), you will then upload your CSV
21-
the same way you would upload your images.
22-
- When you deploy your file the csv will be downloaded to the media files
23-
24-
**Things to note**
25-
26-
- This will work both on KoboCollect app and Enketo (web form).
27-
- Compress a large .csv file into a .zip archive before uploading it.
28-
- Save .csv file in UTF-8 format if pre-loaded data contains non-English fonts
29-
or special characters this enables your Android device to render the text
30-
correctly.
31-
- Data fields pulled from a .csv file are considered to be text strings
32-
therefore use the int() or number() functions to convert a pre-loaded field
33-
into numeric form.
34-
- If the .csv file contains sensitive data that you may not want to upload to
35-
the server, upload a blank .csv file as part of your form, then replace it
36-
with the real .csv file by hand-copying the file onto each of your devices.
37-
- If you are facing error messages when using this feature, try changing the file name to remove any underscores or symbols.
4+
The `pulldata()` function in XLSForm lets you dynamically retrieve information from an external CSV file while completing a form. This allows you to reference existing datasets and automatically pull in related details, avoiding the need for enumerators to re-enter the same information.
5+
6+
For example, you can use `pulldata()` to:
7+
- **Auto-fill related information:** When an ID, code, or key is entered, automatically retrieve linked details such as a name, category, or description.
8+
- **Preload background data:** Load information from external files so enumerators only need to collect new or updated data.
9+
10+
Using `pulldata()` helps reduce errors, saves time during data collection, and ensures that forms remain consistent with external reference datasets. This function is supported in both the <a href="https://support.kobotoolbox.org/kobocollect_on_android_latest.html">KoboCollect Android app</a> and <a href="https://support.kobotoolbox.org/enketo.html">Enketo web forms</a>. We recommend using <a href="https://support.kobotoolbox.org/getting_started_xlsform.html">XLSForm</a> to set up the `pulldata()` function.
11+
12+
This article covers the following steps for pulling data from an external CSV:
13+
- Setting up your external CSV
14+
- Setting up your XLSForm
15+
- Uploading your external CSV to KoboToolbox
16+
17+
## Setting up your external CSV
18+
19+
To use `pulldata()`, first prepare an external CSV file containing the reference data you want to retrieve. Each row should represent a unique record (for example, a participant, location, or item) and the file should include at least two columns. One column must contain the **index variable** that matches the values entered in your form.
20+
21+
The index variable acts as the [primary key](https://en.wikipedia.org/wiki/Primary_key) that links your XLSForm to the external CSV. It should be a unique identifier that exists in both files, such as a participant ID, a district name, or another matching code.
22+
23+
The remaining columns can include any additional details you want to retrieve, such as names, categories, or descriptions. Ensure the CSV file is clean, consistently formatted, and saved with the `.csv` extension.
24+
25+
## Setting up your XLSForm
26+
27+
Once you have set up your external CSV, configure your XLSForm in the following way:
28+
29+
1. Ensure your XLSForm includes a question that serves as the **index variable**.
30+
2. Add a `calculate` field to your survey. Give the field a `name`.
31+
3. In the `calculation` column, use the **pulldata()** function to specify which field in the CSV to pull from. Use the following syntax: `pulldata(‘csv’,‘pull_from’, ‘csv_index’, ‘${survey_index}’)`.
32+
- `csv` is the name of the CSV file, without the extension.
33+
- `pull_from` refers to the column in your CSV file that contains the data you want to import into your form.
34+
- `csv_index` is the column in your CSV file that contains the **index variable.**
35+
- `survey_index` is the name of the question in your survey that contains the **index variable.**
36+
37+
**survey worksheet**
38+
39+
| type | name | label | calculation |
40+
|:-----------|:------------------|:-------------------------------------------|:-------------|
41+
| text | respondent_id | Respondent ID | |
42+
| calculate | eligibility_status | | pulldata('eligibility', 'status', 'ID', 'respondent_id') |
43+
| note | eligibility_not | Respondent is ${eligibility_status} for the study. | |
44+
| survey |
45+
46+
In the example above, the calculation retrieves the value from the `status` column of the `eligibility.csv` file, in the row where the `ID` in the CSV matches the ID entered in the `respondent_id` question of your form.
47+
48+
<p class="note">
49+
<strong>Note:</strong> After using the <code>pulldata()</code> function to retrieve external CSV data, you can reference that field in subsequent skip logic conditions, constraints, and labels, just like any other field or calculation.
50+
</p>
51+
52+
## Uploading your external CSV to KoboToolbox
53+
54+
The final step in linking your external CSV file to your form is uploading the file to KoboToolbox. To do this:
55+
1. Navigate to your project **SETTINGS**, and open the **Media** tab.
56+
2. Upload the CSV file(s) with the exact name you have used in your XLSForm.
57+
3. Deploy or redeploy the form.
58+
59+
![Upload media](images/pull_data_kobotoolbox/upload_media.png)
60+
61+
## Troubleshooting
62+
63+
<details>
64+
<summary><strong>Non-English fonts or special characters are not displaying correctly</strong></summary>
65+
Save your CSV file in <strong>UTF-8 format</strong>. This ensures that Android devices can render non-English text or special characters properly.
66+
</details>
67+
68+
<br>
69+
70+
<details>
71+
<summary><strong>Numeric values are not working as expected</strong></summary>
72+
All data pulled from a CSV file is treated as text. To use these values as numbers, apply the <code>int()</code> or <code>number()</code> <a href="https://support.kobotoolbox.org/functions_xls.html">functions</a> to the retrieved value in your XLSForm.
73+
</details>
74+
75+
<br>
76+
77+
<details>
78+
<summary><strong>Protecting sensitive data</strong></summary>
79+
If your CSV contains sensitive information that you do not want uploaded to the server, upload a blank CSV file with your form. Then manually replace it on each device with the real CSV file. This approach only works with the KoboCollect app.
80+
</details>
81+
82+
<br>
83+
84+
<details>
85+
<summary><strong>Slow form loading with large files</strong></summary>
86+
If you are using very large CSV files, you may experience slow form loading in KoboCollect. To resolve this, we recommend using the <a href="https://support.kobotoolbox.org/question_options_xls.html#additional-question-options">trigger</a> column to pull data from the external file once, rather than recurrently in the background.
87+
</details>
88+
89+
<br>
90+
91+
<details>
92+
<summary><strong>Pulling dates from external CSV files</strong></summary>
93+
If you are storing dates in an external CSV file and want to pull them into a form, ensure they are in the format YYYY-MM-DD. If you are editing your CSV in Excel, add a single quote (') in front of the date to avoid automatic date formatting in Excel.
94+
</details>
95+
96+
<br>
97+
98+
<details>
99+
<summary><strong>Pull data not working properly</strong></summary>
100+
If the pulldata() functionality is not working properly, try the following:
101+
<ul>
102+
<li>Rename your CSV file to remove underscores or special symbols.</li>
103+
<li>Check that your CSV file is adequately set up with one variable per column (see <a href="https://community.kobotoolbox.org/t/pulldata-is-not-working-on-kobocollect-android/6462/39">Community Forum</a> post).</li>
104+
<li>Check that you are using the exact spelling for file names and column names.</li>
105+
<li>Check that the cells in your CSV file do not include extra spaces before or after the value.</li>
106+
</ul>
107+
</details>
108+
109+
<br>
110+

0 commit comments

Comments
 (0)