Skip to content

Commit 8ffa743

Browse files
Adding Portal Project dataset for 2023-05-02 (#556)
* Adding dataset for 2023-04-25 * Delete duplicate tt_data_type.csv * Update data/2023/2023-04-25/post_vars.json * adding Portal Project data for 2023-05-02 * fixing data cleaning section * adding link in data cleaning section * Update data/2023/2023-05-02/readme.md * Update data/2023/2023-05-02/readme.md * Update data/2023/2023-05-02/readme.md * Update data/2023/2023-05-02/readme.md * Update data/2023/readme.md * Update README.md * Update data/2023/2023-05-02/readme.md --------- Co-authored-by: Jon Harmon <[email protected]>
1 parent 1dcf45e commit 8ffa743

File tree

10 files changed

+28631
-0
lines changed

10 files changed

+28631
-0
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -139,6 +139,7 @@ Need a **reminder**? There are [extensions](https://chrome.google.com/webstore/d
139139
| 15 | `2023-04-11` | [US Egg Production Data](data/2023/2023-04-11/readme.md) | [US Egg Production Data 2007-2021](https://osf.io/z2gxn/) | [The Humane League Labs US Egg Production Dataset](https://thehumaneleague.org/article/E008R01-us-egg-production-data) |
140140
| 16 | `2023-04-18` | [Neolithic Founder Crops](data/2023/2023-04-18/readme.md) | [The "Neolithic Founder Crops"" in Southwest Asia: Research Compendium](https://github.com/joeroe/SWAsiaNeolithicFounderCrops/) | [Revisiting the concept of the 'Neolithic Founder Crops' in southwest Asia](https://link.springer.com/article/10.1007/s00334-023-00917-1) |
141141
| 17 | `2023-04-25` | [London Marathon](data/2023/2023-04-25/readme.md) | [London Marathon R package](https://github.com/nrennie/LondonMarathon) | [Scraping London Marathon data with {rvest}](https://nrennie.rbind.io/blog/web-scraping-rvest-london-marathon/) |
142+
| 18 | `2023-05-02` | [The Portal Project](data/2023/2023-05-02/readme.md) | [Portal Project Data](https://github.com/weecology/portaldata) | [Portal Project](https://portal.weecology.org/) |
142143

143144
***
144145

data/2023/2023-05-02/pic1.png

3.16 MB
Loading

data/2023/2023-05-02/pic2.png

351 KB
Loading

data/2023/2023-05-02/plots.csv

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
"plot","treatment"
2+
"3","exclosure"
3+
"4","control"
4+
"11","control"
5+
"14","control"
6+
"15","exclosure"
7+
"17","control"
8+
"19","exclosure"
9+
"21","exclosure"
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"article_link": "https://www.biorxiv.org/content/10.1101/332783v3.full",
3+
"pic1_alt": "Layout of the experimental plots and sampling locations. A: Location of all 24 plots (numbered and outlined in white). Blue flag marks the weather station location. A fence to exclude cattle surrounds the entire site (blue line). The outline for plot 24 also shows how, from 1995 to 2016, the northeast corner of plot 24 was cut off, excluding 2 rodent stakes and one plant quadrat. B: Permanent rodent trapping locations (‘stakes’, purple diamonds) shown for plot 2. C: Permanent plant sampling locations. Quadrats for counting abundance (blue triangles) are interlaced with the rodent trapping locations (purple diamonds). Transects for assessing perennial cover (yellow lines) span the plot diagonally from corner to corner (yellow circles).",
4+
"pic2_alt": "Rodent treatment assignments on plots (y-axis) over time (x-axis). All plots were controls during an initial burn-in period, before treatments were applied. Long-term plots (those which have not had their treatments altered since the original assignment) are labeled in bold and underlined. They are plots 3, 4, 10, 11, 14, 15, 16, 17, 19, 21, 23. Plots have large gates to allow access to all rodents (“control”), medium gates to allow access to all rodents but the largest species of kangaroo rat, the Banner-tailed kangaroo rat (“D. spectabilis exclosure”), small gates to exclude all species of Dipodomys (the dominant genus) but allow access to all other rodents (“k-rat exclosure”), or no gates to exclude all species of rodent (“total rodent exclosure;” any rodents trapped on these plots are removed from the site)."
5+
}

data/2023/2023-05-02/readme.md

Lines changed: 225 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
---
2+
editor_options:
3+
markdown:
4+
wrap: 72
5+
---
6+
7+
### Please add alt text to your posts
8+
9+
Please add alt text (alternative text) to all of your posted graphics
10+
for `#TidyTuesday`.
11+
12+
Twitter provides
13+
[guidelines](https://help.twitter.com/en/using-twitter/picture-descriptions)
14+
for how to add alt text to your images.
15+
16+
The DataViz Society/Nightingale by way of Amy Cesal has an
17+
[article](https://medium.com/nightingale/writing-alt-text-for-data-visualization-2a218ef43f81)
18+
on writing *good* alt text for plots/graphs.
19+
20+
> Here's a simple formula for writing alt text for data visualization:
21+
> \### Chart type It's helpful for people with partial sight to know
22+
> what chart type it is and gives context for understanding the rest of
23+
> the visual. Example: Line graph \### Type of data What data is
24+
> included in the chart? The x and y axis labels may help you figure
25+
> this out. Example: number of bananas sold per day in the last year
26+
> \### Reason for including the chart Think about why you're including
27+
> this visual. What does it show that's meaningful. There should be a
28+
> point to every visual and you should tell people what to look for.
29+
> Example: the winter months have more banana sales \### Link to data or
30+
> source Don't include this in your alt text, but it should be included
31+
> somewhere in the surrounding text. People should be able to click on a
32+
> link to view the source data or dig further into the visual. This
33+
> provides transparency about your source and lets people explore the
34+
> data. Example: Data from the USDA
35+
36+
Penn State has an
37+
[article](https://accessibility.psu.edu/images/charts/) on writing alt
38+
text descriptions for charts and tables.
39+
40+
> Charts, graphs and maps use visuals to convey complex images to users.
41+
> But since they are images, these media provide serious accessibility
42+
> issues to colorblind users and users of screen readers. See the
43+
> [examples on this page](https://accessibility.psu.edu/images/charts/)
44+
> for details on how to make charts more accessible.
45+
46+
The `{rtweet}` package includes the [ability to post
47+
tweets](https://docs.ropensci.org/rtweet/reference/post_tweet.html) with
48+
alt text programatically.
49+
50+
Need a **reminder**? There are
51+
[extensions](https://chrome.google.com/webstore/detail/twitter-required-alt-text/fpjlpckbikddocimpfcgaldjghimjiik/related)
52+
that force you to remember to add Alt Text to Tweets with media.
53+
54+
# The Portal Project
55+
56+
The data this week comes from the [Portal
57+
Project](https://portal.weecology.org/). This is a long-term ecological
58+
research site studying the dynamics of desert rodents, plants, ants and
59+
weather in Arizona.
60+
61+
> The Portal Project is a long-term ecological study being conducted
62+
> near Portal, AZ. Since 1977, the site has been used to study the
63+
> interactions among rodents, ants and plants and their respective
64+
> responses to climate. To study the interactions among organisms, they
65+
> experimentally manipulate access to 24 study plots. This study has
66+
> produced over 100 scientific papers and is one of the longest running
67+
> ecological studies in the U.S.
68+
69+
The [Weecology research group](https://www.weecology.org/) monitors
70+
rodents, plants, ants, and weather. All data from the Portal Project are
71+
made openly available in near real-time so that they can provide the
72+
maximum benefit to scientific research and outreach. The core dataset is
73+
managed using an automated living data workflow run using GitHub and
74+
Continuous Analysis.
75+
76+
This dataset focuses on the rodent data. Full data is available through
77+
these resources:
78+
79+
- [GitHub Data Repository](https://github.com/weecology/PortalData)
80+
- [Live Updating Zenodo
81+
Archive](https://doi.org/10.5281/zenodo.1215988)
82+
- [Data
83+
Paper](https://www.biorxiv.org/content/early/2018/05/28/332783)
84+
- [Methods
85+
Documentation](https://github.com/weecology/PortalData/blob/master/SiteandMethods/Methods.md)
86+
87+
The Portal Project data can also be accessed through the Data Retriever,
88+
a package manager for data.
89+
90+
[Data Retriever](https://www.data-retriever.org/)
91+
92+
A teaching focused version of the dataset is also maintained with some
93+
of the complexities of the data removed to make it easy to use for
94+
computational training purposes. This dataset serves as the core dataset
95+
for the [Data Carpentry
96+
Ecology](https://datacarpentry.org/ecology-workshop/) material and has
97+
been downloaded almost 50,000 times.
98+
99+
### Get the data here
100+
101+
```{r}
102+
# Get the Data
103+
104+
# Read in with tidytuesdayR package
105+
# Install from CRAN via: install.packages("tidytuesdayR")
106+
# This loads the readme and all the datasets for the week of interest
107+
108+
# Either ISO-8601 date or year/week works!
109+
110+
tuesdata <- tidytuesdayR::tt_load('2023-05-02')
111+
tuesdata <- tidytuesdayR::tt_load(2023, week = 18)
112+
113+
plots <- tuesdata$plots
114+
species <- tuesdata$species
115+
surveys <- tuesdata$surveys
116+
117+
118+
# Or read in the data manually
119+
120+
plots <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2023/2023-05-02/plots.csv')
121+
species <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2023/2023-05-02/species.csv')
122+
surveys <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2023/2023-05-02/surveys.csv')
123+
124+
125+
```
126+
127+
### Data Dictionary
128+
129+
# `plots.csv`
130+
131+
| variable | class | description |
132+
|:----------|:----------|:---------------|
133+
| plot | double | Plot number |
134+
| treatment | character | Treatment type |
135+
136+
# `species.csv`
137+
138+
| variable | class | description |
139+
|:---------------|:----------|:------------------------|
140+
| species | character | Species |
141+
| scientificname | character | Scientific Name |
142+
| taxa | character | Taxa |
143+
| commonname | character | Common Name |
144+
| censustarget | double | Target species (0 or 1) |
145+
| unidentified | double | Unidentified (0 or 1) |
146+
| rodent | double | Rodent (0 or 1) |
147+
| granivore | double | Granivore (0 or 1) |
148+
| minhfl | double | Minimum hindfoot length |
149+
| meanhfl | double | Mean hindfoot length |
150+
| maxhfl | double | Maximum hindfoot length |
151+
| minwgt | double | Minimum weight |
152+
| meanwgt | double | Mean weight |
153+
| maxwgt | double | Maximum weight |
154+
| juvwgt | double | Juvenile weight |
155+
156+
# `surveys.csv`
157+
158+
| variable | class | description |
159+
|:-----------------------|:---------------------|:-------------------------|
160+
| censusdate | double | Census date |
161+
| month | double | Month |
162+
| day | double | Day |
163+
| year | double | Year |
164+
| treatment | character | Treatment type |
165+
| plot | double | Plot number |
166+
| stake | double | Stake number |
167+
| species | character | Species code |
168+
| sex | character | Sex |
169+
| reprod | character | Reproductive condition |
170+
| age | character | Age |
171+
| testes | character | Testes (Scrotal, Recent, or Minor) |
172+
| vagina | character | Vagina (Swollen, Plugged, or Both) |
173+
| pregnant | character | Pregnant |
174+
| nipples | character | Nipples (Enlarged, Swollen, or Both) |
175+
| lactation | character | Lactating |
176+
| hfl | double | Hindfoot length |
177+
| wgt | double | Weight |
178+
| tag | character | Primary individual identifier |
179+
| note2 | character | Newly tagged individual for 'tag' |
180+
| ltag | character | Secondary tag information when ear tags were used in both ears |
181+
| note3 | character | Newly tagged individual for 'ltag' |
182+
183+
### Cleaning Script
184+
185+
Thanks to @ethanwhite for the data cleaning script. This script
186+
downloads the data using the [{portalr}](https://weecology.github.io/portalr/) package. It filters for the
187+
species and plot data, and years greater than 1977.
188+
189+
``` r
190+
# All packages used in this script:
191+
library(portalr)
192+
library(dplyr)
193+
194+
download_observations(".")
195+
data_tables <- load_rodent_data()
196+
197+
species_data <- data_tables[["species_table"]]
198+
plots_data <- data_tables[["plots_table"]]
199+
200+
plot_treatments <- plots_data %>%
201+
filter(year > 1977) |>
202+
mutate(iso_date = as.Date(paste0(year, "-", month, "-", "01")),
203+
plot = as.factor(plot)) %>%
204+
select(iso_date, plot, treatment)
205+
206+
plots_data_longterm <- plot_treatments |>
207+
group_by(plot) |>
208+
summarize(treatment = case_when(
209+
all(treatment == "control") ~ "control",
210+
all(treatment == "exclosure") ~ "exclosure")) |>
211+
filter(!is.na(treatment))
212+
213+
species_data <- species_data |>
214+
filter(censustarget == 1, unidentified == 0)
215+
216+
survey_data <- summarize_individual_rodents(
217+
time = "date",
218+
length = "Longterm") |>
219+
filter(year > 1977) |>
220+
filter(species %in% unique(species_data$species))
221+
222+
write.csv(survey_data, "surveys.csv", row.names = FALSE, na = "")
223+
write.csv(plots_data_longterm, "plots.csv", row.names = FALSE, na = "")
224+
write.csv(species_data, "species.csv", row.names = FALSE, na = "")
225+
```

data/2023/2023-05-02/species.csv

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
"species","scientificname","taxa","commonname","censustarget","unidentified","rodent","granivore","minhfl","meanhfl","maxhfl","minwgt","meanwgt","maxwgt","juvwgt"
2+
"BA","Baiomys taylori","Rodent","Northern pygmy mouse",1,0,1,1,6,13.25064599,15,6,9.449086162,18,
3+
"PB","Chaetodipus baileyi","Rodent","Bailey's pocket mouse",1,0,1,1,16,26.03125,47,10,31.87351518,79,18.9950495
4+
"PH","Chaetodipus hispidus","Rodent","Hispid pocket mouse",1,0,1,1,21,25.12820513,28,18,30.71794872,48,24
5+
"PI","Chaetodipus intermedius","Rodent","Rock pocket mouse",1,0,1,1,18,21.96261682,24,10,17.46666667,28,10
6+
"PP","Chaetodipus penicillatus","Rodent","Desert pocket mouse",1,0,1,1,11,21.47860867,27,4,17.61851292,42,11.66603416
7+
"DM","Dipodomys merriami","Rodent","Merriam's kangaroo rat",1,0,1,1,21,35.88388533,50,13,43.52721657,66,26.39257813
8+
"DO","Dipodomys ordii","Rodent","Ord's kangaroo rat",1,0,1,1,15,35.5316847,64,12,49.03951957,85,29.47101449
9+
"DS","Dipodomys spectabilis","Rodent","Banner-tailed kangaroo rat",1,0,1,1,39,49.92785388,58,12,119.9275604,190,76.82
10+
"NA","Neotoma albigula","Rodent","White-throated woodrat",1,0,1,0,21,32.07571429,42,30,162.5347738,280,83.75221239
11+
"OL","Onychomys leucogaster","Rodent","Northern Grasshopper Mouse",1,0,1,0,12,20.33408578,39,7,31.12671977,56,18.43243243
12+
"OT","Onychomys torridus","Rodent","Southern grasshopper mouse",1,0,1,0,11,20.02598091,31,5,24.05599577,46,15.53456221
13+
"PF","Perognathus flavus","Rodent","Silky pocket mouse",1,0,1,1,7,15.44696189,22,4,7.899749373,16,5.866666667
14+
"PE","Peromyscus eremicus","Rodent","Cactus mouse",1,0,1,1,11,19.94703118,30,7,21.98281643,71,15.14615385
15+
"PL","Peromyscus leucopus","Rodent","White-footed mouse",1,0,1,1,10,20.44604317,23,8,22.92647059,35,13
16+
"PM","Peromyscus maniculatus","Rodent","Deer Mouse",1,0,1,1,12,20.30291971,32,7,21.7079252,39,13.8125
17+
"RF","Reithrodontomys fulvescens","Rodent","Fulvous harvest mouse",1,0,1,1,15,17.51898734,20,9,13.4125,20,
18+
"RM","Reithrodontomys megalotis","Rodent","Western harvest mouse",1,0,1,1,6,16.36377427,23,4,10.72923441,21,7.145833333
19+
"RO","Reithrodontomys montanus","Rodent","Plains harvest mouse",1,0,1,1,11,15.62352941,17,5,10.76470588,16,
20+
"SF","Sigmodon fulviventer","Rodent","Tawny-bellied cotton rat",1,0,1,0,19,25.94850949,38,13,68.75806452,199,25.8
21+
"SH","Sigmodon hispidus","Rodent","Hispid cotton rat",1,0,1,0,20,28.67895879,39,16,88.46170213,205,36.21052632
22+
"SO","Sigmodon ochrognathus","Rodent","Yellow nosed cotton rat",1,0,1,0,15,25.65853659,36,15,55.41463415,105,42

0 commit comments

Comments
 (0)