Skip to content

Commit 76e9c0d

Browse files
authored
2023 Week 12: Programming Languages (#536)
Closes #530.
1 parent 4e54ec3 commit 76e9c0d

File tree

8 files changed

+4542
-0
lines changed

8 files changed

+4542
-0
lines changed

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,7 @@ Need a **reminder**? There are [extensions](https://chrome.google.com/webstore/d
133133
| 9 | `2023-02-28` | [African Language Sentiment](data/2023/2023-02-28/readme.md) | [AfriSenti: Sentiment Analysis dataset for 14 African languages](https://github.com/afrisenti-semeval/afrisent-semeval-2023) | [AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages](https://arxiv.org/pdf/2302.08956.pdf) |
134134
| 10 | `2023-03-07` | [Numbats in Australia](data/2023/2023-03-07/readme.md) | [Atlas of Living Australia](https://www.ala.org.au) | [Numbat page at the Atlas of Living Australia](https://bie.ala.org.au/species/https://biodiversity.org.au/afd/taxa/6c72d199-f0f1-44d3-8197-224a2f7cff5f) |
135135
| 11 | `2023-03-14` | [European Drug Development](data/2023/2023-03-14/readme.md) | [European Medicines Agency](https://www.ema.europa.eu/en/medicines/download-medicine-data) | [Dissecting 28 years of European pharmaceutical development](https://towardsdatascience.com/dissecting-28-years-of-european-pharmaceutical-development-3affd8f87dc0) |
136+
| 12 | `2023-03-21` | [Programming Languages](data/2023/2023-03-21/readme.md) | [Programming Language DataBase](https://pldb.com/index.html) | [Does every programming language have line comments?](https://pldb.com/posts/does-every-programming-language-support-line-comments.html) |
136137

137138
***
138139

data/2023/2023-03-21/languages.csv

Lines changed: 4304 additions & 0 deletions
Large diffs are not rendered by default.

data/2023/2023-03-21/pic1.png

92.7 KB
Loading

data/2023/2023-03-21/pic2.png

114 KB
Loading
Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"article_link": "https://pldb.com/posts/does-every-programming-language-support-line-comments.html",
3+
"pic1_alt": "A comment on comments. A bar plot with comment tokens on the x-axis, sorted by popularity, and percentage of programming languages that use that comment character on the y-axis. // is highest with about 38% of languages, followed by # at about 21%.",
4+
"pic2_alt": "A plot of comment tokens, with the year the language was created on the x-axis, and the language rank on the y-axis. The comment tokens themselves are the points. The captions shows the semicolon character, and then says The matter seems to be settled. The last time a new line comment token appeared in the top 250 list was 1993."
5+
}

data/2023/2023-03-21/readme.md

Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
1+
### Please add alt text to your posts
2+
3+
Please add alt text (alternative text) to all of your posted graphics for `#TidyTuesday`.
4+
5+
Twitter provides [guidelines](https://help.twitter.com/en/using-twitter/picture-descriptions) for how to add alt text to your images.
6+
7+
The DataViz Society/Nightingale by way of Amy Cesal has an [article](https://medium.com/nightingale/writing-alt-text-for-data-visualization-2a218ef43f81) on writing _good_ alt text for plots/graphs.
8+
9+
> Here’s a simple formula for writing alt text for data visualization:
10+
> ### Chart type
11+
> It’s helpful for people with partial sight to know what chart type it is and gives context for understanding the rest of the visual.
12+
> Example: Line graph
13+
> ### Type of data
14+
> What data is included in the chart? The x and y axis labels may help you figure this out.
15+
> Example: number of bananas sold per day in the last year
16+
> ### Reason for including the chart
17+
> Think about why you’re including this visual. What does it show that’s meaningful. There should be a point to every visual and you should tell people what to look for.
18+
> Example: the winter months have more banana sales
19+
> ### Link to data or source
20+
> Don’t include this in your alt text, but it should be included somewhere in the surrounding text. People should be able to click on a link to view the source data or dig further into the visual. This provides transparency about your source and lets people explore the data.
21+
> Example: Data from the USDA
22+
23+
Penn State has an [article](https://accessibility.psu.edu/images/charts/) on writing alt text descriptions for charts and tables.
24+
25+
> Charts, graphs and maps use visuals to convey complex images to users. But since they are images, these media provide serious accessibility issues to colorblind users and users of screen readers. See the [examples on this page](https://accessibility.psu.edu/images/charts/) for details on how to make charts more accessible.
26+
27+
The `{rtweet}` package includes the [ability to post tweets](https://docs.ropensci.org/rtweet/reference/post_tweet.html) with alt text programatically.
28+
29+
Need a **reminder**? There are [extensions](https://chrome.google.com/webstore/detail/twitter-required-alt-text/fpjlpckbikddocimpfcgaldjghimjiik/related) that force you to remember to add Alt Text to Tweets with media.
30+
31+
# Programming Languages
32+
33+
The data this week comes from the [Programming Language DataBase](https://pldb.com/index.html). Thanks to [Jesus M. Castagnetto](https://github.com/rfordatascience/tidytuesday/issues/530) for the suggestion!
34+
35+
The PLDB has a [blog](https://pldb.com/posts/index.html) with numerous articles exploring the data, such as [Does every programming language have line comments?](https://pldb.com/posts/does-every-programming-language-support-line-comments.html).
36+
37+
The data is user-submitted, so you might want to confirm the accuracy of anything particularly surprising that you find before stating it with certainty!
38+
39+
### Get the data here
40+
41+
```{r}
42+
# Get the Data
43+
44+
# Read in with tidytuesdayR package
45+
# Install from CRAN via: install.packages("tidytuesdayR")
46+
# This loads the readme and all the datasets for the week of interest
47+
48+
# Either ISO-8601 date or year/week works!
49+
50+
tuesdata <- tidytuesdayR::tt_load('2023-03-21')
51+
tuesdata <- tidytuesdayR::tt_load(2023, week = 12)
52+
53+
languages <- languages
54+
55+
# Or read in the data manually
56+
57+
languages <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2023/2023-03-21/languages.csv')
58+
```
59+
60+
### Data Dictionary
61+
62+
# `languages.csv`
63+
64+
The [full data dictionary](https://pldb.com/csv.html) is available from PLDB.com.
65+
66+
|variable |class |description |
67+
|:---------------------------------|:---------|:-----------|
68+
|pldb_id |character |A standardized, uniquified version of the language name, used as an ID on the PLDB site.|
69+
|title |character |The official title of the language.|
70+
|description |character |Description of the repo on GitHub.|
71+
|type |character |Which category in PLDB's subjective ontology does this entity fit into.|
72+
|appeared |double |What year was the language publicly released and/or announced?|
73+
|creators |character |Name(s) of the original creators of the language delimited by " and "|
74+
|website |character |URL of the official homepage for the language project.|
75+
|domain_name |character |If the project website is on its own domain.|
76+
|domain_name_registered |double |When was this domain first registered?|
77+
|reference |character |A link to more info about this entity.|
78+
|isbndb |double |Books about this language from [ISBNdb](https://isbndb.com/).|
79+
|book_count |double |Computed; the number of books found for this language at isbndb.com|
80+
|semantic_scholar |integer |Papers about this language from Semantic Scholar.|
81+
|language_rank |double |Computed; A rank for the language, taking into account various online rankings. The computation for this column is not currently clear.|
82+
|github_repo |character |URL of the official GitHub repo for the project if it hosted there.|
83+
|github_repo_stars |double |How many stars of the repo?|
84+
|github_repo_forks |double |How many forks of the repo?|
85+
|github_repo_updated |double |What year was the last commit made?|
86+
|github_repo_subscribers |double |How many subscribers to the repo?|
87+
|github_repo_created |double |When was the *Github repo* for this entity created?|
88+
|github_repo_description |character |Description of the repo on GitHub.|
89+
|github_repo_issues |double |How many isses on the repo?|
90+
|github_repo_first_commit |double |What year the first commit made in this git repo?|
91+
|github_language |character |GitHub has a set of supported languages as defined [here](https://raw.githubusercontent.com/github/linguist/master/lib/linguist/languages.yml)|
92+
|github_language_tm_scope |character |The TextMate scope that represents this programming language.|
93+
|github_language_type |character |Either data, programming, markup, prose, or nil.|
94+
|github_language_ace_mode |character |A String name of the Ace Mode used for highlighting whenever a file is edited. This must match one of the filenames in http://git.io/3XO_Cg. Use "text" if a mode does not exist.|
95+
|github_language_file_extensions |character |An Array of associated extensions (the first one is considered the primary extension, the others should be listed alphabetically).|
96+
|github_language_repos |double |How many repos for this language does GitHub report?|
97+
|wikipedia |character |URL of the entity on Wikipedia, if and only if it has a page dedicated to it.|
98+
|wikipedia_daily_page_views |double |How many page views per day does this Wikipedia page get? Useful as a signal for rankings. Available via WP api.|
99+
|wikipedia_backlinks_count |double |How many pages on WP link to this page?|
100+
|wikipedia_summary |character |What is the text summary of the language from the Wikipedia page?|
101+
|wikipedia_page_id |double |Waht is the internal ID for this entity on WP?|
102+
|wikipedia_appeared |double |When does Wikipedia claim this entity first appeared?|
103+
|wikipedia_created |double |When was the *Wikipedia page* for this entity created?|
104+
|wikipedia_revision_count |double |How many revisions does this page have?|
105+
|wikipedia_related |character |What languages does Wikipedia have as related?|
106+
|features_has_comments |logical |Does this language have a comment character?|
107+
|features_has_semantic_indentation |logical |Does indentation have semantic meaning in this language?|
108+
|features_has_line_comments |logical |Does this language support inline comments (as opposed to comments that must span an entire line)?|
109+
|line_comment_token |character |Defined as a token that can be placed anywhere on a line and starts a comment that cannot be stopped except by a line break character or end of file.|
110+
|last_activity |double |Computed; The most recent of any year field in the PLDB for this language.|
111+
|number_of_users |double |Computed; "Crude user estimate from a linear model.|
112+
|number_of_jobs |double |Computed; The estimated number of job openings for programmers in this language.|
113+
|origin_community |character |In what community(ies) did the language first originate?|
114+
|central_package_repository_count |double |Number of packages in a central repository. If this value is not known, it is set to 0 (so "0" can mean "no repository exists", "the repository exists but is empty" (unlikely), or "we do not know if a repository exists". This value is definitely incorrect for R.|
115+
|file_type |character |What is the file encoding for programs in this language?|
116+
|is_open_source |logical |Is it an open source project?|
117+
118+
119+
### Cleaning Script
120+
121+
```r
122+
library(tidyverse)
123+
library(janitor)
124+
library(knitr)
125+
126+
languages_url <- "https://pldb.com/languages.csv"
127+
128+
languages_raw <- read_csv(
129+
languages_url,
130+
# Some of the columns are very sparse, so have readr use everything for
131+
# guessing.
132+
guess_max = 4303
133+
) |>
134+
clean_names() |>
135+
# The semantic_scholar column is misformed for a handful of languages. It's ok
136+
# to introduce NAs here.
137+
mutate(
138+
semantic_scholar = as.integer(semantic_scholar)
139+
)
140+
141+
142+
# Almost all columns are almost completely empty. Keep the columns that have
143+
# more than 10% coverage.
144+
good_lang_cols <- languages_raw |>
145+
summarize(
146+
across(everything(), ~sum(!is.na(.x)))
147+
) |>
148+
tidyr::pivot_longer(
149+
everything(),
150+
names_to = "column",
151+
values_to = "non_empty"
152+
) |>
153+
mutate(
154+
coverage = non_empty/nrow(languages_raw)
155+
) |>
156+
filter(coverage > 0.1) |>
157+
pull(column)
158+
159+
languages <- languages_raw |>
160+
select(!!!good_lang_cols) |>
161+
# This column references a site that doesn't want to be used in projects.
162+
select(-hopl) |>
163+
# A couple columns are only relevant in the context of the mixed table with
164+
# non-languages included.
165+
select(-number_of_repos, -rank, -paper_count) |>
166+
# Some columns are internal metadata that is no longer true with this subset.
167+
select(-fact_count, -example_count) |>
168+
# I looked at R specifically, and the "country" column was inaccurate. Let's
169+
# not confuse people with known bad data.
170+
select(-country) |>
171+
# Organize the columns.
172+
select(
173+
pldb_id,
174+
title,
175+
description,
176+
type,
177+
appeared,
178+
creators,
179+
website,
180+
starts_with("domain_name"),
181+
reference,
182+
isbndb,
183+
book_count,
184+
semantic_scholar,
185+
language_rank,
186+
starts_with("github_"),
187+
starts_with("wikipedia"),
188+
starts_with("features_"),
189+
line_comment_token,
190+
everything()
191+
)
192+
193+
write_csv(
194+
languages,
195+
file = here::here(
196+
"data",
197+
"2023",
198+
"2023-03-21",
199+
"languages.csv"
200+
)
201+
)
202+
203+
# Use the online dictionary to help with the dictionary in the post.
204+
dictionary_url <- "https://pldb.com/columns.csv"
205+
dictionary <- read_csv(dictionary_url) |>
206+
clean_names() |>
207+
# I only need the column name and description for our dictionary.
208+
select(column, description) |>
209+
# I cleaned the column names, so let's do the same here.
210+
mutate(
211+
column = make_clean_names(column)
212+
) |>
213+
# We don't need the extras.
214+
filter(
215+
column %in% colnames(languages)
216+
)
217+
# Arrange dictionary to match the order of colnames(languages).
218+
dictionary <- dictionary[match(colnames(languages), dictionary$column), ]
219+
220+
dictionary |>
221+
mutate(
222+
class = map_chr(languages, typeof)
223+
) |>
224+
select(
225+
variable = column,
226+
class,
227+
description
228+
) |>
229+
knitr::kable()
230+
```

data/2023/readme.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,4 @@ Archive of datasets and articles from the 2023 series of `#TidyTuesday` events.
1515
| 9 | `2023-02-28` | [African Language Sentiment](2023-02-28/readme.md) | [AfriSenti: Sentiment Analysis dataset for 14 African languages](https://github.com/afrisenti-semeval/afrisent-semeval-2023) | [AfriSenti: A Twitter Sentiment Analysis Benchmark for African Languages](https://arxiv.org/pdf/2302.08956.pdf) |
1616
| 10 | `2023-03-07` | [Numbats in Australia](2023-03-07/readme.md) | [Atlas of Living Australia](https://www.ala.org.au) | [Numbat page at the Atlas of Living Australia](https://bie.ala.org.au/species/https://biodiversity.org.au/afd/taxa/6c72d199-f0f1-44d3-8197-224a2f7cff5f) |
1717
| 11 | `2023-03-14` | [European Drug Development](2023-03-14/readme.md) | [European Medicines Agency](https://www.ema.europa.eu/en/medicines/download-medicine-data) | [Dissecting 28 years of European pharmaceutical development](https://towardsdatascience.com/dissecting-28-years-of-european-pharmaceutical-development-3affd8f87dc0) |
18+
| 12 | `2023-03-21` | [Programming Languages](2023-03-21/readme.md) | [Programming Language DataBase](https://pldb.com/index.html) | [Does every programming language have line comments?](https://pldb.com/posts/does-every-programming-language-support-line-comments.html) |

static/tt_data_type.csv

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
Week,Date,year,data_files,data_type,delim
2+
12,2023-03-21,2023,languages.csv,csv,","
23
11,2023-03-14,2023,drugs.csv,csv,","
34
10,2023-03-07,2023,numbats.csv,csv,","
45
9,2023-02-28,2023,afrisenti.csv,csv,","

0 commit comments

Comments
 (0)