Add vignette about locale sensitive functions. (#589)

Fixes #404

Author: kylieainslie

.vscode/settings.json

@@ -2,9 +2,5 @@
     "[r]": {
         "editor.formatOnSave": true,
         "editor.defaultFormatter": "Posit.air-vscode"
-    },
-    "[quarto]": {
-        "editor.formatOnSave": true,
-        "editor.defaultFormatter": "quarto.quarto"
     }
 }

NEWS.md

@@ -1,5 +1,6 @@
 # stringr (development version)
 
+* New `vignette("locale-sensitive")` about locale sensitive functions (@kylieainslie, #404)
 * New `str_ilike()` that follows the conventions of the SQL ILIKE operator (@edward-burn, #543).
 * `str_like(ignore_case)` is deprecated, with `str_like()` now always case sensitive to better follow the conventions of the SQL LIKE operator (@edward-burn, #543).
 * `str_sub<-` now gives a more informative error if `value` is not the correct length.

vignettes/.gitignore

@@ -0,0 +1 @@
+/.quarto/

vignettes/locale-sensitive.Rmd

@@ -0,0 +1,97 @@
+---
+title: "Locale sensitive functions"
+output: rmarkdown::html_vignette
+vignette: >
+  %\VignetteIndexEntry{Locale sensitive functions}
+  %\VignetteEngine{knitr::rmarkdown}
+  %\VignetteEncoding{UTF-8}
+---
+
+```{r}
+#| include: FALSE
+knitr::opts_chunk$set(
+  collapse = TRUE,
+  comment = "#>"
+)
+library(stringr)
+```
+
+A locale is a set of parameters that define a user's language, region, and cultural preferences. It determines language-specific rules for text processing, including how to:
+
+- Convert between uppercase and lowercase letters
+- Sort text alphabetically
+- Format dates, numbers, and currency
+- Handle character encoding and display
+
+In stringr, you can control the locale using the `locale` argument, which takes language codes like "en" (English), "tr" (Turkish), or "es_MX" (Mexican Spanish). In general, a locale is a lower-case language abbreviation, optionally followed by an underscore (_) and an upper-case region identifier. You can see which locales are supported in stringr by running `stringi::stri_locale_list()`.
+
+This vignette describes locale-sensitive stringr functions, i.e. functions with a `locale` argument. These functions fall into two broad categories:
+
+1. Case conversion
+2. Sorting and ordering
+
+## Case conversion
+
+`str_to_lower()`, `str_to_upper()`, `str_to_title()`, and `str_to_sentence()` all change the case of their inputs. But while most languages that use the Latin alphabet (like English) have upper and lower case, the rules for converting between the two aren't always the same. For example, Turkish has two forms of the letter "I": as well as "i" and "I", Turkish also has "ı", the dotless lowercase i, and "İ" is the dotted uppercase I. This means the rules for converting i to upper case and I to lower case are different from English:
+
+```{r}
+# English
+str_to_upper("i")
+str_to_lower("I")
+
+# Turkish
+str_to_upper("i", locale = "tr")
+str_to_lower("I", locale = "tr")
+```
+
+Another example is Dutch, where "ij" is a digraph treated as a single letter. This means that `str_to_sentence()` will incorrectly capitalize "ij" at the start of a sentence unless you use a Dutch locale:
+
+```{r}
+#| warning: false
+dutch_sentence <- "ijsland is een prachtig land in Noord-Europa."
+
+# Incorrect
+str_to_sentence(dutch_sentence)
+# Correct
+str_to_sentence(dutch_sentence, locale = "nl")
+```
+
+Case conversion also comes up in another situation: case-insensitive comparison. This is relevant in two contexts. First, `str_equal()` and `str_unique()` can optionally ignore case, so it's important to also supply locale when working with non-English text. For example, imagine we're searching for a Turkish name, ignoring case:
+
+```{r}
+turkish_names <- c("İpek", "Işık", "İbrahim")
+search_name <- "ipek"
+
+# incorrect
+str_equal(turkish_names, search_name, ignore_case = TRUE)
+
+# correct
+str_equal(turkish_names, search_name, ignore_case = TRUE, locale = "tr")
+```
+
+Case conversion also comes up in pattern matching functions like `str_detect()`. You might be accustomed to use `ignore_case = TRUE` with `regex()` or `fixed()`, but if you want to use locale-sensitive comparison you instead need to use `coll()`:
+
+```{r}
+# incorrect
+str_detect(turkish_names, fixed(search_name, ignore_case = TRUE))
+
+# correct
+str_detect(turkish_names, coll(search_name, ignore_case = TRUE, locale = "tr"))
+```
+
+## Sorting and ordering
+
+`str_sort()`, `str_order()`, and `str_rank()` all rely on the alphabetical ordering of letters. But not every language uses the same ordering as English. For example, Lithuanian places 'y' between 'i' and 'k', and Czech treats "ch" as a single compound letter that sorts after all other words beginning with 'h'. This means that to correctly sort words in these languages, you must provide the appropriate locale:
+
+```{r}
+czech_words <- c("had", "chata", "hrad", "chůze")
+lithuanian_words <- c("ąžuolas", "ėglė", "šuo", "yra", "žuvis")
+
+# incorrect
+str_sort(czech_words)
+str_sort(lithuanian_words)
+
+# correct
+str_sort(czech_words, locale = "cs")
+str_sort(lithuanian_words, locale = "lt")
+```