From e17605dc3c3f3d5e7db299415ee53d5e98fc05bf Mon Sep 17 00:00:00 2001 From: Stan Ulbrych Date: Sun, 31 Aug 2025 16:52:19 +0100 Subject: [PATCH 1/4] Commit Co-authored-by: Alexander Belopolsky --- Doc/library/unicodedata.rst | 93 +++++++++++++++++++++++++------------ 1 file changed, 63 insertions(+), 30 deletions(-) diff --git a/Doc/library/unicodedata.rst b/Doc/library/unicodedata.rst index 0aef597d064e0e..c265b9b5fc20fd 100644 --- a/Doc/library/unicodedata.rst +++ b/Doc/library/unicodedata.rst @@ -25,80 +25,131 @@ Standard Annex #44, `"Unicode Character Database" `_. It defines the following functions: +.. seealso:: + + The :ref:`unicode-howto` for more information about Unicode and how to use + this module. + .. function:: lookup(name) Look up character by name. If a character with the given name is found, return the corresponding character. If not found, :exc:`KeyError` is raised. + For example:: + + >>> unicodedata.lookup('LEFT CURLY BRACKET') + '{' + + The characters returned by this function are the same as those produced by + ``\N`` escape sequence in string literals. For example:: + + >>> unicodedata.lookup('MIDDLE DOT') == '\N{MIDDLE DOT}' + True .. versionchanged:: 3.3 Support for name aliases [#]_ and named sequences [#]_ has been added. -.. function:: name(chr[, default]) +.. function:: name(chr, default=None, /) Returns the name assigned to the character *chr* as a string. If no name is defined, *default* is returned, or, if not given, :exc:`ValueError` is - raised. + raised. For example:: + + >>> unicodedata.name('½') + 'VULGAR FRACTION ONE HALF' + >>> unicodedata.name('\uFFFF', 'fallback') + 'fallback' -.. function:: decimal(chr[, default]) +.. function:: decimal(chr, default=None, /) Returns the decimal value assigned to the character *chr* as integer. If no such value is defined, *default* is returned, or, if not given, - :exc:`ValueError` is raised. + :exc:`ValueError` is raised. For example:: + >>> unicodedata.decimal('\N{ARABIC-INDIC DIGIT NINE}') + 9 + >>> unicodedata.decimal('\N{SUPERSCRIPT NINE}', -1) + -1 -.. function:: digit(chr[, default]) + +.. function:: digit(chr, default=None, /) Returns the digit value assigned to the character *chr* as integer. If no such value is defined, *default* is returned, or, if not given, :exc:`ValueError` is raised. + >>> unicodedata.digit('\N{SUPERSCRIPT NINE}') + 9 -.. function:: numeric(chr[, default]) +.. function:: numeric(chr, default=None, /) Returns the numeric value assigned to the character *chr* as float. If no such value is defined, *default* is returned, or, if not given, :exc:`ValueError` is raised. + >>> unicodedata.numeric('½') + 0.5 .. function:: category(chr) Returns the general category assigned to the character *chr* as - string. + string. General category names consist of two letters. + See the `General Category Values section of the Unicode Character + Database documentation `_ + for a list of category codes. For example:: + + >>> unicodedata.category('A') # 'L'etter, 'u'ppercase + 'Lu' .. function:: bidirectional(chr) Returns the bidirectional class assigned to the character *chr* as string. If no such value is defined, an empty string is returned. + See the `Bidirectional Class Values section of the Unicode Character + Database `_ + documentation for a list of bidirectional codes. For example:: + + >>> unicodedata.bidirectional('\N{ARABIC-INDIC DIGIT SEVEN}') # 'A'rabic, 'N'umber + 'AN' .. function:: combining(chr) Returns the canonical combining class assigned to the character *chr* as integer. Returns ``0`` if no combining class is defined. + See the `Canonical Combining Class Values section of the Unicode Character + Database `_ + for more information. .. function:: east_asian_width(chr) Returns the east asian width assigned to the character *chr* as - string. + string. For a list of widths and or more information, see the + `Unicode Standard Annex #11 `_. .. function:: mirrored(chr) Returns the mirrored property assigned to the character *chr* as integer. Returns ``1`` if the character has been identified as a "mirrored" - character in bidirectional text, ``0`` otherwise. + character in bidirectional text, ``0`` otherwise. For example:: + + >>> unicodedata.mirrored('>') + 1 .. function:: decomposition(chr) Returns the character decomposition mapping assigned to the character *chr* as string. An empty string is returned in case no such mapping is - defined. + defined. For example:: + + >>> unicodedata.decomposition('Ã') + '0041 0303' .. function:: normalize(form, unistr) @@ -122,9 +173,9 @@ following functions: normally would be unified with other characters. For example, U+2160 (ROMAN NUMERAL ONE) is really the same thing as U+0049 (LATIN CAPITAL LETTER I). However, it is supported in Unicode for compatibility with existing character - sets (e.g. gb2312). + sets (for example, gb2312). - The normal form KD (NFKD) will apply the compatibility decomposition, i.e. + The normal form KD (NFKD) will apply the compatibility decomposition, that is, replace all compatibility characters with their equivalents. The normal form KC (NFKC) first applies the compatibility decomposition, followed by the canonical composition. @@ -154,24 +205,6 @@ In addition, the module exposes the following constant: Unicode database version 3.2 instead, for applications that require this specific version of the Unicode database (such as IDNA). -Examples: - - >>> import unicodedata - >>> unicodedata.lookup('LEFT CURLY BRACKET') - '{' - >>> unicodedata.name('/') - 'SOLIDUS' - >>> unicodedata.decimal('9') - 9 - >>> unicodedata.decimal('a') - Traceback (most recent call last): - File "", line 1, in - ValueError: not a decimal - >>> unicodedata.category('A') # 'L'etter, 'u'ppercase - 'Lu' - >>> unicodedata.bidirectional('\u0660') # 'A'rabic, 'N'umber - 'AN' - .. rubric:: Footnotes From fdfa73dbef449ca379d9638d3479b1bdc6cf7f44 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych Date: Sun, 31 Aug 2025 16:56:44 +0100 Subject: [PATCH 2/4] Standardize gaps --- Doc/library/unicodedata.rst | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Doc/library/unicodedata.rst b/Doc/library/unicodedata.rst index c265b9b5fc20fd..c9a8a80e083474 100644 --- a/Doc/library/unicodedata.rst +++ b/Doc/library/unicodedata.rst @@ -83,6 +83,7 @@ following functions: >>> unicodedata.digit('\N{SUPERSCRIPT NINE}') 9 + .. function:: numeric(chr, default=None, /) Returns the numeric value assigned to the character *chr* as float. @@ -92,6 +93,7 @@ following functions: >>> unicodedata.numeric('½') 0.5 + .. function:: category(chr) Returns the general category assigned to the character *chr* as @@ -184,6 +186,7 @@ following functions: a human reader, if one has combining characters and the other doesn't, they may not compare equal. + .. function:: is_normalized(form, unistr) Return whether the Unicode string *unistr* is in the normal form *form*. Valid From 02de6ba360a182055b091f7ad122522717f87636 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych Date: Sun, 31 Aug 2025 17:15:51 +0100 Subject: [PATCH 3/4] Codeblock fixup --- Doc/library/unicodedata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/unicodedata.rst b/Doc/library/unicodedata.rst index c9a8a80e083474..9d1c0d2a4881c3 100644 --- a/Doc/library/unicodedata.rst +++ b/Doc/library/unicodedata.rst @@ -78,7 +78,7 @@ following functions: Returns the digit value assigned to the character *chr* as integer. If no such value is defined, *default* is returned, or, if not given, - :exc:`ValueError` is raised. + :exc:`ValueError` is raised:: >>> unicodedata.digit('\N{SUPERSCRIPT NINE}') 9 From aa608e639d420ed5ec804cc820e1ee01e3624ba9 Mon Sep 17 00:00:00 2001 From: Stan Ulbrych Date: Sun, 31 Aug 2025 17:32:30 +0100 Subject: [PATCH 4/4] Codeblock fixup --- Doc/library/unicodedata.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/unicodedata.rst b/Doc/library/unicodedata.rst index 9d1c0d2a4881c3..76f9a038181698 100644 --- a/Doc/library/unicodedata.rst +++ b/Doc/library/unicodedata.rst @@ -88,7 +88,7 @@ following functions: Returns the numeric value assigned to the character *chr* as float. If no such value is defined, *default* is returned, or, if not given, - :exc:`ValueError` is raised. + :exc:`ValueError` is raised:: >>> unicodedata.numeric('½') 0.5