Expand unicodedata module documentation

[abalkin]

BPO	10665
Nosy	@malemburg, @loewis, @birkenfeld, @abalkin, @vstinner, @ezio-melotti
Files	unicodedata-doc.diff issue10665.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2010-12-09.21:31:00.541>
labels = ['type-feature', 'expert-unicode', 'docs']
title = 'Expand unicodedata module documentation'
updated_at = <Date 2012-09-26.18:46:07.734>
user = 'https://github.com/abalkin'

bugs.python.org fields:

activity = <Date 2012-09-26.18:46:07.734>
actor = 'ezio.melotti'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'Unicode']
creation = <Date 2010-12-09.21:31:00.541>
creator = 'belopolsky'
dependencies = []
files = ['19991', '20002']
hgrepos = []
issue_num = 10665
keywords = ['patch', 'needs review']
message_count = 21.0
messages = ['123700', '123704', '123708', '123740', '123753', '123756', '123890', '123905', '130536', '130646', '130699', '130864', '132236', '133845', '133848', '133849', '133852', '133853', '133854', '133857', '133884']
nosy_count = 7.0
nosy_names = ['lemburg', 'loewis', 'georg.brandl', 'belopolsky', 'vstinner', 'ezio.melotti', 'docs@python']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue10665'
versions = ['Python 3.2', 'Python 3.3', 'Python 3.4']

Linked PRs

• gh-54874: Expand unicodedata module documentation #138301
• [3.13] gh-54874: Expand unicodedata module documentation (GH-138301) #138345
• [3.14] gh-54874: Expand unicodedata module documentation (GH-138301) #138346

[abalkin]

unicodedata module documentation has not been updated to reflect transition to 6.0. Attached patch fixes the version and unicode.org links and starts making the documentation rely less on the unicode.org pages for basic understanding of the provided functionality.

I am posting work in progress to solicit feedback on how much of the Unicode Standard information we would want to present here.

On of the goals of this patch is to provide a standard reference that can be used throughout the library manual for basic Unicode concepts without sending the reader over to unicode.org.

[abalkin]

Added more tables semi-automatically produced from http://www.unicode.org/Public/UNIDATA/PropertyValueAliases.txt

[loewis]

Please, one issue per report and checkin, and no work-in-progress on the tracker. The issue of factually correcting claims about the unicodedata module and elaborations on how it works are unrelated issues.

[abalkin]

On Thu, Dec 9, 2010 at 6:10 PM, Martin v. Löwis <report@bugs.python.org> wrote:
..

Please, one issue per report and checkin,

The s/5.2/6.0/ issue is hardly worth a tracker ticket. I've
committed these changes in r87159. (Sorry for the unrelated changes -
reverted in the next checkin.)

and no work-in-progress on the tracker.

Why? I thought "release early, release often" was a good thing. I
wanted to get an early feedback because we certainly don't want to
replicate the Unicode Standard in the Python documentation, but I
think at least for the category() method that returns cryptic 2-letter
codes, we should include a table explaining them. I am not so sure
about bidirectional() or asian_width().

The issue of factually correcting claims about the unicodedata module and elaborations on how it works are unrelated issues.

I am changing the title of the issue to make it cover only the latter.

----------
nosy: +loewis

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue10665\>

---

[abalkin]

In bpo-10665.diff, I completed the character examples in the general categories table.

[loewis]

Why? I thought "release early, release often" was a good thing.

Create a branch for that, or post an issue on Rietveld. W-I-P IMO
confuses people reviewing the patches, running into the same ones
over-and-over again, only to find out every time "it's not ready yet".
So the natural reaction is to close it as rejected, for it being
incomplete.

Regards,
Martin

[abalkin]

On Fri, Dec 10, 2010 at 6:04 PM, Martin v. Löwis <report@bugs.python.org> wrote:
..

> Why?  I thought "release early, release often" was a good thing.

Create a branch for that, or post an issue on Rietveld.

Martin,

This is a documentation patch affecting a single HTML page. An svn
branch for something like this is certainly an overkill. Maybe when
branches become more user-friendly with Hg, it will make sense to do
something like this in a branch. Rietveld is great for code reviews,
but for doc patches its is sometimes desirable to post a rendered page
for a review. In this particular case, however the reST in diff is
quite readable.

W-I-P IMO
confuses people reviewing the patches, running into the same ones
over-and-over again, only to find out every time "it's not ready yet".

I posted this patch with a specific question: "how much of the Unicode
Standard information we would want to present?" The patch included
several tables similar to what a determined reader can find at
unicode.org. I think it is useful to present this information in the
Python docs for several reasons:

1. It makes the information more readily accessible to someone who
   just want to figure out what the code returned by
   unicodedata.category() means.

2. We can present examples using Python notation and focus on what is
   relevant to Python users. For example, what are the digits other than
   0-9, or what is the difference between a digit and a decimal.

3. Other parts of the documentation can refer to this information more
   easily. For example, str.isdigit() can refer to 'Nd' general
   category.

The downside is that we may need to update this info when Unicode
Standard changes. Given the pace of change for this info, I don't
think this is serious burden and most of the data can be
auto-generated from UCD files.

So the natural reaction is to close it as rejected, for it being
incomplete.

Are you going to reject say bpo-2636 on this basis? :-) Has *any*
patch ever been rejected as incomplete?

Seriously, I had a specific reason to post an incomplete patch for
review: formatting reST tables is tedious and if others think we
should not include this info in Python docs, I don't want to spend
more time polishing the patch. On the other hand, an incomplete patch
is helpful because it demonstrates how much information I am proposing
to include. If I just posted a request without a patch, a natural
reaction would be: do you have a patch?

[loewis]

Are you going to reject say bpo-2636 on this basis? :-) Has *any*
patch ever been rejected as incomplete?

I certainly did close patches for that reason. Before that, I keep
asking people not to post W-I-P. As for bpo-2636, I have been really
tempted to, see msg83277, msg83411. In one case, the conversation ended
with the user being banned from the tracker. I personally feel fairly
strongly that it is abuse to post things that you know are incomplete.
I won't review any such code.

[ezio-melotti]

The patch contains non-ascii chars that should be avoided (they break make pdf).

[vstinner]

The patch contains non-ascii chars that should be avoided (they break make pdf).

What is the error? Can't you fix the PDF generator instead?

[ezio-melotti]

I think Georg said that non-ascii chars shouldn't be used directly in the rst files, and one of the reasons is that they break make pdf (I haven't tried though).
I added him to the nosy.