Skip to content

Commit fbfbe6c

Browse files
mihnitamihnita
committed
ICU-22917 Output to files, cleanup, update docs
1 parent 8dc2ec0 commit fbfbe6c

File tree

5 files changed

+635
-458
lines changed

5 files changed

+635
-458
lines changed

‎docs/processes/release/tasks/updating-measure-unit-old.md‎

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
---
2+
layout: default
3+
title: Updating MeasureUnit with new CLDR data
4+
parent: Release & Milestone Tasks
5+
grand_parent: Contributors
6+
nav_order: 120
7+
---
8+
9+
<!--
10+
© 2020 and later: Unicode, Inc. and others.
11+
License & terms of use: http://www.unicode.org/copyright.html
12+
-->
13+
14+
# Updating MeasureUnit with new CLDR data
15+
{: .no_toc }
16+
17+
## Contents
18+
{: .no_toc .text-delta }
19+
20+
1. TOC
21+
{:toc}
22+
23+
---
24+
25+
This document explains how to update the C++ and Java version of the MeasureUnit
26+
class with new CLDR data.
27+
28+
Code is generated by running MeasureUnitTest.java unit tests, which writes
29+
generated code to System.out. Two ways to access this:
30+
31+
1. Within **eclipse**:
32+
- Open MeasureUnitTest.java, run it by clicking on the green play button on
33+
menu bar.
34+
- Copy the generated code from the eclipse console to the clipboard.
35+
36+
2. With **ant**:
37+
- Run: `ant checkTest
38+
-Dtestclass='com.ibm.icu.dev.test.format.MeasureUnitTest'`
39+
- Open the checkTest output: `out/junit-results/checkTest/html/index.html`
40+
- Navigate to "System.out" at the bottom of the MeasureUnitTest page to find
41+
the generated code, and copy to the clipboard.
42+
43+
After syncing CLDR data with ICU do the following. This documentation assumes
44+
that you are updating the MeasureUnit clases for ICU 68.
45+
46+
* Check out
47+
$GIT_ROOT/icu4j/main/common_tests/src/test/java/com/ibm/icu/dev/test/format/MeasureUnitTest.java
48+
* Open MeasureUnitTest.java.
49+
* Find the `testZZZ` test, its code should all be commented out. This test will
50+
execute last and will run the desired code.
51+
52+
Make sure DRAFT_VERSIONS at top of MeasureUnitTest.java is set correctly.
53+
These are the ICU versions that have draft methods.
54+
55+
## Update MeasureUnit.java
56+
57+
* Change `testZZZ` to run `generateConstants(“68”); // ICU 68.`
58+
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
59+
* Open MeasureUnit.java:
60+
$GIT_ROOT/icu4j/main/core/src/main/java/com/ibm/icu/util/MeasureUnit.java
61+
* Look for line containing:
62+
63+
`// Start generated MeasureUnit constants`
64+
* Look for line containing:
65+
66+
`// End generated MeasureUnit constants`
67+
* Replace all the generated code in between with the contents of the clipboard
68+
* Run the MeasureUnitTest.java to ensure that the new code is backward
69+
compatible. These compatibility tests are called something like
70+
`TestCompatible65`, which tests backward compatibility with ICU 65.
71+
* Create a compatibility test for ICU 68. Change `testZZZ` to run
72+
`generateBackwardCompatibilityTest(“68”)`
73+
* Run tests.
74+
* Copy generated test (see instructions above) into MeasureUnitTest.java
75+
* Run tests again to ensure that new code is backward compatible with itself
76+
77+
## Update ICU4C
78+
79+
* checkout ICU4C
80+
81+
### Update measunit.h
82+
83+
* Change testZZZ to run `generateCXXHConstants(“68”); // ICU 68`.
84+
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
85+
* Open $GIT_ROOT/icu4c/source/i18n/unicode/measunit.h. Look for line containing:
86+
87+
`// Start generated createXXX methods`
88+
* Look for line:
89+
90+
`// End generated createXXX methods`
91+
* Replace all the generated code in between with the contents of the clipboard
92+
93+
### Update measunit.cpp
94+
95+
* Change testZZZ to run generateCXXConstants();
96+
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
97+
* Open $GIT_ROOT/icu4c/source/i18n/measunit.cpp. Look for line containing:
98+
99+
`// Start generated code for measunit.cpp`
100+
* Look for lines
101+
102+
`// End generated code for measunit.cpp`
103+
* Replace all the generated code in between with the contents of the clipboard
104+
105+
### Run C++ tests
106+
107+
* Run `./intltest format/MeasureFormatTest` from `test/intltest` to ensure new
108+
code is backward compatible.
109+
* Create a compatibility test for ICU 68. Change `testZZZ` in eclipse to run
110+
`generateCXXBackwardCompatibilityTest(“68”)`
111+
* Run tests.
112+
* Copy generated test (see instructions above) into
113+
$GIT_ROOT/icu4c/source/test/intltest/measfmttest.cpp. Make other necessary
114+
changes to make test compile. You can find these changes by searching for
115+
`TestCompatible65()`
116+
* Run tests again to ensure that new code is backward compatible with itself
117+
118+
## Finishing changes
119+
120+
These last changes are necessary to permanently record the ICU version number of
121+
any new measure units. Without these changes any new functions for this release
122+
will be considered new for the next release too.
123+
124+
* Change `testZZZ` to run `updateJAVAVersions(“68”);`
125+
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
126+
* Append the clipboard contents to the values of the JAVA_VERSIONS variable
127+
near the top of MeasureUnitTest.java.
128+
129+
**Important:** what you are copying are just the new functions for the current
130+
ICU version, in this case 68. Therefore append, do not replace.
131+
132+
## Updating units.txt and unitConstants
133+
134+
The standard ldml2icu process is used to update ICU's resource files (see
135+
[cldr-icu-readme.txt](https://github.com/unicode-org/icu/blob/main/icu4c/source/data/cldr-icu-readme.txt)).
136+
CLDR's units.xml defines conversion rates in terms of some constants defined in
137+
`unitConstants`.
138+
139+
For efficiency and simplicity, ICU does not read `unitConstants` from the
140+
resource file. If any new constants are added, some code changes would be
141+
needed. This would be caught by `testUnitConstantFreshness` unit test in
142+
`units_test.cpp`.
143+
144+
They are hard-coded:
145+
* Java: `UnitConverter.java` has the constant names in
146+
`UnitConverter.Factor.addEntity()` and constant values in
147+
`UnitConverter.Factor.getConversionRate()`.
148+
* C++: `units_converter.cpp` has the constant names in
149+
`addSingleFactorConstant()`, with the constant values in `double
150+
constantsValues[]` in the `units_converter.h` header file.

‎docs/processes/release/tasks/updating-measure-unit.md‎

Lines changed: 89 additions & 109 deletions
Original file line numberDiff line numberDiff line change
@@ -12,127 +12,106 @@ License & terms of use: http://www.unicode.org/copyright.html
1212
-->
1313

1414
# Updating MeasureUnit with new CLDR data
15+
1516
{: .no_toc }
1617

1718
## Contents
19+
1820
{: .no_toc .text-delta }
1921

2022
1. TOC
2123
{:toc}
2224

2325
---
2426

25-
This document explains how to update the C++ and Java version of the MeasureUnit
27+
This document explains how to update the C++ and Java version of the `MeasureUnit`
2628
class with new CLDR data.
2729

28-
Code is generated by running MeasureUnitTest.java unit tests, which writes
29-
generated code to System.out. Two ways to access this:
30-
31-
1. Within **eclipse**:
32-
- Open MeasureUnitTest.java, run it by clicking on the green play button on
33-
menu bar.
34-
- Copy the generated code from the eclipse console to the clipboard.
35-
36-
2. With **ant**:
37-
- Run: `ant checkTest
38-
-Dtestclass='com.ibm.icu.dev.test.format.MeasureUnitTest'`
39-
- Open the checkTest output: `out/junit-results/checkTest/html/index.html`
40-
- Navigate to "System.out" at the bottom of the MeasureUnitTest page to find
41-
the generated code, and copy to the clipboard.
42-
43-
After syncing CLDR data with ICU do the following. This documentation assumes
44-
that you are updating the MeasureUnit clases for ICU 68.
45-
46-
* Check out
47-
$GIT_ROOT/icu4j/main/common_tests/src/test/java/com/ibm/icu/dev/test/format/MeasureUnitTest.java
48-
* Open MeasureUnitTest.java.
49-
* Find the `testZZZ` test, its code should all be commented out. This test will
50-
execute last and will run the desired code.
51-
52-
Make sure DRAFT_VERSIONS at top of MeasureUnitTest.java is set correctly.
53-
These are the ICU versions that have draft methods.
54-
55-
## Update MeasureUnit.java
56-
57-
* Change `testZZZ` to run `generateConstants(“68”); // ICU 68.`
58-
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
59-
* Open MeasureUnit.java:
60-
$GIT_ROOT/icu4j/main/core/src/main/java/com/ibm/icu/util/MeasureUnit.java
61-
* Look for line containing:
62-
63-
`// Start generated MeasureUnit constants`
64-
* Look for line containing:
65-
66-
`// End generated MeasureUnit constants`
67-
* Replace all the generated code in between with the contents of the clipboard
68-
* Run the MeasureUnitTest.java to ensure that the new code is backward
69-
compatible. These compatibility tests are called something like
70-
`TestCompatible65`, which tests backward compatibility with ICU 65.
71-
* Create a compatibility test for ICU 68. Change `testZZZ` to run
72-
`generateBackwardCompatibilityTest(“68”)`
73-
* Run tests.
74-
* Copy generated test (see instructions above) into MeasureUnitTest.java
75-
* Run tests again to ensure that new code is backward compatible with itself
76-
77-
## Update ICU4C
78-
79-
* checkout ICU4C
80-
81-
### Update measunit.h
82-
83-
* Change testZZZ to run `generateCXXHConstants(“68”); // ICU 68`.
84-
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
85-
* Open $GIT_ROOT/icu4c/source/i18n/unicode/measunit.h. Look for line containing:
86-
87-
`// Start generated createXXX methods`
88-
* Look for line:
89-
90-
`// End generated createXXX methods`
91-
* Replace all the generated code in between with the contents of the clipboard
92-
93-
### Update measunit.cpp
94-
95-
* Change testZZZ to run generateCXXConstants();
96-
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
97-
* Open $GIT_ROOT/icu4c/source/i18n/measunit.cpp. Look for line containing:
98-
99-
`// Start generated code for measunit.cpp`
100-
* Look for lines
101-
102-
`// End generated code for measunit.cpp`
103-
* Replace all the generated code in between with the contents of the clipboard
104-
105-
### Run C++ tests
106-
107-
* Run `./intltest format/MeasureFormatTest` from `test/intltest` to ensure new
108-
code is backward compatible.
109-
* Create a compatibility test for ICU 68. Change `testZZZ` in eclipse to run
110-
`generateCXXBackwardCompatibilityTest(“68”)`
111-
* Run tests.
112-
* Copy generated test (see instructions above) into
113-
$GIT_ROOT/icu4c/source/test/intltest/measfmttest.cpp. Make other necessary
114-
changes to make test compile. You can find these changes by searching for
115-
`TestCompatible65()`
116-
* Run tests again to ensure that new code is backward compatible with itself
117-
118-
## Finishing changes
119-
120-
These last changes are necessary to permanently record the ICU version number of
121-
any new measure units. Without these changes any new functions for this release
122-
will be considered new for the next release too.
123-
124-
* Change `testZZZ` to run `updateJAVAVersions(“68”);`
125-
* Run MeasureUnitTest.java, copy the generated code (see instructions above).
126-
* Append the clipboard contents to the values of the JAVA_VERSIONS variable
127-
near the top of MeasureUnitTest.java.
128-
129-
**Important:** what you are copying are just the new functions for the current
130-
ICU version, in this case 68. Therefore append, do not replace.
131-
132-
## Updating units.txt and unitConstants
133-
134-
The standard ldml2icu process is used to update ICU's resource files (see
135-
[cldr-icu-readme.txt](https://github.com/unicode-org/icu/blob/main/icu4c/source/data/cldr-icu-readme.txt)).
30+
This document applies to ICU 77 and later.
31+
For older versions see updating-measure-unit-old.md
32+
33+
Make sure `DRAFT_VERSION_SET` at top of
34+
`./icu4j/main/common_tests/src/test/java/com/ibm/icu/dev/test/format/MeasureUnitGeneratorTest.java`
35+
is set correctly. \
36+
These are the ICU versions that have draft methods.
37+
38+
The code is generated by running `MeasureUnitGeneratorTest.java` unit tests, which writes
39+
generated code to various file.
40+
41+
1. With **maven** (command line):
42+
- Change folder to `{icuRoot}/icu4j`
43+
- run `mvn install -DskipTests -DskipITs`
44+
- run `mvn install -q -Dtest=MeasureUnitGeneratorTest -DgenerateMeasureUnitUpdate -f main/common_tests`
45+
46+
2. Within **Eclipse**:
47+
- Open `MeasureUnitGeneratorTest.java`, find the `generateUnitTestsUpdate` methods
48+
and run it by clicking on the green play button on menu bar. \
49+
Choose "JUnit Test" if asked. \
50+
This will not generate the update, but it will run the test and create a "Run Configuration". \
51+
Open it (Main menu -- "Run" -- "Run Configurations"), select the one named
52+
`MeasureUnitGeneratorTest.generateUnitTestsUpdate`, go to the "Arguments" tab and add
53+
`-DgenerateMeasureUnitUpdate` to the "VM Arguments" text area.
54+
55+
Both methods will generate files with in `icu4j/main/common_tests/target/` folder. \
56+
The file names and the logging to the standard output will guide you.
57+
58+
It currently looks something like this:
59+
```
60+
Copy the generated code fragments from / to
61+
/some/absolute/path/icu4j/main/common_tests/target/MeasureUnit.java \
62+
/some/absolute/path/icu4j/main/core/src/main/java/com/ibm/icu/util/MeasureUnit.java
63+
64+
Copy the generated code fragments from / to
65+
/some/absolute/path/icu4j/main/common_tests/target/MeasureUnitCompatibilityTest.java \
66+
/some/absolute/path/icu4j/main/common_tests/src/test/java/com/ibm/icu/dev/test/format/MeasureUnitCompatibilityTest.java
67+
68+
Copy the generated code fragments from / to
69+
/some/absolute/path/icu4j/main/common_tests/target/measunit.h \
70+
/some/absolute/path/icu4c/source/i18n/unicode/measunit.h
71+
72+
Copy the generated code fragments from / to
73+
/some/absolute/path/icu4j/main/common_tests/target/measunit.cpp \
74+
/some/absolute/path/icu4c/source/i18n/measunit.cpp
75+
76+
Copy the generated code fragments from / to
77+
/some/absolute/path/icu4j/main/common_tests/target/measfmttest.cpp \
78+
/some/absolute/path/icu4c/source/test/intltest/measfmttest.cpp
79+
80+
Copy the generated code fragments from / to
81+
/some/absolute/path/icu4j/main/common_tests/target/MeasureUnitGeneratorTest.java \
82+
/some/absolute/path/icu4j/main/common_tests/src/test/java/com/ibm/icu/dev/test/format/MeasureUnitGeneratorTest.java
83+
```
84+
85+
Some kind of diff tool or editor (for example `vi -d`) work nicely.
86+
87+
Look for line containing `// Start generated ...` and `// End generated ...`
88+
These lines exist in both the original files, and the generated one. \
89+
Replace all the generated code in between with the contents of the clipboard.
90+
91+
If the generated code has no `// Start` ... `// End ...` pair then the new
92+
code should be appended at some fixed place (details below).
93+
94+
* **`MeasureUnit.java`:** replace range.
95+
* **`MeasureUnitCompatibilityTest.java`:** append the new generated method at the end. \
96+
It is named something like `TestCompatible<version>()`. \
97+
Don't add it if it already exists.
98+
* **`measunit.h`:** replace range.
99+
* **`measunit.cpp`:** replace range.
100+
* **`measfmttest.cpp`:** append the new generated method after the last
101+
`MeasureFormatTest::TestCompatible<version>()` method. \
102+
Don't add it if it already exists. \
103+
WARNING: here you should add the method in two places. The method proper, with code,
104+
as generated, and the declaration in the class definition.
105+
* **`MeasureUnitGeneratorTest.java`:** append the new pairs of measure + version at
106+
the end of the `JAVA_VERSIONS` structure. \
107+
Don't add them if they already exist.
108+
109+
## Run tests for both `icu4c` and `icu4j`
110+
111+
## Updating `units.txt` and `unitConstants`
112+
113+
The standard `ldml2icu` process is used to update ICU's resource files (see
114+
[`cldr-icu-readme.txt`](https://github.com/unicode-org/icu/blob/main/icu4c/source/data/cldr-icu-readme.txt)).
136115
CLDR's units.xml defines conversion rates in terms of some constants defined in
137116
`unitConstants`.
138117

@@ -142,6 +121,7 @@ needed. This would be caught by `testUnitConstantFreshness` unit test in
142121
`units_test.cpp`.
143122

144123
They are hard-coded:
124+
145125
* Java: `UnitConverter.java` has the constant names in
146126
`UnitConverter.Factor.addEntity()` and constant values in
147127
`UnitConverter.Factor.getConversionRate()`.

0 commit comments

Comments
 (0)