---------------------------------------------------------------------------- *** Unicode 18.0 update for ICU 79 Note: Between Unicode 18 beta and Unicode 18 final, the Chisoi script was removed. The log below reflects its addition as of the beta. https://www.unicode.org/versions/Unicode18.0.0/ https://www.unicode.org/versions/beta-18.0.0.html https://www.unicode.org/Public/draft/ https://www.unicode.org/reports/uax-proposed-updates.html https://www.unicode.org/reports/tr44/tr44-37.html https://unicode-org.atlassian.net/browse/ICU-23343 Unicode 18 https://unicode-org.atlassian.net/browse/CLDR-19339 BRS Unicode 18 Markus: export UNIDATA_ROOT=~/unidata export UNICODE_DATA=$UNIDATA_ROOT/uni18/final export CLDR_SRC=~/cldr/uni/src export ICU_ROOT=~/icu/uni export ICU_SRC=$ICU_ROOT/src export ICU_OUT=$ICU_ROOT/dbg export ICUDT=icudt79b export ICU4C_DATA_IN=$ICU_SRC/icu4c/source/data/in export ICU4C_UNIDATA=$ICU_SRC/icu4c/source/data/unidata export LD_LIBRARY_PATH=$ICU_OUT/icu4c/lib export UNICODE_TOOLS=~/unitools/mine/src Elango: export UNIDATA_ROOT=~/oss/unidata export UNICODE_DATA=$UNIDATA_ROOT/uni18/beta export CLDR_SRC=~/oss/cldr/mine/src export ICU_ROOT=~/oss/icu export ICU_SRC=$ICU_ROOT export ICU_OUT=$ICU_ROOT export ICUDT=icudt79b export ICU4C_DATA_IN=$ICU_SRC/icu4c/source/data/in export ICU4C_UNIDATA=$ICU_SRC/icu4c/source/data/unidata export LD_LIBRARY_PATH=$ICU_OUT/icu4c/lib export UNICODE_TOOLS=~/oss/unicodetools/mine/src *** Unicode version numbers - icu4c/source/data/makedata.mak - icu4c/source/common/unicode/uchar.h - com.ibm.icu.util.VersionInfo - com.ibm.icu.dev.test.lang.UCharacterTest.VERSION_ *** Configure: Build Unicode data for ICU4J - Run ICU4C "configure" _after_ updating the Unicode version number in uchar.h so that the makefiles see the new version number. - FYI: The option that adds the additional Unicode data files for ICU4J is ICU_DATA_BUILDTOOL_OPTS=--include_uni_core_data - Markus's version: cd $ICU_OUT/icu4c ICU_DATA_BUILDTOOL_OPTS=--include_uni_core_data CXXFLAGS="-DU_USING_ICU_NAMESPACE=0 -Wimplicit-fallthrough" CPPFLAGS="-DU_NO_DEFAULT_INCLUDE_UTF_HEADERS=1 -fsanitize=bounds" LDFLAGS=-fsanitize=bounds ../../src/icu4c/source/runConfigureICU --enable- debug --disable-release Linux/clang --prefix=/usr/local/google/home/mscherer/icu/mine/inst/icu4c > config.out 2>&1 ; tail config.out - Elango's version (diff default C++ compiler & in-source build paths): cd $ICU_OUT/icu4c/source ICU_DATA_BUILDTOOL_OPTS=--include_uni_core_data CXXFLAGS="-DU_USING_ICU_NAMESPACE=0 -Wimplicit-fallthrough" CPPFLAGS="-DU_NO_DEFAULT_INCLUDE_UTF_HEADERS=1 -fsanitize=bounds" LDFLAGS=-fsanitize=bounds ./runConfigureICU --enable-debug --disable-release Linux/gcc --prefix=/usr/local/google/home/elango/oss/icu/icu4c > config.out 2>&1 ; tail config.out *** data files & enums & parser code $ICU_OUT/icu4c$ echo;echo; date; make -j20 tests &> out.txt ; tail -n 30 out.txt ; date See https://unicode-org.github.io/icu/processes/unicode-update#bazel-build- process for an overview and for setup instructions. Consider running `bazelisk --version` outside of the $ICU_SRC folder to find out the latest `bazel` version, and copying that version number into the $ICU_SRC/.bazeliskrc config file. (Revert if you find incompatibilities, or, better, update our build & config files.) - remember to define the environment variables (see the start of the section for this Unicode version) - cd $ICU_SRC - optional but not necessary: bazelisk clean or even bazelisk clean --expunge - build/bootstrap/generate new files: icu4c/source/data/unidata/generate.sh - If you get seemingly unrelated compiler/linker errors, first try to bazelisk clean --expunge and if that still does not help, try to use a different compiler, for example via export CC=clang export CXX=clang++ - For debugging: bazelisk build -c dbg - UCA DUCET goes into Mark's Unicode tools, and a tool-tailored version goes into CLDR, see https://github.com/unicode-org/unicodetools/blob/main/docs/uca/index.md - update source/data/unidata/FractionalUCA.txt with FractionalUCA_SHORT.txt cp -v $CLDR_SRC/common/uca/FractionalUCA_SHORT.txt $ICU4C_UNIDATA/FractionalUCA.txt - update source/data/unidata/UCARules.txt with UCA_Rules_SHORT.txt cp -v $ICU4C_UNIDATA/UCARules.txt /tmp/UCARules-old.txt (note removing the underscore before "Rules") cp -v $CLDR_SRC/common/uca/UCA_Rules_SHORT.txt $ICU4C_UNIDATA/UCARules.txt - restore TODO diffs in UCARules.txt; adjust boundaries as needed, e.g., for new currency symbols meld /tmp/UCARules-old.txt $ICU4C_UNIDATA/UCARules.txt - update (ICU4C)/source/test/testdata/CollationTest_*.txt and (ICU4J)/main/tests/collate/src/com/ibm/icu/dev/data/CollationTest_*.txt from the CLDR root files (..._CLDR_..._SHORT.txt) cp -v $CLDR_SRC/common/uca/CollationTest_CLDR_NON_IGNORABLE_SHORT.txt $ICU_SRC/icu4c/source/test/testdata/CollationTest_NON_IGNORABLE_SHORT.txt cp -v $CLDR_SRC/common/uca/CollationTest_CLDR_SHIFTED_SHORT.txt $ICU_SRC/icu4c/source/test/testdata/CollationTest_SHIFTED_SHORT.txt cp -v $ICU_SRC/icu4c/source/test/testdata/CollationTest_*.txt $ICU_SRC/icu4j/main/collate/src/test/resources/com/ibm/icu/dev/data - if CLDR common/uca/unihan-index.txt changes, then update CLDR common/collation/root.xml and regenerate (or update in parallel) $ICU_SRC/icu4c/source/data/coll/root.txt - update CollationFCD.java: copy & paste the initializers of lcccIndex[] etc. from $ICU_SRC/icu4c/source/i18n/collationfcd.cpp to $ICU_SRC/icu4j/main/collate/src/main/java/com/ibm/icu/impl/coll/CollationFCD .java - generate data files, as above (generate.sh), now to pick up new collation data icu4c/source/data/unidata/generate.sh - rebuild ICU4C (make clean, make check, as usual) - run gencolusb for Unicode updates + https://github.com/markusicu/icu/blob/main/icu4c/source/tools/gencolusb/READ ME.md + until ICU-12062 is done + currently wants its own configure line; make clean, configure, build, generate + configure again using the command line in this file + rebuild ICU4C (make clean, make check, as usual) *** API additions - send notice to icu-design about new born-@stable API (enum constants etc.) *** CLDR numbering systems - look for new sets of decimal digits (gc=ND & nv=4) and add to CLDR for example: ~/unitools/mine/src$ diff -u unicodetools/data/ucd/17.0.0/extracted/DerivedGeneralCategory.txt unicodetools/data/ucd/dev/extracted/DerivedGeneralCategory.txt | grep '; Nd' | egrep '^\+' --> +16DA0..16DA9 ; Nd # [10] CHISOI DIGIT ZERO..CHISOI DIGIT NINE --> CLDR-19339 bring back Chisoi script for Unicode 18 --> removed once more from Unicode 18 *** merge the Unicode update branch back onto the main branch - make sure that changes to Unicode tools are checked in: https://github.com/unicode-org/unicodetools