blob: 67ee1c147b1a7e4b7bb18f4abb99c124859fec79 [file] [log] [blame]
How to rebuild the ICU data files.
This is based off of section B in the README.chromium file.
1. First revert to the non-Starboardized version of ICU in the Cobalt source
tree. This is necessary because regenerating the data files requires building
ICU, and this is best done with a non-Starboardized version of the library.
You can do this by looking at the git log for third_party/icu and tracing
it back until you find the first non-Cobalt change, and then checking out
that commit.
2. Set ICU_ROOT to the directory of the ICU source code in the Cobalt source
tree, e.g. `$COBALT_SRC_ROOT/src/third_party/icu`.
3. Create a build directory outside of the Cobalt codebase, and make that your
current working directory. Set ICU_DATA_ROOT to this directory.
4. Configure the ICU build. Note that we setup our compiler defines in the same
way that they are set when building ICU within Cobalt, according to our
icu.gyp file. Note that below, `-DUCONFIG_NO_LEGACY_CONVERSION=1` is left
out on purpose even though it is used when building ICU for Cobalt because
it causes an issue when converting to big endian
( https://unicode-org.atlassian.net/browse/ICU-11046 ). In addition,
`-DU_CHARSET_IS_UTF8=1` is also left out for the same reason as it was found
to produce a similar error.
```
env CPPFLAGS="-DUCONFIG_ONLY_HTML_CONVERSION=1 -DUCONFIG_NO_TRANSLITERATION=1 -DUCONFIG_NO_COLLATION=1" $ICU_ROOT/source/runConfigureICU Linux --disable-layout
```
5. Run make.
```
make -j8
```
Note that it's okay for it to end in an error around `css3transform.res`,
enough of it should have built successfully for us to complete the remaining
instructions.
6. Revert your ICU source checkout back to the tip, there have been local
modifications to it (specifically `source/data/mappings/convrtrs.txt`) that
we want to pick up here.
```
cd $ICU_ROOT
git checkout origin/COBALT
cd $ICU_DATA_ROOT
```
7. Run `$ICU_ROOT/scripts/trim_data.sh`
8. Run `$ICU_ROOT/android/patch_locale.sh`.
9. Run `$ICU_ROOT/scripts/make_data.sh`.
10. Run `$ICU_ROOT/scripts/clean_up_data_source.sh` to clean up the changes
made to $ICU_ROOT from the call to `patch_locale.sh`.
11. Create a new directory where we will rebuild ICU with different options
because the options used above will produce a version of the tool `icupkg` (
which we need to use to translate from little to big endian) that does not
work. Call this new directory $ICU_PKG_ROOT.
12. Change directory to $ICU_PKG_ROOT.
13. Run `$ICU_ROOT/source/runConfigureICU Linux --disable-layout && make -j8`
14. Run `$ICU_ROOT/scripts/make_data.sh`
15. Run `export LD_LIBRARY_PATH=$ICU_PKG_ROOT/lib`
16. Run `sudo apt install icu-devtools` to obtain the `icupkg` command and its
dependencies. Even though you will be running a locally-built version of
`icupkg`, you will need the shared libraries installed by this command.
17. Replace the files in this directory with the new files that you have built.
They will exist as a little-endian package file, but you can use the
`icupkg` tool to both unpack the files out of the package (which Cobalt
prefers), and switch from little-endian to big-endian.
Run:
```
cd $COBALT_SRC_ROOT/src/cobalt/content/icu
rm -rf *
mkdir icudt56l icudt56b
$ICU_PKG_ROOT/bin/icupkg -tl -x '*' -d icudt56l $ICU_DATA_ROOT/data/out/tmp/icudt56l.dat
$ICU_PKG_ROOT/bin/icupkg -tb -x '*' -d icudt56b $ICU_DATA_ROOT/data/out/tmp/icudt56l.dat
```
You should now have the ICU data files generated and ready to go in the Cobalt
source tree, and you can clean up any outstanding files. If there are any
leftover changes in your $ICU_ROOT directory, you can revert them. At this
point you can re-gyp, re-ninja, and run Cobalt and it should work.
Note that at the time of this writing, the final resulting contents had a size
of 6.9MB, from running the command
`du -sh $COBALT_SRC_ROOT/src/cobalt/content/icu/icudt56l`.