| commit | 07854239257cc3eff8759503246595d2629f9367 | [log] [tgz] |
|---|---|---|
| author | David 'Digit' Turner <digit@google.com> | Mon Feb 03 18:12:12 2020 +0100 |
| committer | Commit Bot <commit-bot@chromium.org> | Wed Mar 11 17:25:40 2020 +0000 |
| tree | b55ad7e85222381c37bc143d7f45187e9a469be6 | |
| parent | 992e927e217baa8a74e6e2c5d7417cb65cf24824 [diff] |
Reduce RAM usage and improve speed of "gn gen" This CL continues previous optimization work [1] to improve GN's speed and RAM usage, especially for the Zircon build (see [2] for measurements). It consists in the following changes: - Introduce src/gn/hash_table_base.h to generalize the fast hash table used by StringAtom's implementation through a custom template class, and document its usage properly (with a disclaimer not to use it unless profiling shows it's worthwhile). - Use it to simplify and optimize the implementation of the UniqueVector<> template. - Implement SourceDir and Label with StringAtom instead of std::string instances. + Remove a small unwanted const Label copy in src/gn/analyzer.cc [1] https://gn-review.googlesource.com/c/gn/+/7280 [2] Measurement results for "gn gen" runs on my local machine, both using recent Chromium and Fuchsia source checkouts. CHROMIUM: RAM saved: 68 MiB Speedup: 8.4% ZIRCON: RAM saved: 405 MiB Speedup: 24.5% // CHROMIUM BEFORE /work/chromium0/src$ repeat_cmd 5 /tmp/gn-master gen out/default/ Done. Made 12150 targets from 2121 files in 4256ms Done. Made 12150 targets from 2121 files in 4218ms Done. Made 12150 targets from 2121 files in 4286ms Done. Made 12150 targets from 2121 files in 4201ms * Done. Made 12150 targets from 2121 files in 4244ms /work/chromium0/src$ /usr/bin/time -v /tmp/gn-master gen out/default Maximum resident set size (kbytes): 535480 // CHROMIUM AFTER /work/chromium0/src$ repeat_cmd 5 /tmp/gn-after gen out/default/ Done. Made 12150 targets from 2121 files in 3992ms Done. Made 12150 targets from 2121 files in 3874ms * Done. Made 12150 targets from 2121 files in 3951ms Done. Made 12150 targets from 2121 files in 3907ms Done. Made 12150 targets from 2121 files in 3905ms /work/chromium0/src$ /usr/bin/time -v /tmp/gn-after gen out/default Maximum resident set size (kbytes): 466056 // ZIRCON BEFORE /work/fuchsia0$ repeat_cmd 5 /tmp/gn-master gen --root=zircon out/default.zircon Done. Made 47910 targets from 782 files in 8915ms * Done. Made 47910 targets from 782 files in 8945ms Done. Made 47910 targets from 782 files in 9141ms Done. Made 47910 targets from 782 files in 9094ms Done. Made 47910 targets from 782 files in 9512ms /work/fuchsia0$ /usr/bin/time -v /tmp/gn-master gen --root=zircon out/default.zircon Maximum resident set size (kbytes): 1595364 // ZIRCON AFTER /work/fuchsia0$ repeat_cmd 5 /tmp/gn-after gen --root=zircon out/default.zircon Done. Made 47910 targets from 782 files in 7156ms * Done. Made 47910 targets from 782 files in 7715ms Done. Made 47910 targets from 782 files in 7327ms Done. Made 47910 targets from 782 files in 7491ms Done. Made 47910 targets from 782 files in 7573ms /work/fuchsia0$ /usr/bin/time -v /tmp/gn-master gen --root=zircon out/default.zircon Maximum resident set size (kbytes): 1180640 Change-Id: Id315564a9b2285cd89ca07cd69962f008439c9fb Reviewed-on: https://gn-review.googlesource.com/c/gn/+/7340 Reviewed-by: Brett Wilson <brettw@chromium.org> Reviewed-by: Brett Wilson <brettw@google.com> Commit-Queue: Brett Wilson <brettw@chromium.org>
GN is a meta-build system that generates build files for Ninja.
Related resources:
You can download the latest version of GN binary for Linux, macOS and Windows.
Alternatively, you can build GN from source:
git clone https://gn.googlesource.com/gn cd gn python build/gen.py ninja -C out # To run tests: out/gn_unittests
On Windows, it is expected that cl.exe, link.exe, and lib.exe can be found in PATH, so you'll want to run from a Visual Studio command prompt, or similar.
On Linux and Mac, the default compiler is clang++, a recent version is expected to be found in PATH. This can be overridden by setting CC, CXX, and AR.
There is a simple example in examples/simple_build directory that is a good place to get started with the minimal configuration.
To build and run the simple example with the default gcc compiler:
cd examples/simple_build ../../out/gn gen -C out ninja -C out ./out/hello
For a maximal configuration see the Chromium setup:
and the Fuchsia setup:
If you find a bug, you can see if it is known or report it in the bug database.
GN uses Gerrit for code review. The short version of how to patch is:
Register at https://gn-review.googlesource.com. ... edit code ... ninja -C out && out/gn_unittests
Then, to upload a change for review:
git commit git push origin HEAD:refs/for/master
The first time you do this you'll get an error from the server about a missing change-ID. Follow the directions in the error message to install the change-ID hook and run git commit --amend to apply the hook to the current commit.
When revising a change, use:
git commit --amend git push origin HEAD:refs/for/master
which will add the new changes to the existing code review, rather than creating a new one.
We ask that all contributors sign Google's Contributor License Agreement (either individual or corporate as appropriate, select ‘any other Google project’).
You may ask questions and follow along with GN‘s development on Chromium’s gn-dev@ Google Group.