Commit Graph
235 Commits
Author SHA1 Message Date
Fangrui Song 7a31ac3118 [InstrProf] Stabilize --show-prof-sym-list dump order
D118181 leverages the iteration order of StringSet, which is not
guaranteed to be deterministic.
2023-07-22 15:47:44 -07:00
Kazu Hirata 3b30e077ab [ProfileData] Remove unused class SoftInstrProfErrors
The last use was removed by:

  commit 98cce00371
  Author: David Blaikie <dblaikie@gmail.com>
  Date:   Mon Jul 10 03:04:59 2017 +0000

Differential Revision: https://reviews.llvm.org/D152640
2023-06-11 13:19:21 -07:00
Ellis Hoag 1117b9a284 [InstrProf] Use BalancedPartitioning to order temporal profiling trace data
In [0] we described an algorithm called //BalancedPartitioning// (bp) to consume function traces [1] and compute a function order that reduces the number of page faults during startup.

This patch adds the `order` command to the `llvm-profdata` tool which uses bp to output a function order that can be passed to the linker via `--symbol-ordering-file=`.

Special thanks to Sergey Pupyrev and Julian Mestre for designing this balanced partitioning algorithm.

[0] https://discourse.llvm.org/t/rfc-temporal-profiling-extension-for-irpgo/68068
[1] https://reviews.llvm.org/D147287

Reviewed By: spupyrev

Differential Revision: https://reviews.llvm.org/D147812
2023-06-06 11:59:57 -07:00
Craig Topper 6006d43e2d LLVM_FALLTHROUGH => [[fallthrough]]. NFC
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D150996
2023-05-24 12:40:10 -07:00
Jessica Paquette 17cfd2e025 [profiling] Improve error message for raw profile header mismatches
When a user uses a mismatched clang + llvm-profdata, they didn't get a very
informative error message. It would just say "unsupported version".

As a result, users are often confused as to what they are supposed to do and
tend to assume that it's a bug in the profiling runtime.

This patch improves the error message by:

- Adding a new class of error (`raw_profile_version_mismatch`) to make it clear
  that, specifically, the *raw profile* version is unsupported because of a
  tool mismatch.

- Adding an error message that tells the user which raw profile version was
  encountered, which version was expected, and instructs them to align their
  tool versions.

To support this, this patch also updates `InstrProfError::take` to also
propagate the optional error message.

Differential Revision: https://reviews.llvm.org/D149361
2023-04-27 14:51:38 -07:00
Ellis Hoag 244be0b0de [InstrProf] Temporal Profiling
As described in [0], this extends IRPGO to support //Temporal Profiling//.

When `-pgo-temporal-instrumentation` is used we add the `llvm.instrprof.timestamp()` intrinsic to the entry of functions which in turn gets lowered to a call to the compiler-rt function `INSTR_PROF_PROFILE_SET_TIMESTAMP()`. A new field in the `llvm_prf_cnts` section stores each function's timestamp. Then in `llvm-profdata merge` we convert these function timestamps into a //trace// and add it to the indexed profile.

Since these traces could significantly increase the profile size, we've added `-max-temporal-profile-trace-length` and `-temporal-profile-trace-reservoir-size` to limit the length of a trace and the number of traces in a profile, respectively.

In a future diff we plan to use these traces to construct an optimized function order to reduce the number of page faults during startup.

Special thanks to Julian Mestre for helping with reservoir sampling.

[0] https://discourse.llvm.org/t/rfc-temporal-profiling-extension-for-irpgo/68068

Reviewed By: snehasish

Differential Revision: https://reviews.llvm.org/D147287
2023-04-11 08:30:52 -07:00
Archibald Elliott 62c7f035b4 [NFC][TargetParser] Remove llvm/ADT/Triple.h
I also ran `git clang-format` to get the headers in the right order for
the new location, which has changed the order of other headers in two
files.
2023-02-07 12:39:46 +00:00
Steven Wu 516e301752 [NFC][Profile] Access profile through VirtualFileSystem
Make the access to profile data going through virtual file system so the
inputs can be remapped. In the context of the caching, it can make sure
we capture the inputs and provided an immutable input as profile data.

Reviewed By: akyrtzi, benlangmuir

Differential Revision: https://reviews.llvm.org/D139052
2023-02-01 09:25:02 -08:00
serge-sans-paille 38818b60c5 Move from llvm::makeArrayRef to ArrayRef deduction guides - llvm/ part
Use deduction guides instead of helper functions.

The only non-automatic changes have been:

1. ArrayRef(some_uint8_pointer, 0) needs to be changed into ArrayRef(some_uint8_pointer, (size_t)0) to avoid an ambiguous call with ArrayRef((uint8_t*), (uint8_t*))
2. CVSymbol sym(makeArrayRef(symStorage)); needed to be rewritten as CVSymbol sym{ArrayRef(symStorage)}; otherwise the compiler is confused and thinks we have a (bad) function prototype. There was a few similar situation across the codebase.
3. ADL doesn't seem to work the same for deduction-guides and functions, so at some point the llvm namespace must be explicitly stated.
4. The "reference mode" of makeArrayRef(ArrayRef<T> &) that acts as no-op is not supported (a constructor cannot achieve that).

Per reviewers' comment, some useless makeArrayRef have been removed in the process.

This is a follow-up to https://reviews.llvm.org/D140896 that introduced
the deduction guides.

Differential Revision: https://reviews.llvm.org/D140955
2023-01-05 14:11:08 +01:00
Gulfem Savrun Yeniceri 1ae7d83803 [profile] Add binary ids into indexed profiles
This patch adds support for including binary ids in an indexed profile.
It adds a new field into the header that points to the offset of the
binary id section. The binary id section consists of a size of the
section, and a list of binary ids (if they are present) that consist
of two parts: length and data.

This patch guarantees that indexed profile is backwards compatible
after adding binary ids.

Differential Revision: https://reviews.llvm.org/D135929
2022-12-29 18:46:56 +00:00
Gulfem Savrun Yeniceri 59b3d8f1db Revert "[profile] Add binary ids into indexed profiles"
This reverts commit 7734053fd9
because it broke powerpc64 bot:
https://lab.llvm.org/buildbot#builders/231/builds/6229
2022-12-14 21:48:41 +00:00
Gulfem Savrun Yeniceri 7734053fd9 [profile] Add binary ids into indexed profiles
This patch adds support for including binary ids in an indexed profile.
It adds a new field into the header that points to the offset of the
binary id section. The binary id section consists of a size of the
section, and a list of binary ids (if they are present) that consist
of two parts: length and data.

This patch guarantees that indexed profile is backwards compatible
after adding binary ids.

Differential Revision: https://reviews.llvm.org/D135929
2022-12-14 20:26:36 +00:00
Nathan James 6aa050a690 Reland "[llvm][NFC] Use c++17 style variable type traits"
This reverts commit 632a389f96.

This relands commit
1834a310d0.

Differential Revision: https://reviews.llvm.org/D137493
2022-11-08 14:15:15 +00:00
Nathan James 632a389f96 Revert "[llvm][NFC] Use c++17 style variable type traits"
This reverts commit 1834a310d0.
2022-11-08 13:11:41 +00:00
Nathan James 1834a310d0 [llvm][NFC] Use c++17 style variable type traits
This was done as a test for D137302 and it makes sense to push these changes

Reviewed By: dblaikie

Differential Revision: https://reviews.llvm.org/D137493
2022-11-08 12:22:52 +00:00
Alex Brachet 443e2a10f6 Reland "[PGO] Make emitted symbols hidden"
This was reverted because it was breaking when targeting Darwin which
tried to export these symbols which are now hidden. It should be safe
to just stop attempting to export these symbols in the clang driver,
though Apple folks will need to change their TAPI allow list described
in the commit where these symbols were originally exported
https://github.com/llvm/llvm-project/commit/f5380185623be243ba0f1b18d4bd594ac5cc7163

Then reverted again because it broke tests on MacOS, they should be
fixed now.

Bug: https://github.com/llvm/llvm-project/issues/58265

Differential Revision: https://reviews.llvm.org/D135340
2022-10-26 17:13:05 +00:00
Nico Weber 76745d2b58 Revert "[PGO] Make emitted symbols hidden"
This reverts commit 04877284b4.
Looks like this is still breaking the test
Profile-x86_64 :: instrprof-darwin-dead-strip.c
(see comment on https://reviews.llvm.org/D135340).
2022-10-25 08:54:47 -04:00
Alex Brachet 04877284b4 [PGO] Make emitted symbols hidden
This was reverted because it was breaking when targeting Darwin which
tried to export these symbols which are now hidden. It should be safe
to just stop attempting to export these symbols in the clang driver,
though Apple folks will need to change their TAPI allow list described
in the commit where these symbols were originally exported
https://github.com/llvm/llvm-project/commit/f5380185623be243ba0f1b18d4bd594ac5cc7163

Bug: https://github.com/llvm/llvm-project/issues/58265

Differential Revision: https://reviews.llvm.org/D135340
2022-10-24 19:05:10 +00:00
Zequan Wu 82035ec777 Revert "[PGO] Make emitted symbols hidden"
This reverts commit ecac223b0e.

The commit causes instrprof-darwin-dead-strip.c to fail on mac.
2022-10-14 15:23:26 -07:00
Alex Brachet ecac223b0e [PGO] Make emitted symbols hidden
This was reverted because it was breaking when targeting Darwin which
tried to export these symbols which are now hidden. It should be safe
to just stop attempting to export these symbols in the clang driver,
though Apple folks will need to change their TAPI allow list described
in the commit where these symbols were originally exported
https://github.com/llvm/llvm-project/commit/f5380185623be243ba0f1b18d4bd594ac5cc7163

Bug: https://github.com/llvm/llvm-project/issues/58265

Differential Revision: https://reviews.llvm.org/D135340
2022-10-13 19:47:15 +00:00
Alex Brachet deb82d4a20 Revert "[PGO] Make emitted symbols hidden"
This reverts commit 4ea1a647ff.

This breaks on Darwin which tries to export these symbols
https://github.com/llvm/llvm-project/blob/ebb258d3b0785f6dcc65e1f277d0690891ddc94d/clang/lib/Driver/ToolChains/Darwin.cpp#L1363

I'll try to reland which that removed and approval from
Apple folks.
2022-10-10 14:37:59 +00:00
Alex Brachet 4ea1a647ff [PGO] Make emitted symbols hidden
Differential Revision: https://reviews.llvm.org/D135340
2022-10-06 18:28:16 +00:00
Fangrui Song 367997d0d6 [Support] Rename llvm::compression::{zlib,zstd}::uncompress to more appropriate decompress
This improves consistency with other places (e.g. llvm::compression::decompress,
llvm::object::Decompressor::decompress, llvm-objcopy).
Note: when zstd::uncompress was added, we noticed that the API `ZSTD_decompress`
is fine while the zlib API `uncompress` is a misnomer.
2022-09-17 12:35:17 -07:00
Rong Xu d7ef0c3970 [llvm-profdata] Improve profile supplementation
Current implementation promotes a non-cold function in the SampleFDO profile
into a hot function in the FDO profile. This is too aggressive. This patch
promotes a hot functions in the SampleFDO profile into a hot function, and a
warm function in SampleFDO into a warm function in FDO.

Differential Revision: https://reviews.llvm.org/D132601
2022-08-29 16:50:42 -07:00
Fangrui Song de9d80c1c5 [llvm] LLVM_FALLTHROUGH => [[fallthrough]]. NFC
With C++17 there is no Clang pedantic warning or MSVC C5051.
2022-08-08 11:24:15 -07:00
Fangrui Song e690137dde [Support] Change compression::zlib::{compress,uncompress} to use uint8_t *
It's more natural to use uint8_t * (std::byte needs C++17 and llvm has
too much uint8_t *) and most callers use uint8_t * instead of char *.
The functions are recently moved into `llvm::compression::zlib::`, so
downstream projects need to make adaption anyway.
2022-07-13 16:26:54 -07:00
Nicolai Hähnle ede600377c ManagedStatic: remove many straightforward uses in llvm
(Reapply after revert in e9ce1a5880 due to
Fuchsia test failures. Removed changes in lib/ExecutionEngine/ other
than error categories, to be checked in more detail and reapplied
separately.)

Bulk remove many of the more trivial uses of ManagedStatic in the llvm
directory, either by defining a new getter function or, in many cases,
moving the static variable directly into the only function that uses it.

Differential Revision: https://reviews.llvm.org/D129120
2022-07-10 10:29:15 +02:00
Nicolai Hähnle e9ce1a5880 Revert "ManagedStatic: remove many straightforward uses in llvm"
This reverts commit e6f1f06245.

Reverting due to a failure on the fuchsia-x86_64-linux buildbot.
2022-07-10 09:54:30 +02:00
Nicolai Hähnle e6f1f06245 ManagedStatic: remove many straightforward uses in llvm
Bulk remove many of the more trivial uses of ManagedStatic in the llvm
directory, either by defining a new getter function or, in many cases,
moving the static variable directly into the only function that uses it.

Differential Revision: https://reviews.llvm.org/D129120
2022-07-10 09:15:08 +02:00
Cole Kissane ea61750c35 [NFC] Refactor llvm::zlib namespace
* Refactor compression namespaces across the project, making way for a possible
  introduction of alternatives to zlib compression.
  Changes are as follows:
  * Relocate the `llvm::zlib` namespace to `llvm::compression::zlib`.

Reviewed By: MaskRay, leonardchan, phosek

Differential Revision: https://reviews.llvm.org/D128953
2022-07-08 11:19:07 -07:00
Fangrui Song 407c721ceb [Support] Change zlib::compress to return void
With a sufficiently large output buffer, the only failure is Z_MEM_ERROR.
Check it and call the noreturn report_bad_alloc_error if applicable.
resize_for_overwrite may call report_bad_alloc_error as well.

Now that there is no other error type, we can replace the return type with void
and simplify call sites.

Reviewed By: ikudrin

Differential Revision: https://reviews.llvm.org/D121512
2022-03-14 11:38:04 -07:00
Snehasish Kumar 0a4184909a Reland "[memprof] Extend the index prof format to include memory profiles."
This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.

The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.

This commit also includes the changes reviewed separately in D120093.

Differential Revision: https://reviews.llvm.org/D120103
2022-02-17 22:09:52 -08:00
Snehasish Kumar 19bdf44d85 Revert "Reland "[memprof] Extend the index prof format to include memory profiles.""
This reverts commit 807ba7aace.
2022-02-17 15:51:04 -08:00
Snehasish Kumar 807ba7aace Reland "[memprof] Extend the index prof format to include memory profiles."
This reverts commit 85355a560a.

This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.

The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.

Differential Revision: https://reviews.llvm.org/D118653
2022-02-17 13:14:17 -08:00
Snehasish Kumar a3beb34015 Reland "[InstrProf] Make the IndexedInstrProf header backwards compatible."
This reverts commit 9fd2cb21fb.

Fixes an issue on big endian systems where the format version
was not converted to little endian prior to passing to GET_VERSION.

Differential Revision: https://reviews.llvm.org/D118390
2022-02-17 11:40:32 -08:00
Snehasish Kumar 9fd2cb21fb Revert "[InstrProf] Make the IndexedInstrProf header backwards compatible."
This reverts commit 14cc41a020. [2/4]
2022-02-14 11:42:58 -08:00
Snehasish Kumar 85355a560a Revert "Reland "[memprof] Extend the index prof format to include memory profiles.""
This reverts commit de54e4ab78 [1/4]
2022-02-14 11:42:58 -08:00
Snehasish Kumar de54e4ab78 Reland "[memprof] Extend the index prof format to include memory profiles."
This reverts commit 0f73fb18ca.

Use llvm/Profile/MIBEntryDef.inc instead of relative path.

Generated the raw profile data with `-mllvm
-enable-name-compression=false` so that builbots where the reader is
built without zlib do not fail.

Also updated the test build instructions.
2022-02-14 10:52:13 -08:00
Snehasish Kumar 0f73fb18ca Revert "[memprof] Extend the index prof format to include memory profiles."
This reverts commit 43c2348c5b.

Buildbots are failing with an error on reading memprof testdata.
"Inputs/basic.profraw: profile uses zlib
compression but the profile reader was built without zlib support"

https://lab.llvm.org/buildbot/#/builders/16/builds/24490
2022-02-14 10:25:01 -08:00
Snehasish Kumar 43c2348c5b [memprof] Extend the index prof format to include memory profiles.
This patch adds support for optional memory profile information to be
included with and indexed profile. The indexed profile header adds a new
field which points to the offset of the memory profile section (if
present) in the indexed profile. For users who do not utilize this
feature the only overhead is a 64-bit offset in the header.

The memory profile section contains (1) profile metadata describing the
information recorded for each entry (2) an on-disk hashtable containing
the profile records indexed via llvm::md5(function_name). We chose to
introduce a separate hash table instead of the existing one since the
indexing for the instrumented fdo hash table is based on a CFG hash
which itself is perturbed by memprof instrumentation.

Differential Revision: https://reviews.llvm.org/D118653
2022-02-14 09:53:45 -08:00
Snehasish Kumar 14cc41a020 [InstrProf] Make the IndexedInstrProf header backwards compatible.
While the contents of the profile are backwards compatible the header
itself is not. For example, when adding new fields to the header results
in significant issues. This change adds allows for portable
instantiation of the header across indexed format versions.

Differential Revision: https://reviews.llvm.org/D118390
2022-02-14 09:53:45 -08:00
Ellis Hoag eea002a9c4 [InstrProf][NFC] Move function out of InstrProf.h
`createIRLevelProfileFlagVar()` seems to be only used in
`PGOInstrumentation.cpp` so we move it to that file. Then it can also
take advantage of directly using options rather than passing them as
arguments.

Reviewed By: kyulee, phosek

Differential Revision: https://reviews.llvm.org/D118097
2022-01-28 09:24:26 -08:00
Ellis Hoag 11d3074267 [InstrProf] Add single byte coverage mode
Use the llvm flag `-pgo-function-entry-coverage` to create single byte "counters" to track functions coverage. This mode has significantly less size overhead in both code and data because
  * We mark a function as "covered" with a store instead of an increment which generally requires fewer assembly instructions
  * We use a single byte per function rather than 8 bytes per block

The trade off of course is that this mode only tells you if a function has been covered. This is useful, for example, to detect dead code.

When combined with debug info correlation [0] we are able to create an instrumented Clang binary that is only 150M (the vanilla Clang binary is 143M). That is an overhead of 7M (4.9%) compared to the default instrumentation (without value profiling) which has an overhead of 31M (21.7%).

[0] https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D116180
2022-01-27 17:38:55 -08:00
Ellis Hoag 6d5239113c [InstrProf][Correlate] Improve error messages
Improve the error messages when using `llvm-profdata` to correlate profiles with debug info.

Reviewed By: kyulee, phosek

Differential Revision: https://reviews.llvm.org/D118166
2022-01-26 11:33:59 -08:00
Ellis Hoag 65d7fd0239 [Try2][InstrProf] Add Correlator class to read debug info
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).

This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

This was first landed in https://reviews.llvm.org/D114566 but had to be reverted due to build errors.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115915
2021-12-17 10:45:59 -08:00
Ellis Hoag bdc68ee70f Revert "[InstrProf] Add Correlator class to read debug info"
Also reverts an attempt to fix the build errors https://reviews.llvm.org/D115911

The original diff https://reviews.llvm.org/D114566 causes some build
errors that I need to investigate.

https://lab.llvm.org/buildbot/#/builders/118/builds/7037

This reverts commit 95946d2f85.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115913
2021-12-16 16:28:19 -08:00
Ellis Hoag 95946d2f85 [InstrProf] Add Correlator class to read debug info
Extend `llvm-profdata` to read in a `.proflite` file and also a debug info file to generate a normal `.profdata` profile. This reduces the binary size by 8.4% when building an instrumented Clang binary without value profiling (164 MB vs 179 MB).

This work is part of the "lightweight instrumentation" RFC: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D114566
2021-12-16 15:18:12 -08:00
Ellis Hoag 58d9c1aec8 [Try2][InstrProf] Attach debug info to counters
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.

Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

The original diff https://reviews.llvm.org/D114565 was reverted because of the `Instrumentation/InstrProfiling/debug-info-correlate.ll` test, which is fixed in this commit.

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D115693
2021-12-16 14:20:30 -08:00
Ellis Hoag c809da7d9c Revert "[InstrProf] Attach debug info to counters"
This reverts commit 800bf8ed29.

The `Instrumentation/InstrProfiling/debug-info-correlate.ll` test was
failing because I forgot the `llc` commands are architecture specific.
I'll follow up with a fix.

Differential Revision: https://reviews.llvm.org/D115689
2021-12-13 18:15:17 -08:00
Ellis Hoag 800bf8ed29 [InstrProf] Attach debug info to counters
Add the llvm flag `-debug-info-correlate` to attach debug info to instrumentation counters so we can correlate raw profile data to their functions. Raw profiles are dumped as `.proflite` files. The next diff enables `llvm-profdata` to consume `.proflite` and debug info files to produce a normal `.profdata` profile.

Part of the "lightweight instrumentation" work: https://groups.google.com/g/llvm-dev/c/r03Z6JoN7d4

Reviewed By: kyulee

Differential Revision: https://reviews.llvm.org/D114565
2021-12-13 17:51:22 -08:00