Commit Graph
402 Commits
Author SHA1 Message Date
Aliaksei Urbanski 5e26548403 [NFC] Fix "is is" typos (#206288)
These changes aim to fix common "is is" typos.
2026-07-06 18:07:45 +00:00
Simi Pallipurath 431c300137 [ARM][Libc] Fix ARM big-endian low-end inline_memset byte fills (#198777)
Fix inline_memset_arm_low_end to use the splatted 32-bit fill value for
all byte stores. This ensures that all stores, including unaligned and
trailing byte fills, write the requested repeated byte value on
big-endian ARM.

The low-end ARM memset path already builds value32 as value 0x01010101U
for aligned word/block stores, but its byte-wise prefix and tail
handling still passed the raw 8-bit value * widened to uint32_t. Use
value32 for the byte-wise alignment prefix and final tail loop as well.
This makes the low-end path endian-safe and consistent with
inline_memset_arm_mid_end, fixing incorrect NUL/truncated output in
printf-family formatting and direct memset tests on big-endian ARM
no-unaligned-access targets.
2026-06-17 14:25:38 +01:00
lntue f4076e1c18 [libc][math] Add gcc-7 to 9 compatibility for shared math. (#197868)
- Add more fine-grained constexpr annotations.
- Apply fine-grained constexpr annotations to `src/__support`,
`src/__support/FPUtil` folder.
- Remove or update constexpr annotations for functions in
`src/__support/math` folder.
- Fix or ignore old gcc's warnings.

Assisted-by: Gemini 3.1 Flash with step 3.
2026-05-21 13:24:30 -04:00
lntue a070a15671 [libc] Add config option to use memory builtin functions. (#197977)
Add a new CMake and C++ definition configuration option
`LIBC_CONF_USE_MEM_BUILTINS` to allow users to use compiler builtins for
memory utility functions (memcpy, memset, memmove, memcmp, and bcmp)
instead of LLVM libc's internal implementations. Main use-cases are:
- when users want to bring their own memory functions implementations
that are highly optimized for their targets
- improve portability by providing a fallback for targets for which LLVM
libc does not have memory utility implementations yet
- to be used for libc/shared functions and their testings, as we expect
libc/shared functions to provide their own memory functions.
2026-05-15 20:05:03 -04:00
Nico Weber aeea9650b3 [libc] Fix -Wshadow warnings in inline_strlen.h (#196855) 2026-05-11 14:24:45 -04:00
Victor Campos e455e6c9ec [libc] Add Annex K strnlen_s function (#186112)
This patch adds the `strnlen_s` function from Annex K.

In order to reduce duplication between `strnlen` and `strnlen_s`, the
common logic has been extracted to a new internal function which both
now call.

In addition to the function definition, the patch adds a unit test and a
fuzzing test.
2026-04-13 13:10:27 +01:00
Roland McGrath 71122d8694 [libc] Move LLVM_LIBC_IS_DEFINED macro to its own header (#190081)
This moves the LLVM_LIBC_IS_DEFINED macro to its own header is
__support/macros.  Its implementation leverages cpp::string_view
instead of rolling its own strcmp; this necessitated fixing
several missing constexpr in the string_view implementation.

The new __support/macros/macro-utils.h is also broken out to hold
the stringification macro and can be used in future for token
pasting shenanigans and other such generic macro machinery.
2026-04-02 14:13:51 -07:00
Zorojuro a9c6f38e8d [libc] Fixes all guard comments of libc (#188701)
This PR intends to fix ALL the wrong guard comments for libc


Script used:
[guard_checker](https://github.com/Sukumarsawant/guard_checker/blob/main/check_headers.py)
2026-03-26 15:23:49 -04:00
Jeff Bailey 4e290b3429 [libc][strings] Refactor load_aligned for cleaner endianness handling (#186360)
Replace the explicit `if constexpr` branching for big and little
endianness with compile-time calculated shift constants `VAL_SHIFT` and
`NEXT_SHIFT`. This simplifies the logic and reduces code duplication,
relying on the compiler to constant-fold the zero shifts into no-ops.
2026-03-24 20:40:22 +00:00
Ilya Tokar bb369f1c30 [libc][x86] Add Non-temporal code path for large memcpy (#187108)
Large memcopies are pretty rare, but are more common in ML workloads
(copying large matrixes/tensors, often to/from CPU host).

For large copies NTA stores can provide performance advantages for both
memcpy itself and the rest of the workload (by reducing cache
pollution). Other runtimes already have NTA path for large copies, so
add 1 to the llvm-libc.

Internal whole-program loadtests shows small, but statistically
significant improvement of 0.1%. ML specific bencahmrks showed 10-20%
performance gain, and fleetbench (https://github.com/google/fleetbench,
which has more up-to-date version of libc benchmarks) shows ~3% gain
(ns/byte for distributions taken from various applications).

```
[Memcpy_0]_L1      0.01950n ± 3%   0.01900n ± 5%       ~ (p=0.390 n=20)
[Memcpy_0]_L2      0.02300n ± 0%   0.02300n ± 0%       ~ (p=0.256 n=20)
[Memcpy_0]_LLC     0.1335n ± 1%    0.1310n ± 1%   -1.87% (p=0.000 n=20)
[Memcpy_0]_Cold    0.1540n ± 2%    0.1520n ± 1%   -1.30% (p=0.021 n=20)
[Memcpy_1]_L1      0.04300n ± 5%   0.04200n ± 2%  -2.33% (p=0.000 n=20)
[Memcpy_1]_L2      0.05000n ± 2%   0.04800n ± 0%  -4.00% (p=0.000 n=20)
[Memcpy_1]_LLC     0.2500n ± 2%    0.2390n ± 1%   -4.40% (p=0.000 n=20)
[Memcpy_1]_Cold    0.2750n ± 1%    0.2640n ± 1%   -4.00% (p=0.000 n=20)
[Memcpy_2]_L1      0.03800n ± 3%   0.03800n ± 3%       ~ (p=0.420 n=20)
[Memcpy_2]_L2      0.04400n ± 2%   0.04300n ± 0%  -2.27% (p=0.000 n=20)
[Memcpy_2]_LLC     0.2320n ± 1%    0.2220n ± 1%   -4.31% (p=0.000 n=20)
[Memcpy_2]_Cold    0.2565n ± 1%    0.2460n ± 1%   -4.09% (p=0.000 n=20)
[Memcpy_3]_L1      0.1380n ± 1%    0.1355n ± 2%        ~ (p=0.095 n=20)
[Memcpy_3]_L2      0.1490n ± 1%    0.1430n ± 1%   -4.03% (p=0.000 n=20)
[Memcpy_3]_LLC     0.7955n ± 1%    0.7450n ± 0%   -6.35% (p=0.000 n=20)
[Memcpy_3]_Cold    0.8495n ± 1%    0.7935n ± 0%   -6.59% (p=0.000 n=20)
[Memcpy_4]_L1      0.04000n ± 3%   0.03900n ± 3%       ~ (p=0.466 n=20)
[Memcpy_4]_L2      0.04500n ± 2%   0.04400n ± 2%       ~ (p=0.130 n=20)
[Memcpy_4]_LLC     0.2040n ± 1%    0.1950n ± 1%   -4.41% (p=0.000 n=20)
[Memcpy_4]_Cold    0.2240n ± 1%    0.2150n ± 1%   -4.02% (p=0.000 n=20)
[Memcpy_5]_L1      0.05800n ± 3%   0.06050n ± 1%  +4.31% (p=0.000 n=20)
[Memcpy_5]_L2      0.06400n ± 0%   0.06400n ± 2%   0.00% (p=0.004 n=20)
[Memcpy_5]_LLC     0.3320n ± 1%    0.3140n ± 1%   -5.42% (p=0.000 n=20)
[Memcpy_5]_Cold    0.3620n ± 1%    0.3430n ± 0%   -5.25% (p=0.000 n=20)
[Memcpy_6]_L1      0.05700n ± 2%   0.05750n ± 3%       ~ (p=0.403 n=20)
[Memcpy_6]_L2      0.06500n ± 0%   0.06250n ± 1%  -3.85% (p=0.000 n=20)
[Memcpy_6]_LLC     0.3410n ± 1%    0.3205n ± 1%   -6.01% (p=0.000 n=20)
[Memcpy_6]_Cold    0.3670n ± 1%    0.3470n ± 1%   -5.45% (p=0.000 n=20)
[Memcpy_7]_L1      0.05900n ± 2%   0.05900n ± 2%       ~ (p=0.296 n=20)
[Memcpy_7]_L2      0.06400n ± 2%   0.06400n ± 0%       ~ (p=0.327 n=20)
[Memcpy_7]_LLC     0.3145n ± 1%    0.2965n ± 1%   -5.72% (p=0.000 n=20)
[Memcpy_7]_Cold    0.3410n ± 1%    0.3220n ± 0%   -5.57% (p=0.000 n=20)
[Memcpy_8]_L1      0.03600n ± 3%   0.03600n ± 3%       ~ (p=0.804 n=20)
[Memcpy_8]_L2      0.04200n ± 0%   0.04100n ± 2%  -2.38% (p=0.000 n=20)
[Memcpy_8]_LLC     0.2210n ± 1%    0.2090n ± 1%   -5.43% (p=0.000 n=20)
[Memcpy_8]_Cold    0.2415n ± 1%    0.2300n ± 1%   -4.76% (p=0.000 n=20)
geomean            0.1184n         0.1148n        -3.03%
```
2026-03-20 15:36:08 -04:00
Simon Tatham 317972681c [libc] Fix load_aligned big-endian handling. (#185937)
The variadic template helper `load_aligned` performs a specific case of
an unaligned integer load, by loading a sequence of integers from memory
at addresses expected to be aligned, and glues the results back together
with shifts and ORs into an output.

The implementation works by performing the first load, recursing on a
shorter parameter type list for the rest, and recombining via

  first | (rest << size_of_first)           // if little-endian
  (first << size_of_first) | rest           // if big-endian

But the big-endian case is wrong: it should shift left by the size of
the _rest_ of the types, not the size of the first. In the case where
you load 8, 16 and 8 bits from an odd address, you want

  (first_byte << 24) | (middle_halfword << 8) | (last_byte)

but in fact we were calculating

  (first_byte << 8) | (middle_halfword << 16) | (last_byte)

leading to three out of every four bytes being permuted in a memcpy from
an odd to an even address.
2026-03-13 08:59:55 +00:00
Jeff Bailey 7a85921a7d [libc] Use unsigned char in strcmp, strncmp, and strcoll comparisons (#185393)
According to section 7.24.1 of the C standard, character comparison in
string functions must be performed as if the characters had the type
`unsigned char`.

The previous implementations of `strcmp`, `strncmp`, and `strcoll` were
doing a direct subtraction of `char` values. On platforms where `char`
is signed, this resulted in incorrect negative values being returned
when characters exceeding 127 were being compared.

This patch fixes the comparison functions to explicitly cast the
character values to `unsigned char` prior to computing their difference.
It also adds regression tests to ensure the comparison behaves correctly
for ASCII values greater than 127.
2026-03-09 18:19:25 +00:00
Sterling-Augustine ecee70e210 Implement vector version of memchr, and dispatch to same (#177711)
As in the description. 

This implementation shares quite a bit of code with the wide-read
versions of string_length.
2026-02-13 12:38:19 -08:00
Schrodinger ZHU YifanandGuillaume Chatelet 55174f936b [libc][SVE] add sve handling for memcpy with count less than 32b (#167446)
Add SVE optimization for AArch64 architectures. The idea is to use
predicate registers to avoid branching.
Microbench in repo shows considerable improvements on NV GB10 (locked on
largest X925):

```
======================================================================
BENCHMARK STATISTICS (time in nanoseconds)
======================================================================

memcpy_Google_A:
  Old - Mean: 3.1257 ns, Median: 3.1162 ns
  New - Mean: 2.8402 ns, Median: 2.8265 ns
  Improvement: +9.14% (mean), +9.30% (median)

memcpy_Google_B:
  Old - Mean: 2.3171 ns, Median: 2.3159 ns
  New - Mean: 1.6589 ns, Median: 1.6593 ns
  Improvement: +28.40% (mean), +28.35% (median)

memcpy_Google_D:
  Old - Mean: 8.7602 ns, Median: 8.7645 ns
  New - Mean: 8.4307 ns, Median: 8.4308 ns
  Improvement: +3.76% (mean), +3.81% (median)

memcpy_Google_L:
  Old - Mean: 1.7137 ns, Median: 1.7091 ns
  New - Mean: 1.4530 ns, Median: 1.4553 ns
  Improvement: +15.22% (mean), +14.85% (median)

memcpy_Google_M:
  Old - Mean: 1.9823 ns, Median: 1.9825 ns
  New - Mean: 1.4826 ns, Median: 1.4840 ns
  Improvement: +25.20% (mean), +25.15% (median)

memcpy_Google_Q:
  Old - Mean: 1.6812 ns, Median: 1.6784 ns
  New - Mean: 1.1538 ns, Median: 1.1517 ns
  Improvement: +31.37% (mean), +31.38% (median)

memcpy_Google_S:
  Old - Mean: 2.1816 ns, Median: 2.1786 ns
  New - Mean: 1.6297 ns, Median: 1.6287 ns
  Improvement: +25.29% (mean), +25.24% (median)

memcpy_Google_U:
  Old - Mean: 2.2851 ns, Median: 2.2825 ns
  New - Mean: 1.7219 ns, Median: 1.7187 ns
  Improvement: +24.65% (mean), +24.70% (median)

memcpy_Google_W:
  Old - Mean: 2.0408 ns, Median: 2.0361 ns
  New - Mean: 1.5260 ns, Median: 1.5252 ns
  Improvement: +25.23% (mean), +25.09% (median)

uniform_384_to_4096:
  Old - Mean: 26.9067 ns, Median: 26.8845 ns
  New - Mean: 26.8083 ns, Median: 26.8149 ns
  Improvement: +0.37% (mean), +0.26% (median)
```
The beginning of the memcpy function looks like the following:
```
Dump of assembler code for function _ZN22__llvm_libc_22_0_0_git6memcpyEPvPKvm:
   0x0000000000001340 <+0>:     cbz     x2, 0x143c <_ZN22__llvm_libc_22_0_0_git6memcpyEPvPKvm+252>
   0x0000000000001344 <+4>:     cbz     x0, 0x1440 <_ZN22__llvm_libc_22_0_0_git6memcpyEPvPKvm+256>
   0x0000000000001348 <+8>:     cbz     x1, 0x1444 <_ZN22__llvm_libc_22_0_0_git6memcpyEPvPKvm+260>
   0x000000000000134c <+12>:    subs    x8, x2, #0x20
   0x0000000000001350 <+16>:    b.hi    0x1374 <_ZN22__llvm_libc_22_0_0_git6memcpyEPvPKvm+52>  // b.pmore
   0x0000000000001354 <+20>:    rdvl    x8, #1
   0x0000000000001358 <+24>:    whilelo p0.b, xzr, x2
   0x000000000000135c <+28>:    ld1b    {z0.b}, p0/z, [x1]
   0x0000000000001360 <+32>:    whilelo p1.b, x8, x2
   0x0000000000001364 <+36>:    ld1b    {z1.b}, p1/z, [x1, #1, mul vl]
   0x0000000000001368 <+40>:    st1b    {z0.b}, p0, [x0]
   0x000000000000136c <+44>:    st1b    {z1.b}, p1, [x0, #1, mul vl]
   0x0000000000001370 <+48>:    ret
```

---------

Co-authored-by: Guillaume Chatelet <chatelet.guillaume@gmail.com>
2026-02-03 10:26:44 -05:00
google-yfyang 8206c81433 [libc] Use generic/builtin.h for Emscripten in memory utils (#177474)
WASM supports bulk memory operations and can benefit from using
generic/builtin.h.
2026-01-23 13:48:32 -06:00
Roland McGrath 2e16cadd56 [libc] Split out src/__support/alloc-checker.h (#173104)
This moves the libc-internal AllocChecker API out of
src/__support/CPP/new.h and updates CPP/README.md to state the
intent to keep src/__support/CPP and the LIBC_NAMESPACE::cpp
namespace a "pure" subset of standard C++ API polyfills.
2025-12-22 11:04:33 -08:00
Sterling-Augustine a1eff73a8e Include inline_strlen.h on aarch64 only if the target has vector instrucions (#170892) 2025-12-05 13:11:19 -08:00
Sterling-Augustine ed7e66a41d Reland Refactor WIDE_READ to allow finer control over high-performance function selection (#165613) (#170738)
[Previous commit had an incorrect default case when
FIND_FIRST_CHARACTER_WIDE_READ_IMPL was not specified in config.json.
This PR is identical to that one with one line fixed.]

As we implement more high-performance string-related functions, we have
found a need for better control over their selection than the big-hammer
LIBC_CONF_STRING_LENGTH_WIDE_READ. For example, I have a memchr
implementation coming, and unless I implement it in every variant, a
simple binary value doesn't work.

This PR makes gives finer-grained control over high-performance
functions than the generic LIBC_CONF_UNSAFE_WIDE_READ option. For any
function they like, the user can now select one of four implementations
at build time:

1. element, which reads byte-by-byte (or wchar by wchar)
2. wide, which reads by unsigned long
3. generic, which uses standard clang vector implemenations, if
available
4. arch, which uses an architecture-specific implemenation

(Reading the code carefully, you may note that a user can actually
specify any namespace they want, so we aren't technically limited to
those 4.)

We may also want to switch from command-line #defines as it is currently
done, to something more like
llvm-project/llvm/include/llvm/Config/llvm-config.h.cmake, and
complexity out of the command-line. But that's a future problem.
2025-12-04 15:58:14 -08:00
Sterling-Augustine 4e5b114ce2 Revert "Refactor WIDE_READ to allow finer control over high-performance function selection" (#170717)
Reverts llvm/llvm-project#165613

Breaks build bot
2025-12-04 10:34:46 -08:00
Sterling-Augustine 8701c2a910 Refactor WIDE_READ to allow finer control over high-performance function selection (#165613)
[This is more of a straw-proposal than a ready-for-merging PR. I got
started thinking about what this might look like, and ended up just
implementing something as a proof-of-concept. Totally open to other
methods an ideas.]

As we implement more high-performance string-related functions, we have
found a need for better control over their selection than the big-hammer
LIBC_CONF_STRING_LENGTH_WIDE_READ. For example, I have a memchr
implementation coming, and unless I implement it in every variant, a
simple binary value doesn't work.

This PR makes gives finer-grained control over high-performance
functions than the generic LIBC_CONF_UNSAFE_WIDE_READ option. For any
function they like, the user can now select one of four implementations
at build time:

1. element, which reads byte-by-byte (or wchar by wchar)
2. wide, which reads by unsigned long
3. generic, which uses standard clang vector implemenations, if
available
4. arch, which uses an architecture-specific implemenation

(Reading the code carefully, you may note that a user can actually
specify any namespace they want, so we aren't technically limited to
those 4.)

We may also want to switch from command-line #defines as it is currently
done, to something more like
llvm-project/llvm/include/llvm/Config/llvm-config.h.cmake, and
#including the resulting file, which would move quite a bit of
complexity out of the command-line. But that's a future problem.
2025-12-04 10:21:53 -08:00
Schrodinger ZHU Yifan 8751f26a1b [libc] add an SVE implementation of strlen (#167259)
This PR creates an SVE-based implementation for strlen by translating
from the AOR code in tree. Microbenchmark shows improvements against
NEON when N>=64. Although both implementations fall behind glibc by a
large margin,
this may be a good start point to explore SVE implementations.

Together with the PR:

1. Added two more tests of strlen with special nul symbols.
2. Added strlen's fuzzer and fix a typo in previous heap fuzzer.

```
=== strlen(16 bytes) ===
libc: 1.56115 ns/call, 9.54499 GiB/s
neon: 1.59393 ns/call, 9.34867 GiB/s
sve: 1.66097 ns/call, 8.97134 GiB/s

=== strlen(64 bytes) ===
libc: 2.06967 ns/call, 28.7991 GiB/s
neon: 2.59914 ns/call, 22.9325 GiB/s
sve: 2.58628 ns/call, 23.0465 GiB/s

=== strlen(256 bytes) ===
libc: 3.74165 ns/call, 63.7202 GiB/s
neon: 8.98243 ns/call, 26.5428 GiB/s
sve: 7.36426 ns/call, 32.3751 GiB/s

=== strlen(1024 bytes) ===
libc: 10.5327 ns/call, 90.5438 GiB/s
neon: 34.363 ns/call, 27.7529 GiB/s
sve: 26.9329 ns/call, 35.4092 GiB/s

=== strlen(4096 bytes) ===
libc: 37.7304 ns/call, 101.104 GiB/s
neon: 145.911 ns/call, 26.144 GiB/s
sve: 103.208 ns/call, 36.9612 GiB/s

=== strlen(1048576 bytes) ===
libc: 9623.4 ns/call, 101.478 GiB/s
neon: 36138.2 ns/call, 27.023 GiB/s
sve: 26605.6 ns/call, 36.7051 GiB/s
```
2025-11-10 19:15:34 -05:00
Sterling-Augustine 7ff8a51754 [libc] Fix stale char_ptr for find_first_character_wide read (#166594)
On exit from the loop, char_ptr had not been updated to match block_ptr,
resulting in erroneous results. Moving all updates out of the loop fixes
that.

Adjust derefences to always be inside bounds checks.
2025-11-06 06:48:14 -08:00
Alexey Samsonov 120689e466 [libc] Migrate ctype_utils to use char instead of int where applicable. (#166225)
Functions like isalpha / tolower can operate on chars internally. This
allows us to get rid of unnecessary casts and open a way to creating
wchar_t overloads with the same names (e.g. for isalpha), that would
simplify templated code for conversion functions (see
315dfe5865).

Add the int->char converstion to public entrypoints implementation
instead. We also need to introduce bounds check on the input argument
values - these functions' behavior is unspecified if the argument is
neither EOF nor fits in "unsigned char" range, but the tests we've had
verified that they always return false for small negative values. To
preserve this behavior, cover it explicitly.
2025-11-05 11:02:28 -08:00
Sterling-Augustine 7d1538cd3d Move LIBC_CONF_STRING_UNSAFE_WIDE_READ to top-level libc-configuration (#165046)
This options sets a compile option when building sources inside the
string directory, and this option affects string_utils.h. But
string_utils.h is #included from more places than just the string
directory (such as from __support/CPP/string.h), leading to both
narrow-reads in those cases, but more seriously, ODR violations when the
two different string_length implementations are included int he same
program.

Having this option at the top level avoids this problem.
2025-10-27 14:39:46 -07:00
Guillaume Chatelet 57726bdca2 Revert "[libc] Implement branchless head-tail comparison for bcmp" (#162859)
Reverts llvm/llvm-project#107540

This PR demonstrated improvements on micro-benchmarks but the gains did
not seem to materialize in production. We are reverting this change for
now to get more data. This PR might be reintegrated later once we're
more confident in its effects.
2025-10-13 23:25:42 +02:00
Peter Collingbourne 7905ec387e [libc] Use UMAXV.4S to reduce bcmp result.
We can use UMAXV.4S to reduce the comparison result in a single
instruction. This improves performance by roughly 4% on Apple M1:

Summary
  bin/libc.src.string.bcmp_benchmark3 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10 ran
    1.01 ± 0.02 times faster than bin/libc.src.string.bcmp_benchmark3 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.01 ± 0.03 times faster than bin/libc.src.string.bcmp_benchmark3 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.01 ± 0.03 times faster than bin/libc.src.string.bcmp_benchmark3 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.01 ± 0.02 times faster than bin/libc.src.string.bcmp_benchmark2 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.02 ± 0.03 times faster than bin/libc.src.string.bcmp_benchmark2 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.03 ± 0.03 times faster than bin/libc.src.string.bcmp_benchmark2 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.03 ± 0.03 times faster than bin/libc.src.string.bcmp_benchmark2 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.05 ± 0.02 times faster than bin/libc.src.string.bcmp_benchmark1 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.05 ± 0.02 times faster than bin/libc.src.string.bcmp_benchmark1 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.05 ± 0.03 times faster than bin/libc.src.string.bcmp_benchmark1 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10
    1.05 ± 0.02 times faster than bin/libc.src.string.bcmp_benchmark1 --study-name="new bcmp" --sweep-mode --sweep-max-size=128 --output=/dev/null --num-trials=10

(1 = original, 2 = a variant of this patch that uses UMAXV.16B, 3 = this patch)

Reviewers: michaelrj-google, gchatelet, overmighty, SchrodingerZhu

Pull Request: https://github.com/llvm/llvm-project/pull/99260
2025-10-13 11:21:48 -07:00
Alexey Samsonov 5f0f4972c4 [libc] Unify and extend no_sanitize attributes for strlen. (#161316)
Fast strlen implementations (naive wide-reads, SIMD-based, and
x86_64/aarch64-optimized versions) all may perform
technically-out-of-bound reads, which leads to reports under ASan,
HWASan (on ARM machines), and also TSan (which also has the capability
to detect heap out-of-bound reads). So, we need to explicitly disable
instrumentation in all three cases.

Tragically, Clang didn't support `[[gnu::no_sanitize]]` syntax until
recently, and since we're supporting both GCC and Clang, we have to
revert to `__attribute__` syntax.
2025-10-01 17:26:22 -07:00
Schrodinger ZHU Yifan 38a4c9c639 [libc][msvc] fix mathlib build on WoA (#161258)
Fix build errors encountered when building math library on WoA.

1. Skip FEnv equality check for MSVC
2. Provide a placeholder type for vector types.
2025-09-29 15:40:21 -04:00
Joseph Huber e2d5efd59f [libc] Update the memory helper functions for simd types (#160174)
Summary:
This unifies the interface to just be a bunch of `load` and `store`
functions that optionally accept a mask / indices for gathers and
scatters with masks.

I had to rename this from `load` and `store` because it conflicts with
the other version in `op_generic`. I might just work around that with a
trait instead.
2025-09-26 08:18:00 -05:00
Joseph Huber b76dc2b9ba [libc] Clean up mask helpers after allowing implicit conversions (#158681)
Summary:
I landed a change in clang that allows integral vectors to implicitly
convert to boolean ones. This means I can simplify the interface and
remove the need to cast to bool on every use. Also do some other
cleanups of the traits.
2025-09-16 12:39:56 -05:00
lntue 7aad3830fb [libc] Some MSVC compatibility changes for src/string/memory_utils. (#158393) 2025-09-12 21:49:34 -04:00
lntue f019e2368b [libc] Change __builtin_memcpy to inline_memcpy. (#158345) 2025-09-12 17:57:08 -04:00
enh-google ded5f433de [libc] fixed signed char issues in strsep()/strtok()/strtok_r(). (#156705)
Also add the missing tests for all the related functions (even the ones
that were already right), and add the missing bazel build rules.
2025-09-03 14:00:38 -04:00
Joseph Huber 4aeb290083 [libc] Add more elementwise wrapper functions (#156515)
Summary:
Fills out some of the missing fundamental floating point operations.
These just wrap the elementwise builtin of the same name.
2025-09-02 21:53:03 -05:00
Joseph Huber eb7b162ea0 [libc] Implement generic SIMD helper 'simd.h' and implement strlen (#152605)
Summary:
This PR introduces a new 'simd.h' header that implements an interface
similar to the proposed `stdx::simd` in C++. However, we instead wrap
around the LLVM internal type. This makes heavy use of the clang vector
extensions and boolean vectors, instead using primitive vector types
instead of a class (many benefits to this).

I use this interface to implement a generic strlen implementation, but
propse we use this for math. Right now this requires a feature only
introduced in clang-22.
2025-09-02 06:11:06 -05:00
enh-google 71dd4e17dd [libc] fix strsep()/strtok()/strtok_r() "subsequent searches" behavior. (#154370)
These functions turned out to have the same bug that was in wcstok()
(fixed by 4fc9801), so add the missing tests and fix the code in a way
that matches wcstok().

Also fix incorrect test expectations in existing tests.

Also update the BUILD.bazel files to actually build the strsep() test.
2025-08-21 09:32:35 -04:00
Joseph Huber e90ce511e0 Disable asan on last wide string function 2025-08-21 08:30:47 -05:00
Joseph Huber 3596005148 Fix wide read defaults 2025-08-21 08:17:06 -05:00
Joseph Huber 6ac01d12d6 Reapply "[libc] Enable wide-read memory operations by default on Linux (#154602)" (#154640)
Reland afterr the sanitizer and arm32 builds complained.
2025-08-21 07:40:23 -05:00
Joseph Huber 27fc9671f9 Revert "[libc] Enable wide-read memory operations by default on Linux (#154602)"
This reverts commit c80d1483c6.
2025-08-20 17:27:13 -05:00
Joseph Huber c80d1483c6 [libc] Enable wide-read memory operations by default on Linux (#154602)
Summary:
This patch changes the linux build to use the wide reads on the memory
operations by default. These memory functions will now potentially read
outside of the bounds explicitly allowed by the current function. While
technically undefined behavior in the standard, plenty of C library
implementations do this. it will not cause a segmentation fault on linux
as long as you do not cross a page boundary, and because we are only
*reading* memory it should not have atomic effects.
2025-08-20 17:17:12 -05:00
Sterling-Augustine 317920063b Add vector-based strlen implementation for x86_64 and aarch64 (#152389)
These replace the default LIBC_CONF_STRING_UNSAFE_WIDE_READ
implementation
on x86_64 and aarch64.

These are substantially faster than both the character-by-character
implementation and the original unsafe_wide_read implementation. Some
below
I have been unable to performance-test the aarch64 version, but I
suspect
speedups similar to avx2.

```
Function: strlen
Variant:
                                    char	       wide              	ull	        sse2	                 avx2           	avx512
=============================================================================================================================================================
   length=1, alignment=1:        13.18	       20.47 (-55.24%)	       20.21 (-53.27%)	       32.50 (-146.54%)	       26.05 (-97.61%)	       18.03 (-36.74%)
   length=1, alignment=0:        12.80	       34.92 (-172.89%)	       20.01 (-56.39%)	       17.52 (-36.86%)	       17.78 (-38.92%)	       18.04 (-40.94%)
   length=2, alignment=2:         9.91	       19.02 (-91.95%)	       12.64 (-27.52%)	       11.06 (-11.59%)	        9.48 (  4.38%)	        9.48 (  4.34%)
   length=2, alignment=0:         9.56	       26.88 (-181.24%)	       12.64 (-32.31%)	       11.06 (-15.73%)	       11.06 (-15.72%)	       11.83 (-23.80%)
   length=3, alignment=3:         8.31	       10.45 (-25.84%)	        8.28 (  0.32%)	        8.28 (  0.36%)	        6.21 ( 25.28%)	        6.21 ( 25.24%)
   length=3, alignment=0:         8.39	       14.53 (-73.20%)	        8.28 (  1.33%)	        7.24 ( 13.69%)	        7.56 (  9.94%)	        7.25 ( 13.65%)
   length=4, alignment=4:         9.84	       21.76 (-121.24%)	       15.55 (-58.11%)	        6.57 ( 33.18%)	        5.02 ( 48.98%)	        6.00 ( 39.00%)
   length=4, alignment=0:         8.64	       13.70 (-58.51%)	        7.28 ( 15.73%)	        6.37 ( 26.31%)	        6.36 ( 26.36%)	        6.36 ( 26.36%)
   length=5, alignment=5:        11.85	       23.81 (-100.97%)	       12.17 ( -2.67%)	        5.68 ( 52.09%)	        4.87 ( 58.94%)	        6.48 ( 45.33%)
   length=5, alignment=0:        11.82	       13.64 (-15.42%)	        7.27 ( 38.45%)	        6.36 ( 46.15%)	        6.37 ( 46.11%)	        6.36 ( 46.14%)
   length=6, alignment=6:        10.50	       19.37 (-84.56%)	       13.64 (-29.93%)	        6.54 ( 37.71%)	        6.89 ( 34.35%)	        9.45 ( 10.01%)
   length=6, alignment=0:        14.96	       14.05 (  6.04%)	        6.49 ( 56.62%)	        5.68 ( 62.04%)	        5.68 ( 62.04%)	       13.15 ( 12.05%)
   length=7, alignment=7:        10.97	       18.02 (-64.35%)	       14.59 (-33.06%)	        6.36 ( 41.96%)	        5.46 ( 50.25%)	        5.46 ( 50.25%)
   length=7, alignment=0:        10.96	       15.76 (-43.77%)	       15.37 (-40.15%)	        6.96 ( 36.51%)	        5.68 ( 48.22%)	        7.04 ( 35.83%)
   length=4, alignment=0:         8.66	       13.69 (-58.02%)	        7.28 ( 16.00%)	        6.37 ( 26.44%)	        6.37 ( 26.52%)	        6.61 ( 23.74%)
   length=4, alignment=7:         8.87	       17.35 (-95.73%)	       12.18 (-37.39%)	        5.68 ( 35.94%)	        4.87 ( 45.11%)	        6.00 ( 32.36%)
   length=4, alignment=2:         8.67	       10.05 (-15.91%)	        7.28 ( 16.01%)	        7.37 ( 15.02%)	        5.46 ( 37.02%)	        5.47 ( 36.89%)
   length=2, alignment=2:         5.64	       10.01 (-77.64%)	        7.29 (-29.34%)	        6.37 (-13.04%)	        5.46 (  3.19%)	        5.46 (  3.19%)
   length=8, alignment=0:        12.78	       16.52 (-29.33%)	       18.27 (-43.00%)	       11.82 (  7.47%)	        9.83 ( 23.03%)	       11.46 ( 10.27%)
   length=8, alignment=7:        14.24	       17.30 (-21.49%)	       12.16 ( 14.59%)	        5.68 ( 60.14%)	        4.87 ( 65.83%)	        6.23 ( 56.28%)
   length=8, alignment=3:        12.34	       26.15 (-111.98%)	       12.20 (  1.14%)	        6.50 ( 47.34%)	        4.87 ( 60.54%)	        6.18 ( 49.94%)
   length=5, alignment=3:        10.95	       19.74 (-80.30%)	       12.17 (-11.11%)	        5.68 ( 48.16%)	        4.87 ( 55.56%)	        5.96 ( 45.55%)
   length=16, alignment=0:        20.33	       29.29 (-44.08%)	       36.18 (-77.97%)	        5.68 ( 72.06%)	        5.68 ( 72.08%)	       10.60 ( 47.86%)
   length=16, alignment=7:        19.29	       17.52 (  9.16%)	       12.98 ( 32.73%)	        7.05 ( 63.47%)	        4.87 ( 74.75%)	        6.23 ( 67.71%)
   length=16, alignment=4:        20.54	       25.18 (-22.56%)	       15.42 ( 24.92%)	        7.31 ( 64.43%)	        4.87 ( 76.29%)	        5.98 ( 70.88%)
   length=10, alignment=4:        14.59	       21.26 (-45.71%)	       12.17 ( 16.58%)	        5.68 ( 61.07%)	        4.87 ( 66.65%)	        6.00 ( 58.91%)
   length=32, alignment=0:        35.46	       22.00 ( 37.95%)	       16.22 ( 54.26%)	        7.32 ( 79.35%)	        5.68 ( 83.98%)	        7.01 ( 80.22%)
   length=32, alignment=7:        35.23	       24.14 ( 31.48%)	       16.22 ( 53.96%)	        7.30 ( 79.28%)	        8.76 ( 75.12%)	        6.14 ( 82.58%)
   length=32, alignment=5:        35.16	       28.56 ( 18.76%)	       16.22 ( 53.87%)	        7.30 ( 79.23%)	        6.77 ( 80.75%)	        9.82 ( 72.07%)
   length=21, alignment=5:        26.47	       27.66 ( -4.49%)	       15.04 ( 43.17%)	        6.90 ( 73.95%)	        4.87 ( 81.60%)	        6.04 ( 77.18%)
   length=64, alignment=0:        66.45	       25.16 ( 62.14%)	       22.70 ( 65.83%)	       12.99 ( 80.44%)	        7.47 ( 88.77%)	        8.70 ( 86.90%)
   length=64, alignment=7:        64.75	       27.78 ( 57.10%)	       22.72 ( 64.91%)	       10.85 ( 83.25%)	        7.46 ( 88.48%)	        8.68 ( 86.60%)
   length=64, alignment=6:        67.26	       28.58 ( 57.51%)	       22.70 ( 66.24%)	       11.26 ( 83.25%)	        9.46 ( 85.94%)	       13.90 ( 79.33%)
   length=42, alignment=6:        73.42	       27.97 ( 61.91%)	       19.46 ( 73.49%)	        8.92 ( 87.84%)	        6.49 ( 91.16%)	        6.00 ( 91.83%)
  length=128, alignment=0:       172.07	       39.18 ( 77.23%)	       35.68 ( 79.26%)	       13.02 ( 92.43%)	       12.98 ( 92.46%)	        9.76 ( 94.33%)
  length=128, alignment=7:       163.98	       43.79 ( 73.30%)	       36.03 ( 78.03%)	       15.68 ( 90.44%)	       11.35 ( 93.08%)	       10.51 ( 93.59%)
  length=128, alignment=7:       185.86	       40.27 ( 78.33%)	       36.04 ( 80.61%)	       13.78 ( 92.58%)	       11.35 ( 93.89%)	       10.49 ( 94.36%)
   length=85, alignment=7:       121.61	       55.66 ( 54.23%)	       32.34 ( 73.40%)	       13.88 ( 88.59%)	        7.30 ( 94.00%)	        8.72 ( 92.83%)
  length=256, alignment=0:       295.54	       66.48 ( 77.50%)	       61.63 ( 79.15%)	       19.54 ( 93.39%)	       12.97 ( 95.61%)	       12.45 ( 95.79%)
  length=256, alignment=7:       308.06	       78.92 ( 74.38%)	       61.63 ( 80.00%)	       22.90 ( 92.57%)	       12.97 ( 95.79%)	       13.23 ( 95.71%)
  length=256, alignment=8:       295.32	       65.83 ( 77.71%)	       61.62 ( 79.13%)	       23.19 ( 92.15%)	       12.97 ( 95.61%)	       13.50 ( 95.43%)
  length=170, alignment=8:       234.39	       48.79 ( 79.18%)	       43.79 ( 81.32%)	       16.22 ( 93.08%)	       13.97 ( 94.04%)	       10.48 ( 95.53%)
  length=512, alignment=0:       563.75	      116.89 ( 79.27%)	      114.99 ( 79.60%)	       62.71 ( 88.88%)	       19.58 ( 96.53%)	       17.76 ( 96.85%)
  length=512, alignment=7:       580.53	      120.91 ( 79.17%)	      114.47 ( 80.28%)	       37.75 ( 93.50%)	       19.55 ( 96.63%)	       18.68 ( 96.78%)
  length=512, alignment=9:       584.05	      128.35 ( 78.02%)	      114.74 ( 80.35%)	       39.09 ( 93.31%)	       19.76 ( 96.62%)	       18.71 ( 96.80%)
  length=341, alignment=9:       405.84	       90.87 ( 77.61%)	       78.79 ( 80.59%)	       28.77 ( 92.91%)	       14.60 ( 96.40%)	       14.15 ( 96.51%)
 length=1024, alignment=0:      1143.61	      247.03 ( 78.40%)	      243.70 ( 78.69%)	       75.59 ( 93.39%)	       67.02 ( 94.14%)	       28.99 ( 97.46%)
 length=1024, alignment=7:      1124.55	      267.87 ( 76.18%)	      259.16 ( 76.95%)	       64.96 ( 94.22%)	       33.05 ( 97.06%)	       30.91 ( 97.25%)
length=1024, alignment=10:      1459.58	      257.79 ( 82.34%)	      239.91 ( 83.56%)	       65.00 ( 95.55%)	       33.10 ( 97.73%)	       30.33 ( 97.92%)
 length=682, alignment=10:       732.89	      163.67 ( 77.67%)	      170.54 ( 76.73%)	       46.48 ( 93.66%)	       24.32 ( 96.68%)	       21.44 ( 97.07%)
 length=2048, alignment=0:      2141.96	      451.61 ( 78.92%)	      448.00 ( 79.08%)	      133.24 ( 93.78%)	       61.22 ( 97.14%)	       80.08 ( 96.26%)
 length=2048, alignment=7:      2145.05	      458.26 ( 78.64%)	      449.99 ( 79.02%)	      140.19 ( 93.46%)	       60.26 ( 97.19%)	       51.71 ( 97.59%)
length=2048, alignment=11:      2162.61	      463.37 ( 78.57%)	      448.07 ( 79.28%)	      140.29 ( 93.51%)	       59.51 ( 97.25%)	       51.59 ( 97.61%)
length=1365, alignment=11:      1439.74	      322.86 ( 77.58%)	      310.84 ( 78.41%)	      116.08 ( 91.94%)	       42.43 ( 97.05%)	       36.15 ( 97.49%)
 length=4096, alignment=0:      4278.68	      871.60 ( 79.63%)	      865.25 ( 79.78%)	      252.50 ( 94.10%)	      161.17 ( 96.23%)	       94.97 ( 97.78%)
 length=4096, alignment=7:      4253.01	      871.62 ( 79.51%)	      864.21 ( 79.68%)	      243.90 ( 94.27%)	      171.17 ( 95.98%)	       95.14 ( 97.76%)
length=4096, alignment=12:      4252.18	      879.66 ( 79.31%)	      863.68 ( 79.69%)	      244.26 ( 94.26%)	      185.36 ( 95.64%)	       93.61 ( 97.80%)
length=2730, alignment=12:      2868.22	      597.65 ( 79.16%)	      586.22 ( 79.56%)	      175.09 ( 93.90%)	      120.35 ( 95.80%)	      101.35 ( 96.47%)
    length=0, alignment=0:         4.87	        8.11 (-66.73%)	        6.49 (-33.34%)	        5.80 (-19.26%)	        5.68 (-16.67%)	        6.86 (-40.91%)
   length=32, alignment=0:        33.82	       22.36 ( 33.89%)	       17.03 ( 49.66%)	        7.30 ( 78.42%)	        5.68 ( 83.22%)	        7.50 ( 77.83%)
   length=64, alignment=0:        66.20	       26.76 ( 59.58%)	       23.22 ( 64.93%)	       12.99 ( 80.37%)	        7.34 ( 88.92%)	        8.44 ( 87.25%)
   length=96, alignment=0:       130.26	       31.62 ( 75.72%)	       30.00 ( 76.97%)	       11.39 ( 91.26%)	       10.54 ( 91.91%)	        8.68 ( 93.34%)
  length=128, alignment=0:       164.66	       39.05 ( 76.29%)	       35.68 ( 78.33%)	       13.07 ( 92.07%)	       12.97 ( 92.12%)	        9.59 ( 94.18%)
  length=160, alignment=0:       196.63	       45.18 ( 77.02%)	       42.16 ( 78.56%)	       14.65 ( 92.55%)	       10.87 ( 94.47%)	        9.31 ( 95.27%)
  length=192, alignment=0:       225.50	       52.71 ( 76.63%)	       49.61 ( 78.00%)	       16.22 ( 92.81%)	       11.36 ( 94.96%)	       11.08 ( 95.09%)
  length=224, alignment=0:       261.08	       57.57 ( 77.95%)	       55.82 ( 78.62%)	       17.84 ( 93.17%)	       12.16 ( 95.34%)	       11.51 ( 95.59%)
  length=256, alignment=0:       295.13	       65.56 ( 77.79%)	       62.59 ( 78.79%)	       19.46 ( 93.41%)	       13.12 ( 95.56%)	       12.33 ( 95.82%)
  length=288, alignment=0:       325.69	       72.16 ( 77.84%)	       69.20 ( 78.75%)	       21.08 ( 93.53%)	       13.94 ( 95.72%)	       12.32 ( 96.22%)
  length=320, alignment=0:       364.18	       78.78 ( 78.37%)	       75.69 ( 79.21%)	       22.71 ( 93.77%)	       14.70 ( 95.96%)	       14.46 ( 96.03%)
  length=352, alignment=0:       391.40	       84.87 ( 78.32%)	       82.15 ( 79.01%)	       24.50 ( 93.74%)	       15.62 ( 96.01%)	       14.27 ( 96.35%)
  length=384, alignment=0:       428.50	       91.43 ( 78.66%)	       88.70 ( 79.30%)	       26.16 ( 93.90%)	       17.29 ( 95.97%)	       15.04 ( 96.49%)
  length=416, alignment=0:       457.30	       98.23 ( 78.52%)	       95.02 ( 79.22%)	       27.81 ( 93.92%)	       17.22 ( 96.23%)	       15.05 ( 96.71%)
  length=448, alignment=0:       488.38	      104.52 ( 78.60%)	      101.87 ( 79.14%)	       31.22 ( 93.61%)	       18.07 ( 96.30%)	       16.89 ( 96.54%)
  length=480, alignment=0:       526.44	      109.61 ( 79.18%)	      108.11 ( 79.46%)	       31.11 ( 94.09%)	       18.88 ( 96.41%)	       17.10 ( 96.75%)
  length=512, alignment=0:       556.50	      117.29 ( 78.92%)	      113.78 ( 79.56%)	       62.57 ( 88.76%)	       19.88 ( 96.43%)	       17.80 ( 96.80%)
  length=576, alignment=0:       622.17	      152.93 ( 75.42%)	      127.58 ( 79.49%)	       39.34 ( 93.68%)	       21.31 ( 96.58%)	       19.99 ( 96.79%)
  length=640, alignment=0:       691.01	      142.56 ( 79.37%)	      161.78 ( 76.59%)	       39.20 ( 94.33%)	       22.98 ( 96.67%)	       20.13 ( 97.09%)
  length=704, alignment=0:       756.90	      156.31 ( 79.35%)	      176.19 ( 76.72%)	       45.03 ( 94.05%)	       24.82 ( 96.72%)	       22.33 ( 97.05%)
  length=768, alignment=0:       826.23	      193.17 ( 76.62%)	      188.41 ( 77.20%)	       50.81 ( 93.85%)	       27.46 ( 96.68%)	       23.25 ( 97.19%)
  length=832, alignment=0:       890.17	      204.81 ( 76.99%)	      201.61 ( 77.35%)	       53.77 ( 93.96%)	       27.73 ( 96.88%)	       25.06 ( 97.18%)
  length=896, alignment=0:       959.52	      217.89 ( 77.29%)	      213.86 ( 77.71%)	       57.99 ( 93.96%)	       29.53 ( 96.92%)	       26.29 ( 97.26%)
  length=960, alignment=0:      1024.52	      231.06 ( 77.45%)	      227.05 ( 77.84%)	       60.36 ( 94.11%)	       32.29 ( 96.85%)	       27.94 ( 97.27%)
 length=1024, alignment=0:      1086.71	      244.17 ( 77.53%)	      239.87 ( 77.93%)	       64.72 ( 94.04%)	       72.38 ( 93.34%)	       28.72 ( 97.36%)
 length=1152, alignment=0:      1231.48	      270.22 ( 78.06%)	      266.47 ( 78.36%)	       73.38 ( 94.04%)	       40.24 ( 96.73%)	       32.42 ( 97.37%)
 length=1280, alignment=0:      1349.29	      295.45 ( 78.10%)	      292.69 ( 78.31%)	      111.80 ( 91.71%)	       42.44 ( 96.85%)	       34.59 ( 97.44%)
 length=1408, alignment=0:      1487.13	      322.57 ( 78.31%)	      318.18 ( 78.60%)	       84.47 ( 94.32%)	       44.35 ( 97.02%)	       37.31 ( 97.49%)
 length=1536, alignment=0:      1623.52	      347.98 ( 78.57%)	      344.24 ( 78.80%)	      108.31 ( 93.33%)	       49.82 ( 96.93%)	       39.94 ( 97.54%)
 length=1664, alignment=0:      1748.88	      373.80 ( 78.63%)	      370.03 ( 78.84%)	      118.76 ( 93.21%)	       52.89 ( 96.98%)	       42.93 ( 97.55%)
 length=1792, alignment=0:      1886.22	      399.59 ( 78.82%)	      397.39 ( 78.93%)	      127.32 ( 93.25%)	       53.64 ( 97.16%)	       45.39 ( 97.59%)
 length=1920, alignment=0:      2018.37	      425.98 ( 78.89%)	      422.31 ( 79.08%)	      126.70 ( 93.72%)	       57.08 ( 97.17%)	       48.12 ( 97.62%)
 length=2048, alignment=0:      2167.09	      451.70 ( 79.16%)	      447.70 ( 79.34%)	      141.68 ( 93.46%)	       61.63 ( 97.16%)	       79.06 ( 96.35%)
 length=2304, alignment=0:      2422.03	      503.63 ( 79.21%)	      502.23 ( 79.26%)	      149.62 ( 93.82%)	       73.10 ( 96.98%)	       56.97 ( 97.65%)
 length=2560, alignment=0:      2678.68	      556.84 ( 79.21%)	      553.24 ( 79.35%)	      161.06 ( 93.99%)	      127.74 ( 95.23%)	       58.81 ( 97.80%)
 length=2816, alignment=0:      2941.95	      608.70 ( 79.31%)	      604.03 ( 79.47%)	      171.85 ( 94.16%)	       87.11 ( 97.04%)	       67.08 ( 97.72%)
 length=3072, alignment=0:      3229.89	      660.14 ( 79.56%)	      659.19 ( 79.59%)	      183.85 ( 94.31%)	      140.25 ( 95.66%)	       73.01 ( 97.74%)
 length=3328, alignment=0:      3496.08	      713.05 ( 79.60%)	      710.00 ( 79.69%)	      209.72 ( 94.00%)	      138.78 ( 96.03%)	       77.81 ( 97.77%)
 length=3584, alignment=0:      3756.52	      766.19 ( 79.60%)	      763.94 ( 79.66%)	      214.16 ( 94.30%)	      146.36 ( 96.10%)	       83.43 ( 97.78%)
 length=3840, alignment=0:      4017.15	      817.43 ( 79.65%)	      819.77 ( 79.59%)	      242.07 ( 93.97%)	      164.56 ( 95.90%)	       89.72 ( 97.77%)
 length=4096, alignment=0:      4281.59	      867.87 ( 79.73%)	      864.71 ( 79.80%)	      243.33 ( 94.32%)	      173.11 ( 95.96%)	       95.65 ( 97.77%)
 length=4608, alignment=0:      4810.30	      977.80 ( 79.67%)	      985.03 ( 79.52%)	      271.13 ( 94.36%)	      190.62 ( 96.04%)	      107.82 ( 97.76%)
 length=5120, alignment=0:      5380.16	     1075.77 ( 80.00%)	     1071.80 ( 80.08%)	      294.27 ( 94.53%)	      206.04 ( 96.17%)	      141.90 ( 97.36%)
 length=5632, alignment=0:      5925.70	     1195.61 ( 79.82%)	     1193.68 ( 79.86%)	      323.42 ( 94.54%)	      223.55 ( 96.23%)	      125.28 ( 97.89%)
 length=6144, alignment=0:      6402.20	     1285.52 ( 79.92%)	     1281.04 ( 79.99%)	      342.68 ( 94.65%)	      234.84 ( 96.33%)	      167.01 ( 97.39%)
 length=6656, alignment=0:      6997.01	     1387.32 ( 80.17%)	     1384.21 ( 80.22%)	      365.93 ( 94.77%)	      269.89 ( 96.14%)	      176.40 ( 97.48%)
 length=7168, alignment=0:      7454.76	     1492.10 ( 79.98%)	     1488.45 ( 80.03%)	      391.92 ( 94.74%)	      280.81 ( 96.23%)	      187.73 ( 97.48%)
 length=7680, alignment=0:      8163.34	     1608.43 ( 80.30%)	     1615.98 ( 80.20%)	      460.03 ( 94.36%)	      299.86 ( 96.33%)	      201.40 ( 97.53%)
```
2025-08-19 15:18:04 -07:00
Uzair Nawaz f26c0d00df [libc] Implemented wcsdup libc function (#150453)
Implemented wcsdup by templating internal strdup function
2025-07-24 11:29:40 -07:00
lntue 66603dd1f1 [libc][NFC] Add stdint.h proxy header to fix dependency issue with <stdint.h> includes. (#150303)
https://github.com/llvm/llvm-project/issues/149993
2025-07-23 20:19:52 -04:00
William Huynh 6932080866 [libc] Add dependency <stdint.h> to src/string/string_utils.h (#149849)
string_utils.h uses uintptr_t, and there seems to be no tracking of this
dependency. It seems upstream builds are unaffected but downstream this
is causing a lot of flaky builds.
2025-07-21 17:56:21 +01:00
Guillaume Chatelet e0cce5cdcb [libc] Improve Cortex memset and memcpy functions (#149044)
The code for `memcpy` is the same as in #148204 but it fixes the build
bot error by using `static_assert(cpp::always_false<decltype(access)>)`
instead of `static_assert(false)` (older compilers fails on
`static_assert(false)` in `constexpr` `else` bodies).

The code for `memset` is new and vastly improves performance over the
current byte per byte implementation.

Both `memset` and `memcpy` implementations use prefetching for sizes >=
64. This lowers a bit the performance for sizes between 64 and 256 but
improves throughput for greater sizes.
2025-07-17 10:15:43 +02:00
Guillaume Chatelet 999a8fb812 Revert "[libc][NFC] refactor Cortex memcpy code" (#149035)
Reverts llvm/llvm-project#148204

`libc-arm32-qemu-debian-dbg` is failing, reverting and investigating
2025-07-16 10:29:11 +02:00
Guillaume Chatelet 7c69c3b0ba [libc][NFC] refactor Cortex memcpy code (#148204)
This patch is in preparation for the Cortex `memset` implementation.
It improves the codegen by generating a prefetch for large sizes.
2025-07-16 10:06:18 +02:00
Guillaume Chatelet 7289b6789e [libc] Improve memcpy for ARM Cortex-M supporting unaligned accesses. (#144872)
This implementation has been compiled with the [pigweed toolchain](https://pigweed.dev/toolchain.html) and tested on:
 - Raspberry Pi Pico 2 with the following options\
`--target=armv8m.main-none-eabi`
`-march=armv8m.main+fp+dsp`
`-mcpu=cortex-m33` 
 - Raspberry Pi Pico with the following options\
`--target=armv6m-none-eabi`
`-march=armv6m`
`-mcpu=cortex-m0+` 

They both compile down to a little bit more than 200 bytes and are between 2 and 10 times faster than byte per byte copies.

For best performance the following options can be set in the `libc/config/baremetal/arm/config.json`
```
{
  "codegen": {
    "LIBC_CONF_KEEP_FRAME_POINTER": {
      "value": false
    }
  },
  "general": {
    "LIBC_ADD_NULL_CHECKS": {
      "value": false
    }
  }
}
```
2025-06-26 10:18:02 +02:00
Amy Huang c42912b8c9 Fix string_length function so that it always returns. (#144148)
Previously setting LIBC_COPT_STRING_UNSAFE_WIDE_READ would cause a build
error because there is a path in the ifdef that doesn't return anything.
2025-06-13 13:07:39 -07:00