Commit Graph
566 Commits
Author SHA1 Message Date
Chuanqi Xu 2203a4e6e0 [NFC] [Serialization] Improve AST serialization by reordering packed
bits and extract big bits from packed bits

Previously I tried to improve the size of .pcm files by introducing
packed bits. And I find we can improve it further by reordering the
bits.

The secret comes from the VBR format. We can find the formal definition
of VBR format in the doc of LLVM. The VBR format will be pretty
efficicent for small numbers.

For example, if we need to pack 8 bits into a value and the stored value
is 0xf0, the actual stored value will be 0b000111'110000, which takes 12
bits actually. However, if we changed the order to be 0x0f, then we
can store it as 0b001111, which takes 6 bits only now.

So we can improve the size by placing bits with lower probability to be
1 in the higher bits and extract bit bigs from the packed bits to make
it possible to be optimized by VBR.

After this patch, the size of std module becomes to 27.7MB from 28.1MB.
2023-12-21 16:35:20 +08:00
Chuanqi Xu 565e5e861f Recommit [NFC] [Serialization] Packing more bits and refactor AbbrevToUse
This patch tries to pack more bits into a value to reduce the size of
.pcm files. Also, after we introduced BitsPackers, it may slightly
better to adjust the way we use Abbrev.

After this patch, the size of the BMI for std module reduce from 28.94MB
to 28.1 MB.

This was reverted due to it broke the build of lldb. The reason that we
skip the serialization of a source location incorrectly. And this patch
now fixes that.
2023-12-21 10:30:12 +08:00
Augusto Noronha eccc1cca71 Revert "[NFC] [Serialization] Packing more bits and refactor AbbrevToUse"
This reverts commit 9cdb825a4f.
2023-12-15 14:43:25 -08:00
Fangrui Song ce1faec3ad [Serialization] Fix -Wpessimizing-move after 9cdb825a4f 2023-12-14 19:40:09 -08:00
Chuanqi Xu 9cdb825a4f [NFC] [Serialization] Packing more bits and refactor AbbrevToUse
This patch tries to pack more bits into a value to reduce the size of
.pcm files. Also, after we introduced BitsPackers, it may slightly
better to adjust the way we use Abbrev.

After this patch, the size of the BMI for std module reduce from 28.94MB
to 28.1 MB.
2023-12-15 11:12:52 +08:00
Chuanqi Xu 65b12a8af3 Recommit [NFC] [Serialization] Packing more bits
This patch tries to reduce the size of the BMIs by packing more bits
into an unsigned integer.

This patch was reverted due to buildbot failure report. But it should be
irrevelent after I took a double look. So I tried to recommit this NFC
change again.
2023-12-11 16:47:51 +08:00
Chuanqi Xu 9a46518630 Revert "[clang] Remove unused variable 'ExprDependenceBits' in ASTWriterDecl.cpp (NFC)"
This reverts commit 10951050b5.
This should be part of 8c33462781 to revert
9406ea3fe3 completely.
2023-12-11 14:15:16 +08:00
Chuanqi Xu 8c33462781 Revert "[NFC] [Serialization] Packing more bits"
This reverts commit 9406ea3fe3.

There are build bots complaining this. Revert it first to try to keep
the bots green.
2023-12-11 14:06:32 +08:00
Jie Fu 10951050b5 [clang] Remove unused variable 'ExprDependenceBits' in ASTWriterDecl.cpp (NFC)
llvm-project/clang/lib/Serialization/ASTWriterDecl.cpp:2342:12:
 error: unused variable 'ExprDependenceBits' [-Werror,-Wunused-variable]
 2342 |   unsigned ExprDependenceBits = llvm::BitWidth<ExprDependence>;
      |            ^~~~~~~~~~~~~~~~~~
1 error generated.
2023-12-11 10:45:56 +08:00
Chuanqi Xu 9406ea3fe3 [NFC] [Serialization] Packing more bits
This patch tries to reduce the size of the BMIs by packing more bits
into an unsigned integer.
2023-12-11 10:18:12 +08:00
Chuanqi Xu 0f7aaeb324 [C++20] [Modules] Allow export from language linkage
Close https://github.com/llvm/llvm-project/issues/71347

Previously I misread the concept of module purview. I thought if a
declaration attached to a unnamed module, it can't be part of the module
purview. But after the issue report, I recognized that module purview is
more of a concept about locations instead of semantics.

Concretely, the things in the language linkage after module declarations
can be exported.

This patch refactors `Module::isModulePurview()` and introduces some
possible code cleanups.
2023-11-09 17:44:41 +08:00
Vlad Serebrennikov dda8e3de35 [clang][NFC] Refactor ImplicitParamDecl::ImplicitParamKind
This patch converts `ImplicitParamDecl::ImplicitParamKind` into a scoped enum at namespace scope, making it eligible for forward declaring. This is useful for `preferred_type` annotations on bit-fields.
2023-11-06 12:01:09 +03:00
Vlad Serebrennikov edd690b02e [clang][NFC] Refactor TagTypeKind (#71160)
This patch converts TagTypeKind into scoped enum. Among other benefits,
this allows us to forward-declare it where necessary.
2023-11-03 21:45:39 +04:00
Chuanqi Xu 48be81e172 [NFC] [Serializer] Pack information in serializer (#69287)
Previously, the boolean values will occupy spaces that can contain
integers. It wastes the spaces especially if the boolean values are
serialized consecutively. The patch tries to pack such consecutive
boolean values (and enum values) so that we can save more spaces and so
the times.

Before the patch, we need 4.478s (in my machine) to build the std module
(https://libcxx.llvm.org/Modules.html) with 28712 bytes for size of the
BMI. After the patch, the time becomes to 4.374s and the size becomes to
27388 bytes for the size of the BMI.

This is intended to be a NFC patch.

This patch doesn't optimize all such cases. We can do it later after we
have consensus on this.
2023-11-03 21:59:44 +08:00
Vlad Serebrennikov 8775947633 [clang][NFC] Refactor clang::Linkage (#71049)
This patch introduces a new enumerator `Invalid = 0`, shifting other enumerators by +1. Contrary to how it might sound, this actually affirms status quo of how this enum is stored in `clang::Decl`:
```
  /// If 0, we have not computed the linkage of this declaration.
  /// Otherwise, it is the linkage + 1.
  mutable unsigned CacheValidAndLinkage : 3;
```
This patch makes debuggers to not be mistaken about enumerator stored in this bit-field. It also converts `clang::Linkage` to a scoped enum.
2023-11-02 20:57:29 +04:00
Vlad Serebrennikov 65761200ce [clang][NFC] Refactor LinkageSpecDecl::LanguageIDs
This patch converts `LinkageSpecDecl::LanguageIDs` into scoped enum, and moves it to namespace scope, so that it can be forward-declared where required.
2023-11-01 16:44:34 +03:00
Vlad Serebrennikov aaba3761db [clang][NFC] Refactor ObjCMethodDecl::ImplementationControl
This patch moves `ObjCMethodDecl::ImplementationControl` to a DeclBase.h so that it's complete at the point where corresponsing bit-field is declared. This patch also converts it to a scoped enum `clang::ObjCImplementationControl`.
2023-11-01 13:40:11 +03:00
Vlad Serebrennikov 50dec541f3 [clang][NFC] Refactor OMPDeclareReductionDecl::InitKind
This patch moves `OMPDeclareReductionDecl::InitKind` to DeclBase.h, so that it's complete at the point where corresponding bit-field is declared. This patch also converts it to scoped enum named `OMPDeclareReductionInitKind`
2023-11-01 12:40:13 +03:00
Vlad Serebrennikov b120fe8d32 [clang][NFC] Refactor ArgPassingKind
This patch moves `RecordDecl::ArgPassingKind` to DeclBase.h to namespace scope, so that it's complete at the time bit-field is declared.
2023-11-01 11:49:59 +03:00
Vlad Serebrennikov a6d0e8791e [clang][NFC] Refactor enums that hold size of Type and DeclContext bit-fields (#70296)
This patch refactor said enums to hold total size of a bit-field, and not just non-inherited bits. This brings `Type` and `DeclContext` in line with `Comment` and `Stmt`. It also makes it unnecessary to list all transitive bases of a bit-field as unnamed bit-fields, which makes it more friendly towards debuggers.
2023-10-26 19:22:52 +04:00
Krystian Stasiowski 3a3b84b180 [clang] remove ClassScopeFunctionSpecializationDecl (#66636)
This removes the `ClassScopeFunctionSpecializationDecl` `Decl` node, and
instead uses `DependentFunctionTemplateSpecializationInfo` to handle
such declarations. `DependentFunctionTemplateSpecializationInfo` is also
changed to store a `const ASTTemplateArgumentListInfo*` to be more in
line with `FunctionTemplateSpecializationInfo`.

This also changes `FunctionDecl::isFunctionTemplateSpecialization` to
return `true` for dependent specializations, and
`FunctionDecl::getTemplateSpecializationKind`/`FunctionDecl::getTemplateSpecializationKindForInstantiation`
to return `TSK_ExplicitSpecialization` for non-friend dependent
specializations (the same behavior as dependent class scope
`ClassTemplateSepcializationDecl` & `VarTemplateSepcializationDecl`).
2023-10-07 10:55:31 +04:00
Corentin Jabot af4751738d [C++] Implement "Deducing this" (P0847R7)
This patch implements P0847R7 (partially),
CWG2561 and CWG2653.

Reviewed By: aaron.ballman, #clang-language-wg

Differential Revision: https://reviews.llvm.org/D140828
2023-10-02 14:33:02 +02:00
Jens Massberg c2bf9baf59 Add a concept AST node.
This patch adds a concept AST node (`ConceptLoc`) and uses it at the corresponding places.

There are three objects that might have constraints via concepts:
`TypeConstraint`,  `ConceptSpecializationExpr` and `AutoTypeLoc`.
The first two inherit from `ConceptReference` while the latter has
the information about a possible constraint directly stored in `AutoTypeLocInfo`. It would be nice if the concept information would be stored the same way in all three cases.

Moreover the current structure makes it difficult to deal with these concepts. For example in Clangd accessing the locations of constraints of a `AutoTypeLoc` can only be done with quite ugly hacks.

So we think that it makes sense to create a new AST node for such concepts.

In details we propose the following:
- Rename `ConceptReference` to `ConceptLoc` (or something else what is approriate)
and make it the new AST node.
- `TypeConstraint` and `ConceptSpecializationExpr` do not longer inherit from `ConceptReference` but store a pointer to a `ConceptLoc`.
- `AutoTypeLoc` stores a pointer to `ConceptLoc` instead of storing the concept info in `AutoTypeLocInfo`.

This patch implements a first version of this idea which compiles and where the existing tests pass.
To make this patch as small as possible we keep the existing member functions to access concept data. Later these can be replaced by directly calling the corresponding functions of the `ConceptLoc`s.

Differential Revision: https://reviews.llvm.org/D155858
2023-08-31 10:22:21 +02:00
dingfei 91c4b5550e [clang] Update NumFunctionDeclBits for FunctionDeclBitfields
NumFunctionDeclBits is not updated when DeductionCandidateKind is
incremented.

Fixes https://github.com/llvm/llvm-project/issues/64171

Reviewed By: cor3ntin, balazske, aaron.ballman

Differential Revision: https://reviews.llvm.org/D158145
2023-08-17 20:28:41 +08:00
Yuanfang Chen 632dd6a4ca [Clang] Implements CTAD for aggregates P1816R0 and P2082R1
Differential Revision: https://reviews.llvm.org/D139837
2023-06-29 14:22:24 -07:00
Corentin Jabot 4676885270 [clang] Implement P2564 "consteval must propagate up"
Reviewed By: aaron.ballman, #clang-language-wg

Differential Revision: https://reviews.llvm.org/D151094
2023-06-07 20:45:36 +02:00
Chuanqi Xu 536b76e873 [NFC] [serialization] Refactor the outdated AbrrevToUse of VarDecl
The implementation and the comment of the AbrrevToUse of VarDecl
looks not consistent with the implementation. This patch refactors it.
2023-05-30 15:07:35 +08:00
Chuanqi Xu 5bbadec2d1 Recommit [C++20] [Modules] Don't load declaration eagerly for named modules
Close https://github.com/llvm/llvm-project/issues/61064.

The root cause of the issue is that we will deserilize some declarations
eagerly when reading the BMI. However, many declarations in the BMI are
not necessary for the importer. So it wastes a lot of time.

The new commit handles the MSVC's extension #pragma comment and #pragma
detect_mismatch to follow MSVC's behavior. See pr61783 for details.
2023-04-06 11:01:58 +08:00
Richard Smith 3242934191 Lazily deserialize default member initializers.
This is important to break deserialization cycles, where a lambda in a
default member initializer can refer to the field as its context
declaration, and the initializer of the field can refer back to the
lambda.

This is a follow-up to bc73ef0031, which applied the same fix to
variable declarations for the same reason.
2023-04-05 15:46:13 -07:00
Richard Smith bc73ef0031 PR60985: Fix merging of lambda closure types across modules.
Previously, distinct lambdas would get merged, and multiple definitions
of the same lambda would not get merged, because we attempted to
identify lambdas by their ordinal position within their lexical
DeclContext. This failed for lambdas within namespace-scope variables
and within variable templates, where the lexical position in the context
containing the variable didn't uniquely identify the lambda.

In this patch, we instead identify lambda closure types by index within
their context declaration, which does uniquely identify them in a way
that's consistent across modules.

This change causes a deserialization cycle between the type of a
variable with deduced type and a lambda appearing as the initializer of
the variable -- reading the variable's type requires reading and merging
the lambda, and reading the lambda requires reading and merging the
variable. This is addressed by deferring loading the deduced type of a
variable until after we finish recursive deserialization.

This also exposes a pre-existing subtle issue where loading a
variable declaration would trigger immediate loading of its initializer,
which could recursively refer back to properties of the variable. This
particularly causes problems if the initializer contains a
lambda-expression, but can be problematic in general. That is addressed
by switching to lazily loading the initializers of variables rather than
always loading them with the variable declaration. As well as fixing a
deserialization cycle, that should improve laziness of deserialization
in general.

LambdaDefinitionData had 63 spare bits in it, presumably caused by an
off-by-one-error in some previous change. This change claims 32 of those bits
as a counter for the lambda within its context. We could probably move the
numbering to separate storage, like we do for the device-side mangling number,
to optimize the likely-common case where all three numbers (host-side mangling
number, device-side mangling number, and index within the context declaration)
are zero, but that's not done in this change.

Fixes #60985.

Reviewed By: #clang-language-wg, aaron.ballman

Differential Revision: https://reviews.llvm.org/D145737
2023-03-30 14:22:40 -07:00
Chuanqi Xu 279c7a2f17 Revert "[C++20] [Modules] Don't load declaration eagerly for named modules"
This reverts commit af86957cbb.

Close https://github.com/llvm/llvm-project/issues/61733.

Previously I banned the eagerly loading for declarations from named
modules to speedup the process of reading modules. But I didn't think
about special decls like PragmaCommentDecl and PragmaDetectMismatchDecl.
So here is the issue https://github.com/llvm/llvm-project/issues/61733.

Note that the current behavior is still incorrect. Given:

```
// mod.cppm
module;

export module mod;
```

and

```
// user.cpp
import mod;
```

Now the IR of `user.cpp` will contain the metadata '!0 =
!{!"msvcprt.lib"}' incorrectly. The root cause of the problem is that
`EagerlyDeserializedDecls` is designed for headers and it didn't take
care for named modules. We need to redesign a new mechanism for named
modules.
2023-03-29 11:15:38 +08:00
Chuanqi Xu af86957cbb [C++20] [Modules] Don't load declaration eagerly for named modules
Close https://github.com/llvm/llvm-project/issues/61064.

The root cause of the issue is that we will deserilize some declarations
eagerly when reading the BMI. However, many declarations in the BMI are
not necessary for the importer. So it wastes a lot of time.
2023-03-10 14:57:21 +08:00
Chuanqi Xu 1ba93c3c30 [Modules] Don't re-generate template specialization in the importer
Close https://github.com/llvm/llvm-project/issues/60693.

In this issue, we can find that the importer will try to generate the
template specialization again in the importer, which is not good and
wastes time. This patch tries to address the problem.
2023-02-14 17:15:33 +08:00
Chuanqi Xu 9b7e574701 [NFC] [Serialization] Add static assert for Num*Declbits
This re-commits part of c79635cce8. It is reverted since it contains
platform-inconsistent constant. Now the patch only contains constant
defined in DeclBase.h so it should be platform-independent. And this
should be still helpful.

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D141992
2023-02-03 14:35:44 +08:00
Volodymyr Sapsai 6ba4afb4d6 [ODRHash] Hash ObjCInterfaceDecl and diagnose discovered mismatches.
When two modules contain interfaces with the same name, check the
definitions are equivalent and diagnose if they are not.

Differential Revision: https://reviews.llvm.org/D140073
2023-01-20 10:18:18 -06:00
Volodymyr Sapsai 160bc160b9 [ODRHash] Hash RecordDecl and diagnose discovered mismatches.
When two modules contain struct/union with the same name, check the
definitions are equivalent and diagnose if they are not. This is similar
to `CXXRecordDecl` where we already discover and diagnose mismatches.

rdar://problem/56764293

Differential Revision: https://reviews.llvm.org/D71734
2023-01-19 15:57:48 -06:00
Chuanqi Xu 38a2f089b4 Revert "[NFC] [Serialization] Add static assert for the size of the decls to"
This reverts commit c79635cce8. Since I
forgot the case for 32-bit machine.
2023-01-19 10:37:48 +08:00
Chuanqi Xu c79635cce8 [NFC] [Serialization] Add static assert for the size of the decls to
mention developers to remember to touch the serializer after them
modified the field of decls

It is easy for the developers to forget to touch the serializer after
they add new field to decls. Then if the existing tests fail to catch
such cases, it may be a bug report from users some day. And it is
time-consuming to solve such bugs.

To mitigate the problem, I add the static_asserts in the serializer. So
that the developers can understand they need to modify the serializer
after they saw the static assertion failure. Although this can't solve
all the problems, I feel the current status can be much better.

Reviewed By: erichkeane

Differential Revision: https://reviews.llvm.org/D141992
2023-01-19 10:20:29 +08:00
Chuanqi Xu 205b5f63a8 [Serialization] Serialize the new added FunctionDeclBits: IsIneligibleOrNotSelected
Close https://github.com/llvm/llvm-project/issues/59719.

The root cause of the problem is that we forgot to serialize a new
introduced bit to FunctionDeclBits. Maybe we need to find some methods
to work for detecting this.
2023-01-18 14:15:27 +08:00
Kazu Hirata 6ad0788c33 [clang] Use std::optional instead of llvm::Optional (NFC)
This patch replaces (llvm::|)Optional< with std::optional<.  I'll post
a separate patch to remove #include "llvm/ADT/Optional.h".

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-14 12:31:01 -08:00
Kazu Hirata a1580d7b59 [clang] Add #include <optional> (NFC)
This patch adds #include <optional> to those files containing
llvm::Optional<...> or Optional<...>.

I'll post a separate patch to actually replace llvm::Optional with
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2023-01-14 11:07:21 -08:00
serge-sans-paille a3c248db87 Move from llvm::makeArrayRef to ArrayRef deduction guides - clang/ part
This is a follow-up to https://reviews.llvm.org/D140896, split into
several parts as it touches a lot of files.

Differential Revision: https://reviews.llvm.org/D141139
2023-01-09 12:15:24 +01:00
Chuanqi Xu 367e618fd6 [C++20] [Modules] Emit full specialization of variable template as available externally in importers
Closes https://github.com/llvm/llvm-project/issues/59780.

In this issue report, when we use full specialization of variable
templates in modules, we will meet the multiple definition errors.

The root cause of the problem is that when we see the full
specialization of the variable template in the importers, we will find
if it is already defined in the external sources and we failed to find
such definitions from external sources. So we generate the definition in
the current TU. We failed to find the definition in the external sources
because we restricted to not write it during writing. However, we don't
know why we restricted it and it doesn't make a lot sense to do such
restriction. Then no test fails after we remove such limitations. So
let's remove it now and add it back later if we found it is necessary
then we can add the corresponding test that time.

Note that the code is only applied to named modules and
PCHWithObjectFiles. So it won't affect the normal clang modules and
header units.
2023-01-03 14:48:29 +08:00
Kazu Hirata 5891420e68 [clang] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-03 11:54:46 -08:00
Vassil Vassilev dc4889357a [clang-repl] Support statements on global scope in incremental mode.
This patch teaches clang to parse statements on the global scope to allow:
```
./bin/clang-repl
clang-repl> int i = 12;
clang-repl> ++i;
clang-repl> extern "C" int printf(const char*,...);
clang-repl> printf("%d\n", i);
13
clang-repl> %quit
```

Generally, disambiguating between statements and declarations is a non-trivial
task for a C++ parser. The challenge is to allow both standard C++ to be
translated as if this patch does not exist and in the cases where the user typed
a statement to be executed as if it were in a function body.

Clang's Parser does pretty well in disambiguating between declarations and
expressions. We have added DisambiguatingWithExpression flag which allows us to
preserve the existing and optimized behavior where needed and implement the
extra rules for disambiguating. Only few cases require additional attention:
  * Constructors/destructors -- Parser::isConstructorDeclarator was used in to
    disambiguate between ctor-looking declarations and statements on the global
    scope(eg. `Ns::f()`).
  * The template keyword -- the template keyword can appear in both declarations
    and statements. This patch considers the template keyword to be a declaration
    starter which breaks a few cases in incremental mode which will be tackled
    later.
  * The inline (and similar) keyword -- looking at the first token in many cases
    allows us to classify what is a declaration.
  * Other language keywords and specifiers -- ObjC/ObjC++/OpenCL/OpenMP rely on
    pragmas or special tokens which will be handled in subsequent patches.

The patch conceptually models a "top-level" statement into a TopLevelStmtDecl.
The TopLevelStmtDecl is lowered into a void function with no arguments.
We attach this function to the global initializer list to execute the statement
blocks in the correct order.

Differential revision: https://reviews.llvm.org/D127284
2022-12-03 07:18:07 +00:00
Nathan James 15e76eed0c [clang] Add [is|set]Nested methods to NamespaceDecl
Adds support for NamespaceDecl to inform if its part of a nested namespace.
This flag only corresponds to the inner namespaces in a nested namespace declaration.
In this example:
namespace <X>::<Y>::<Z> {}
Only <Y> and <Z> will be classified as nested.

This flag isn't meant for assisting in building the AST, more for static analysis and refactorings.

Reviewed By: aaron.ballman

Differential Revision: https://reviews.llvm.org/D90568
2022-11-24 12:44:35 +00:00
Jan Svoboda 0bfc97e4f4 [clang][modules] NFCI: Scaffolding for serialization of adjusted SourceManager offsets
This patch is a NFC prep for D136624, where we start adjusting offsets into `SourceManager`.

Depends on D137213.

Reviewed By: dexonsmith

Differential Revision: https://reviews.llvm.org/D137214
2022-11-01 19:06:55 -07:00
Chuanqi Xu 22914a8229 [NFC] Use isa<...> to replace isa<>||isa<> in clang/Serialization
Now isa supports the variant args, which could simplify the codes
further. This patch simplify the uses in clang/Serialization
2022-10-31 21:17:03 +08:00
Erich Keane 975740bf8d "Reapply "GH58368: Correct concept checking in a lambda defined in concept""
This reverts commit cecc9a92cf.

The problem ended up being how we were handling the lambda-context in
code generation: we were assuming any decl context here would be a
named-decl, but that isn't the case.  Instead, we just replace it with
the concept's owning context.

Differential Revision: https://reviews.llvm.org/D136451
2022-10-24 12:36:54 -07:00
Erich Keane cecc9a92cf Revert "Reapply "GH58368: Correct concept checking in a lambda defined in concept"""
This reverts commit b876f6e2f2.

Still getting build failures on PPC AIX that aren't obvious what is causing
them, so reverting while I try to figure this out.
2022-10-24 12:20:23 -07:00