[CIR] Add CallConvLowering pass + Direct/Ignore ABI rewriting (#195737)

Depends on #195725 (merged).

Stacked on #195725 (PR A1). The diff in this PR is the CIR side; A1 has
the dialect-agnostic infrastructure.

Adds the `cir-call-conv-lowering` pass plus the `CIRABIRewriteContext`
skeleton with handlers for Direct (true pass-through) and Ignore (drop
empty-record args/returns). Subsequent PRs in the split add Extend,
Direct-with-coercion + a new reinterpret op, Indirect/sret,
Indirect/byval, and Expand — each is purely additive because the
rewriter dispatches on `argClass.kind` via a switch with explicit "not
yet implemented" diagnostics for every other kind.

The pass takes one of two driver options: `target=test` uses the test
target from PR A1, and `classification-attr=<name>` reads a pre-built
`DictionaryAttr` from each `cir.func` (also via the helper from PR A1).
It also requires `dlti.dl_spec` on the module and emits a clear
diagnostic otherwise, since classification can't run without
sizes/alignments.

Six `.cir` tests covering the Direct/Ignore behavior via both drivers,
plus the missing-DataLayout error case. `check-clang-cir` and
`check-clang-cir-codegen` both pass with no regressions.

`clang/docs/CIR/ABILowering.rst` gets a short pipeline-position section
(where this pass runs vs CXXABILowering and HoistAllocas, the DataLayout
requirement, the alloca-placement invariant).

The 11 C++ unit tests in #192119 each map onto a `.cir` test in this
series — 6 ported here (Direct passthrough, Ignore arg, Ignore return,
declaration rewrite, plus their call-site variants), 3 deferred to PR B
(Extend), 2 deferred to PR C (Direct-with-coercion). The 5 from #192124
split between PRs D and E. None get dropped, and the original C++ test
file lives only on the closed branches.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
adams381andCursor authored and GitHub committed 2026-05-21 22:42:11 +00:00
1 parent c0da1855e0
commit cfc03d62b4
20 files changed
+1029 -10

No files matched your search

+33
View File
@@ -531,6 +531,39 @@ options or configuration. The dependency direction is: the MLIR ABI pass
depends on ``llvm/lib/ABI``; there is no reverse dependency from the ABI library
to MLIR dialects.
CIR Pass Pipeline Position
--------------------------
For the CIR dialect, the calling-convention lowering pass is named
``cir-call-conv-lowering`` and runs late in the CIR-to-LLVM pipeline:
1. ``cir-target-lowering`` legalizes target-specific operations (e.g.
atomic synchronization scopes).
2. ``cir-cxxabi-lowering`` lowers C++-specific high-level types (member
pointers, vtable lookups, etc.) to ABI-specific representations.
3. ``cir-call-conv-lowering`` rewrites function signatures and call sites
to match the target ABI's calling convention rules.
``cir-call-conv-lowering`` requires a ``dlti.dl_spec`` attribute on the
module so it can query type sizes and alignments through MLIR's
``DataLayout``. When the attribute is missing, the pass emits a
diagnostic and fails rather than silently using a default layout.
The pass takes one of two driver modes via pass options:
- ``target=<name>`` selects a real ABI target. The first supported value
is ``test`` (the MLIR test target in ``mlir/lib/ABI/Targets/Test/``,
used for testing the rewriter without depending on the in-progress
LLVM ABI library targets). Real targets (``x86_64``, ``aarch64``,
...) will be added as the LLVM ABI library ships them.
- ``classification-attr=<name>`` reads a pre-built ``FunctionClassifica
tion`` from a ``DictionaryAttr`` named ``<name>`` on each ``cir.func``
and rewrites accordingly. This driver is for tests that need to
exercise rewriter behavior against arbitrary classifications without
routing through any real classifier.
Exactly one of the two options must be set.
Open Questions
==============
+1
View File
@@ -27,6 +27,7 @@ std::unique_ptr<Pass> createCIRSimplifyPass();
std::unique_ptr<Pass> createCIREHABILoweringPass();
std::unique_ptr<Pass> createCXXABILoweringPass();
std::unique_ptr<Pass> createTargetLoweringPass();
std::unique_ptr<Pass> createCallConvLoweringPass();
std::unique_ptr<Pass> createHoistAllocasPass();
std::unique_ptr<Pass> createLoweringPreparePass();
std::unique_ptr<Pass> createLoweringPreparePass(clang::ASTContext *astCtx);
+47 -6
View File
@@ -108,7 +108,7 @@ def TargetLowering : Pass<"cir-target-lowering", "mlir::ModuleOp"> {
1. The `TargetLowering` pass.
2. The `CXXABILowering` pass.
3. The `CallConvLowering` pass (not implemented yet).
3. The `CallConvLowering` pass.
The `TargetLowering` pass acts more like a legalization pass. It ensures
every operation in CIR conforms to the target's constraints. An example
@@ -117,11 +117,11 @@ def TargetLowering : Pass<"cir-target-lowering", "mlir::ModuleOp"> {
any atomic operations with a different synchronization scope would be
transformed to use the system-wide scope in this pass.
The `CXXABILowering` pass and the (not yet implemented) `CallConvLowering`
pass transform the CIR according to the target's ABI requirements. The
former handles all ABI-related lowering except for calling convention
handling, which is handled specifically in the latter. Example
transformations that the `CXXABILowering` pass could make include:
The `CXXABILowering` pass and the `CallConvLowering` pass transform the
CIR according to the target's ABI requirements. The former handles all
ABI-related lowering except for calling convention handling, which is
handled specifically in the latter. Example transformations that the
`CXXABILowering` pass could make include:
- Replace C/C++ types that have an ABI-defined layout with more
fundamental types corresponding to the ABI requirements. For example,
@@ -195,4 +195,45 @@ def IdiomRecognizer : Pass<"cir-idiom-recognizer", "mlir::ModuleOp"> {
let dependentDialects = ["cir::CIRDialect"];
}
def CallConvLowering : Pass<"cir-call-conv-lowering", "mlir::ModuleOp"> {
let summary = "Lower CIR function signatures and call sites to match target ABI";
let description = [{
This pass rewrites `cir.func` signatures and `cir.call` sites to match the
target ABI's calling convention requirements (extension attributes, struct
coercion, sret/byval indirect passing, struct flattening, etc.).
Requirements on the input IR:
- All C++-specific CIR types must already be lowered to ABI-relevant
shapes. In particular, member pointers and method types must have
been rewritten by `CXXABILowering` so that the pass sees only the
final ABI representation of each value.
- The module must carry a `dlti.dl_spec` attribute so the classifier
can query type sizes and alignments.
Two driver modes select how each function's classification is computed:
- `target=<name>` selects an ABI target. Currently only `"test"` (the
MLIR test target in `mlir/lib/ABI/Targets/Test/`) is supported. Real
targets (x86_64, AArch64, ...) will be added once the LLVM ABI library
ships them.
- `classification-attr=<name>` reads a `DictionaryAttr` named `<name>`
from each `cir.func` and parses it via the test-target injection
helper. Used by tests to inject arbitrary classifications without
depending on a real ABI target.
Exactly one of the two options must be set.
}];
let constructor = "mlir::createCallConvLoweringPass()";
let dependentDialects = ["cir::CIRDialect"];
let options = [
Option<"target", "target", "std::string", /*default=*/"\"\"",
"Target whose ABI rules drive classification (currently: test)">,
Option<"classificationAttr", "classification-attr", "std::string",
/*default=*/"\"\"",
"Function attribute name carrying a pre-built FunctionClassification "
"DictionaryAttr (alternative to target=, used by tests)">,
];
}
#endif // CLANG_CIR_DIALECT_PASSES_TD
@@ -1,6 +1,7 @@
add_subdirectory(TargetLowering)
add_clang_library(MLIRCIRTransforms
CallConvLoweringPass.cpp
CIRCanonicalize.cpp
CIRSimplify.cpp
CIRTransformUtils.cpp
@@ -20,7 +21,9 @@ add_clang_library(MLIRCIRTransforms
clangAST
clangBasic
MLIRABI
MLIRAnalysis
MLIRDLTIDialect
MLIRIR
MLIRPass
MLIRTransformUtils
@@ -0,0 +1,228 @@
//===- CallConvLoweringPass.cpp - Lower CIR to ABI calling convention ----===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// This pass walks every cir.func and cir.call in the module, computes a
// FunctionClassification for it (via either an ABI target or a pre-built
// classification injected as a function attribute), and dispatches to
// CIRABIRewriteContext to perform the actual IR rewriting.
//
// Two driver modes (mutually exclusive):
//
// target=test
// Use the MLIR test ABI target (mlir/lib/ABI/Targets/Test/) to classify
// each function. Predictable rules that approximate x86_64 SysV. Real
// targets (x86_64, AArch64) will be added once the LLVM ABI library
// ships them.
//
// classification-attr=<name>
// Read a DictionaryAttr named <name> from each cir.func and parse it via
// mlir::abi::test::parseClassificationAttr. Used by tests to inject any
// classification (including shapes the test target itself does not
// produce) without depending on a real ABI target.
//
// The pass requires a `dlti.dl_spec` attribute on the module so the
// classifier can query type sizes and alignments.
//
//===----------------------------------------------------------------------===//
#include "PassDetail.h"
#include "TargetLowering/CIRABIRewriteContext.h"
#include "mlir/ABI/ABIRewriteContext.h"
#include "mlir/ABI/Targets/Test/TestTarget.h"
#include "mlir/Dialect/DLTI/DLTI.h"
#include "mlir/IR/Builders.h"
#include "mlir/IR/BuiltinOps.h"
#include "mlir/IR/SymbolTable.h"
#include "mlir/Interfaces/DataLayoutInterfaces.h"
#include "mlir/Pass/Pass.h"
#include "clang/CIR/Dialect/IR/CIRDialect.h"
#include "clang/CIR/Dialect/Passes.h"
using namespace mlir;
using namespace mlir::abi;
using namespace cir;
namespace mlir {
#define GEN_PASS_DEF_CALLCONVLOWERING
#include "clang/CIR/Dialect/Passes.h.inc"
} // namespace mlir
namespace {
bool needsRewrite(const FunctionClassification &fc) {
if ((fc.returnInfo.kind != ArgKind::Direct) || fc.returnInfo.coercedType)
return true;
for (const ArgClassification &ac : fc.argInfos)
if ((ac.kind != ArgKind::Direct) || ac.coercedType)
return true;
return false;
}
struct CallConvLoweringPass
: public impl::CallConvLoweringBase<CallConvLoweringPass> {
using CallConvLoweringBase::CallConvLoweringBase;
void runOnOperation() override;
};
/// Classify \p func using whichever driver mode is configured. Returns
/// std::nullopt and emits an error on the function if classification fails
/// (e.g. injection-driver mode but the function is missing the attribute,
/// or the attribute is malformed).
std::optional<FunctionClassification>
classifyFunction(cir::FuncOp func, const DataLayout &dl, StringRef target,
StringRef classificationAttrName) {
ArrayRef<Type> argTypes = func.getFunctionType().getInputs();
Type returnType = func.getFunctionType().getReturnType();
if (!classificationAttrName.empty()) {
auto attr = func->getAttrOfType<DictionaryAttr>(classificationAttrName);
if (!attr) {
func.emitOpError()
<< "missing classification attribute '" << classificationAttrName
<< "' (CallConvLowering driver mode 'classification-attr')";
return std::nullopt;
}
return mlir::abi::test::parseClassificationAttr(
attr, [&]() { return func.emitOpError(); });
}
if (target == "test")
return mlir::abi::test::classify(argTypes, returnType, dl);
func.emitOpError() << "unknown target '" << target << "' (supported: test)";
return std::nullopt;
}
/// Find the cir.func declaration matching a direct cir.call / cir.try_call
/// callee, if any. Returns nullptr if the callee is indirect or the symbol
/// cannot be resolved. Takes a SymbolTable instead of a ModuleOp so the
/// symbol lookup is amortized across all the call sites the driver walks
/// (ModuleOp::lookupSymbol is linear per call).
cir::FuncOp lookupCallee(Operation *callOp, SymbolTable &symbolTable) {
FlatSymbolRefAttr callee;
if (auto call = dyn_cast<cir::CallOp>(callOp))
callee = call.getCalleeAttr();
else if (auto tryCall = dyn_cast<cir::TryCallOp>(callOp))
callee = tryCall.getCalleeAttr();
else
return nullptr;
if (!callee)
return nullptr;
return symbolTable.lookup<cir::FuncOp>(callee.getValue());
}
void CallConvLoweringPass::runOnOperation() {
ModuleOp moduleOp = getOperation();
MLIRContext *ctx = &getContext();
if (target.empty() == classificationAttr.empty()) {
moduleOp.emitOpError() << "CallConvLowering requires exactly one of "
"'target' or 'classification-attr' pass options";
signalPassFailure();
return;
}
if (!moduleOp->hasAttr(DLTIDialect::kDataLayoutAttrName)) {
moduleOp.emitOpError()
<< "CallConvLowering requires a DataLayout (dlti.dl_spec attribute "
"on the module)";
signalPassFailure();
return;
}
DataLayout dl(moduleOp);
CIRABIRewriteContext rewriteCtx(moduleOp);
SymbolTable symbolTable(moduleOp);
// Classify every cir.func up front. No IR mutation happens here, so
// later walks can consult any function's classification regardless of
// visitation order.
llvm::MapVector<cir::FuncOp, FunctionClassification> classifications;
bool anyFailed = false;
moduleOp.walk([&](cir::FuncOp f) {
auto fc = classifyFunction(f, dl, target, classificationAttr);
if (!fc) {
anyFailed = true;
return;
}
classifications.insert({f, std::move(*fc)});
});
if (anyFailed) {
signalPassFailure();
return;
}
// Build a callee-to-callers index. One module walk collects every direct
// cir.call / cir.try_call to each cir.func; the loop below rewrites a
// function and all of its call sites together. Indirect or unresolved
// callees are skipped here; rewriteCallSite errors on those at the end.
llvm::DenseMap<cir::FuncOp, SmallVector<Operation *>> callers;
moduleOp.walk([&](Operation *op) {
if (!isa<cir::CallOp, cir::TryCallOp>(op))
return;
if (cir::FuncOp callee = lookupCallee(op, symbolTable))
callers[callee].push_back(op);
});
// Rewrite each function together with every direct call to it. By the
// time we move on to function F+1, F's signature and every direct call to
// F have already been brought into alignment, and F+1..FN are still in
// their original (mutually consistent) form, so the IR is verifier-clean
// at every outer-iteration boundary.
//
// There is still a brief inner window where F's signature has been
// rewritten but its callers have not yet caught up -- we have no way to
// mutate both sides of a call atomically. No verifier runs inside the
// pass, and at pass exit the module is verifier-clean. Fusing the inner
// loop here keeps the invalid window per-function rather than module-wide.
OpBuilder builder(ctx);
for (auto &kv : classifications) {
cir::FuncOp func = kv.first;
const FunctionClassification &fc = kv.second;
if (failed(rewriteCtx.rewriteFunctionDefinition(func, fc, builder))) {
signalPassFailure();
return;
}
for (Operation *callOp : callers.lookup(func)) {
if (failed(rewriteCtx.rewriteCallSite(callOp, fc, builder))) {
signalPassFailure();
return;
}
}
}
// Reject indirect calls when the module contains any ABI rewrite that
// would need call-site lowering. We cannot strip or coerce operands
// without a resolved callee symbol.
const FunctionClassification *rewriteFc = nullptr;
for (auto &kv : classifications) {
if (needsRewrite(kv.second)) {
rewriteFc = &kv.second;
break;
}
}
if (rewriteFc) {
moduleOp.walk([&](cir::CallOp c) {
if (!c.isIndirect())
return;
if (failed(rewriteCtx.rewriteCallSite(c, *rewriteFc, builder)))
anyFailed = true;
});
if (anyFailed) {
signalPassFailure();
return;
}
}
}
} // namespace
std::unique_ptr<Pass> mlir::createCallConvLoweringPass() {
return std::make_unique<CallConvLoweringPass>();
}
@@ -0,0 +1,322 @@
//===- CIRABIRewriteContext.cpp - CIR ABI rewrite context ----------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
#include "CIRABIRewriteContext.h"
#include "mlir/IR/Builders.h"
#include "clang/CIR/Dialect/IR/CIRAttrs.h"
#include "clang/CIR/Dialect/IR/CIRTypes.h"
using namespace cir;
using namespace mlir;
using namespace mlir::abi;
// This rewrite context currently supports only the Direct (no coercion) and
// Ignore classifications. All other ArgKinds emit an errorNYI here rather
// than silently passing through, because the IR they would produce is wrong
// (e.g. Expand should flatten an aggregate into multiple primitives, not
// pass it through as a single value). Subsequent PRs in the
// CallConvLowering split series add the remaining kinds and the
// signature-shaping behavior that goes with them (sret / byval insert
// extra arguments, struct coercion replaces one argument with several).
namespace {
bool needsRewrite(const FunctionClassification &fc) {
if ((fc.returnInfo.kind != ArgKind::Direct) || fc.returnInfo.coercedType)
return true;
for (const ArgClassification &ac : fc.argInfos)
if ((ac.kind != ArgKind::Direct) || ac.coercedType)
return true;
return false;
}
SmallVector<unsigned> ignoredArgIndices(const FunctionClassification &fc) {
SmallVector<unsigned> v;
for (auto [idx, ac] : llvm::enumerate(fc.argInfos))
if (ac.kind == ArgKind::Ignore)
v.push_back(idx);
return v;
}
/// Build the new argument-type list for a function whose ABI classification
/// is \p fc. This currently handles only Direct (no coercion) and Ignore;
/// other kinds emit an error. Classifications that add arguments (e.g.
/// Indirect-sret would prepend a return-pointer arg) are not yet
/// implemented and will arrive in a subsequent PR.
LogicalResult buildNewArgTypes(ArrayRef<Type> oldArgTypes,
const FunctionClassification &fc,
SmallVectorImpl<Type> &newArgTypes,
function_ref<InFlightDiagnostic()> emitError) {
assert(newArgTypes.empty() && "expected an empty output vector");
newArgTypes.reserve(oldArgTypes.size());
for (auto [idx, ac] : llvm::enumerate(fc.argInfos)) {
Type origTy = oldArgTypes[idx];
switch (ac.kind) {
case ArgKind::Direct:
if (ac.coercedType) {
emitError() << "Direct with coerced type at arg " << idx
<< " not yet implemented in CallConvLowering";
return failure();
}
newArgTypes.push_back(origTy);
break;
case ArgKind::Ignore:
break;
case ArgKind::Expand:
emitError() << "Expand at arg " << idx
<< " not yet implemented in CallConvLowering";
return failure();
case ArgKind::Extend:
emitError() << "Extend at arg " << idx
<< " not yet implemented in CallConvLowering";
return failure();
case ArgKind::Indirect:
emitError() << "Indirect at arg " << idx
<< " not yet implemented in CallConvLowering";
return failure();
}
}
return success();
}
/// Compute the new return type for a function whose return classification
/// is \p retInfo. As with `buildNewArgTypes`, only Direct (no coercion)
/// and Ignore are implemented here; the remaining kinds emit an error.
Type computeNewReturnType(Type origRetTy, const ArgClassification &retInfo,
MLIRContext *ctx,
function_ref<InFlightDiagnostic()> emitError) {
switch (retInfo.kind) {
case ArgKind::Direct:
if (retInfo.coercedType) {
emitError() << "Direct return with coerced type not yet implemented "
<< "in CallConvLowering";
return nullptr;
}
return origRetTy;
case ArgKind::Ignore:
return cir::VoidType::get(ctx);
case ArgKind::Expand:
emitError() << "Expand return is not allowed (classic codegen rejects "
<< "it in EmitFunctionEpilog)";
return nullptr;
case ArgKind::Extend:
emitError() << "Extend return not yet implemented in CallConvLowering";
return nullptr;
case ArgKind::Indirect:
emitError() << "Indirect return (sret) not yet implemented in "
<< "CallConvLowering";
return nullptr;
}
llvm_unreachable("all ArgKind cases handled");
}
/// Create a typed poison constant to stand in for a value the body of a
/// function (or the result of a call) still references but whose ABI
/// classification is Ignore. Using poison is honest -- the value is
/// genuinely unused at the ABI boundary -- and avoids a fake alloca+load
/// pattern that would suggest we have a value when we don't.
Value createIgnoredValue(OpBuilder &builder, Location loc, Type ty) {
return cir::ConstantOp::create(builder, loc, ty, cir::PoisonAttr::get(ty));
}
} // namespace
LogicalResult CIRABIRewriteContext::rewriteFunctionDefinition(
FunctionOpInterface funcOpInterface, const FunctionClassification &fc,
OpBuilder &builder) {
// The pass driver (CallConvLoweringPass) only ever hands us cir.func ops,
// and the body of this routine is end-to-end CIR (it creates cir.constant,
// cir.return, etc.). Cast once at the top so the rest of the function
// reads in CIR's own vocabulary, and so we can dispatch to the
// CIRGlobalValueInterface for isDefinition() (FunctionOpInterface alone
// does not inherit from CIRGlobalValueInterface).
cir::FuncOp funcOp = cast<cir::FuncOp>(funcOpInterface);
if (!needsRewrite(fc))
return success();
ArrayRef<Type> oldArgTypes = funcOp.getArgumentTypes();
ArrayRef<Type> oldResultTypes = funcOp.getResultTypes();
MLIRContext *ctx = funcOp->getContext();
// CIR follows LLVM IR's single-result rule: a function returns either
// zero or one value. Document the invariant so a future multi-result
// change forces us to revisit the return-handling below.
assert(oldResultTypes.size() <= 1 &&
"CIR functions return zero or one value");
SmallVector<Type> newArgTypes;
if (failed(buildNewArgTypes(oldArgTypes, fc, newArgTypes,
[&]() { return funcOp.emitOpError(); })))
return failure();
Type voidTy = cir::VoidType::get(ctx);
Type origRetTy = oldResultTypes.empty() ? voidTy : oldResultTypes[0];
Type newRetTy = computeNewReturnType(origRetTy, fc.returnInfo, ctx,
[&]() { return funcOp.emitOpError(); });
if (!newRetTy)
return failure();
SmallVector<Type> newResultTypes = {newRetTy};
if (funcOp.isDefinition()) {
Region &body = funcOp->getRegion(0);
if (!body.empty()) {
Block &entry = body.front();
// For each Ignored argument: drop the block argument and, if the
// body still references it, replace those uses with a poison
// constant. Ignore classifications mean the value is empty / not
// passed at the ABI level, so any remaining uses are vacuous;
// poison says exactly that.
SmallVector<unsigned> ignored = ignoredArgIndices(fc);
for (unsigned blockIdx : llvm::reverse(ignored)) {
if (blockIdx >= entry.getNumArguments())
continue;
BlockArgument arg = entry.getArgument(blockIdx);
if (!arg.use_empty()) {
builder.setInsertionPointToStart(&entry);
Value poison =
createIgnoredValue(builder, funcOp.getLoc(), arg.getType());
arg.replaceAllUsesWith(poison);
}
entry.eraseArgument(blockIdx);
}
}
// When the return is classified Ignore but the original function had
// a non-void return type, every cir.return becomes a naked return.
// This relies on the invariant that computeNewReturnType has set
// newRetTy = void for Ignore above, and that the function type is
// updated below to match. Asserting this keeps the dependency
// explicit.
if (fc.returnInfo.kind == ArgKind::Ignore && !oldResultTypes.empty()) {
assert(isa<cir::VoidType>(newRetTy) &&
"Ignore-return path requires the new return type to be void");
SmallVector<cir::ReturnOp> returns;
funcOp.walk([&](cir::ReturnOp r) { returns.push_back(r); });
for (cir::ReturnOp r : returns) {
if (r.getNumOperands() == 0)
continue;
builder.setInsertionPoint(r);
cir::ReturnOp::create(builder, r.getLoc());
r.erase();
}
}
}
Type newFnTy = funcOp.cloneTypeWith(newArgTypes, newResultTypes);
funcOp.setFunctionTypeAttr(TypeAttr::get(newFnTy));
// Keep the arg_attrs array in sync with the new argument count by
// dropping entries for every Ignored argument. Without this the
// attribute array would have stale entries that no longer match any
// block argument.
SmallVector<unsigned> ignored = ignoredArgIndices(fc);
if (!ignored.empty()) {
if (auto existing = funcOp->getAttrOfType<ArrayAttr>("arg_attrs")) {
SmallVector<Attribute> kept;
kept.reserve(newArgTypes.size());
for (auto [oldIdx, attr] : llvm::enumerate(existing.getValue())) {
if (oldIdx >= fc.argInfos.size() ||
fc.argInfos[oldIdx].kind != ArgKind::Ignore)
kept.push_back(attr);
}
funcOp->setAttr("arg_attrs", ArrayAttr::get(ctx, kept));
}
}
return success();
}
LogicalResult CIRABIRewriteContext::rewriteCallSite(
Operation *callOp, const FunctionClassification &fc, OpBuilder &builder) {
if (!needsRewrite(fc))
return success();
if (isa<cir::TryCallOp>(callOp))
return callOp->emitOpError()
<< "TryCallOp not yet implemented in CallConvLowering";
auto call = cast<cir::CallOp>(callOp);
if (call.isIndirect())
return call.emitOpError()
<< "indirect call not yet implemented in CallConvLowering";
for (auto [idx, ac] : llvm::enumerate(fc.argInfos)) {
switch (ac.kind) {
case ArgKind::Direct:
if (ac.coercedType)
return call.emitOpError()
<< "Direct with coerced type at call-site arg " << idx
<< " not yet implemented in CallConvLowering";
break;
case ArgKind::Ignore:
break;
case ArgKind::Expand:
return call.emitOpError() << "Expand at call-site arg " << idx
<< " not yet implemented in CallConvLowering";
case ArgKind::Extend:
return call.emitOpError() << "Extend at call-site arg " << idx
<< " not yet implemented in CallConvLowering";
case ArgKind::Indirect:
return call.emitOpError() << "Indirect at call-site arg " << idx
<< " not yet implemented in CallConvLowering";
}
}
SmallVector<Value> newArgs;
ValueRange argOperands = call.getArgOperands();
newArgs.reserve(argOperands.size());
if (argOperands.size() > fc.argInfos.size())
return call.emitOpError()
<< "variadic arguments not yet implemented in CallConvLowering";
assert(fc.argInfos.size() == argOperands.size() &&
"call operand count must match classified arg count");
for (auto [idx, ac] : llvm::enumerate(fc.argInfos)) {
if (ac.kind == ArgKind::Ignore)
continue;
newArgs.push_back(argOperands[idx]);
}
bool hasResult = call.getNumResults() > 0;
Type origRetTy = hasResult ? call.getResult().getType()
: cir::VoidType::get(callOp->getContext());
Type callRetTy = origRetTy;
if (fc.returnInfo.kind == ArgKind::Ignore && hasResult)
callRetTy = cir::VoidType::get(callOp->getContext());
if ((fc.returnInfo.kind == ArgKind::Direct ||
fc.returnInfo.kind == ArgKind::Extend) &&
fc.returnInfo.coercedType)
return call.emitOpError() << "Direct/Extend return with coerced type at "
<< "call-site not yet implemented in "
<< "CallConvLowering";
builder.setInsertionPoint(call);
auto newCall = cir::CallOp::create(builder, call.getLoc(),
call.getCalleeAttr(), callRetTy, newArgs);
for (NamedAttribute attr : call->getAttrs())
if (!newCall->hasAttr(attr.getName()))
newCall->setAttr(attr.getName(), attr.getValue());
if (hasResult && fc.returnInfo.kind == ArgKind::Ignore) {
// The new call returns void, but the original call's result may still
// have uses. Substitute a poison constant of the original type so
// those uses remain well-formed without pretending we have a real
// value at the ABI boundary.
if (!call.getResult().use_empty()) {
builder.setInsertionPointAfter(newCall);
Value poison = createIgnoredValue(builder, call.getLoc(), origRetTy);
call.getResult().replaceAllUsesWith(poison);
}
} else if (hasResult) {
call.getResult().replaceAllUsesWith(newCall.getResult());
}
call->erase();
return success();
}
@@ -0,0 +1,56 @@
//===- CIRABIRewriteContext.h - CIR ABI rewrite context ---------*- C++ -*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// Defines CIRABIRewriteContext, the CIR dialect's implementation of the
// generic mlir::abi::ABIRewriteContext. Given a FunctionClassification it
// rewrites a cir.func signature, the function body, and call sites to match
// the ABI-lowered shape.
//
// This file currently handles only Direct (pass-through) and Ignore. Other
// ArgKind handlers (Extend, Direct-with-coercion, Indirect, Expand) are
// added by subsequent PRs in the calling-convention-lowering split series.
//
//===----------------------------------------------------------------------===//
#ifndef CLANG_LIB_CIR_DIALECT_TRANSFORMS_TARGETLOWERING_CIRABIREWRITECONTEXT_H
#define CLANG_LIB_CIR_DIALECT_TRANSFORMS_TARGETLOWERING_CIRABIREWRITECONTEXT_H
#include "mlir/ABI/ABIRewriteContext.h"
#include "mlir/IR/BuiltinOps.h"
#include "clang/CIR/Dialect/IR/CIRDialect.h"
namespace cir {
/// CIR-specific implementation of mlir::abi::ABIRewriteContext.
///
/// The driver pass (CallConvLoweringPass) computes a FunctionClassification
/// for each cir.func / cir.call and dispatches to this class to perform the
/// actual IR rewriting using cir dialect operations.
class CIRABIRewriteContext : public mlir::abi::ABIRewriteContext {
public:
explicit CIRABIRewriteContext(mlir::ModuleOp module) : module(module) {}
mlir::LogicalResult
rewriteFunctionDefinition(mlir::FunctionOpInterface funcOp,
const mlir::abi::FunctionClassification &fc,
mlir::OpBuilder &builder) override;
mlir::LogicalResult
rewriteCallSite(mlir::Operation *callOp,
const mlir::abi::FunctionClassification &fc,
mlir::OpBuilder &builder) override;
mlir::StringRef getDialectNamespace() const override { return "cir"; }
private:
mlir::ModuleOp module;
};
} // namespace cir
#endif // CLANG_LIB_CIR_DIALECT_TRANSFORMS_TARGETLOWERING_CIRABIREWRITECONTEXT_H
@@ -1,4 +1,5 @@
add_clang_library(MLIRCIRTargetLowering
CIRABIRewriteContext.cpp
CIRCXXABI.cpp
LowerModule.cpp
LowerItaniumCXXABI.cpp
@@ -15,6 +16,7 @@ add_clang_library(MLIRCIRTargetLowering
LINK_LIBS PUBLIC
clangBasic
MLIRABI
MLIRIR
MLIRPass
MLIRDLTIDialect
@@ -0,0 +1,17 @@
// Shared fixture: minimal x86_64 dlti.dl_spec used by every abi-lowering
// test below. Test files include this verbatim inside their `module
// attributes { ... }` declaration so that CallConvLowering's DataLayout
// queries return sensible sizes and alignments.
//
// Copy this attribute into a test like so:
//
// module attributes {
// dlti.dl_spec = #dlti.dl_spec<
// #dlti.dl_entry<i1, dense<8>: vector<2xi64>>,
// #dlti.dl_entry<i8, dense<8>: vector<2xi64>>,
// #dlti.dl_entry<i16, dense<16>: vector<2xi64>>,
// #dlti.dl_entry<i32, dense<32>: vector<2xi64>>,
// #dlti.dl_entry<i64, dense<64>: vector<2xi64>>,
// #dlti.dl_entry<f32, dense<32>: vector<2xi64>>,
// #dlti.dl_entry<f64, dense<64>: vector<2xi64>>>
// } { ... }
@@ -0,0 +1,17 @@
// RUN: not cir-opt %s -cir-call-conv-lowering=target=test 2>&1 | FileCheck %s
// The pass requires a `dlti.dl_spec` attribute on the module. Without it,
// classification cannot query type sizes / alignments, so the pass emits a
// diagnostic and fails.
!s32i = !cir.int<s, 32>
module {
cir.func @passthrough(%arg0: !s32i) -> !s32i {
cir.return %arg0 : !s32i
}
}
// CHECK: error: 'builtin.module' op CallConvLowering requires a DataLayout (dlti.dl_spec attribute on the module)
@@ -0,0 +1,34 @@
// RUN: cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \
// RUN: | FileCheck %s
// Function declarations (no body) get their signature rewritten just like
// definitions: arg list is updated, return type is updated, but no body
// adaptation runs.
!s32i = !cir.int<s, 32>
#ignore_first_arg = {
return = { kind = "direct" },
args = [ { kind = "ignore" }, { kind = "direct" } ]
}
module attributes {
dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>>
} {
cir.func private @ext_decl(!s32i, !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg }
// The first argument type is dropped from the declaration's signature.
// CHECK: cir.func{{.*}} @ext_decl(!s32i) -> !s32i
cir.func @caller(%arg0: !s32i, %arg1: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
%0 = cir.call @ext_decl(%arg0, %arg1) : (!s32i, !s32i) -> !s32i
cir.return %0 : !s32i
}
// CHECK: cir.func{{.*}} @caller(%arg0: !s32i) -> !s32i
// CHECK: %[[R:.*]] = cir.call @ext_decl(%arg0) : (!s32i) -> !s32i
}
@@ -0,0 +1,42 @@
// RUN: cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \
// RUN: | FileCheck %s
// Same passthrough behavior as direct-passthrough-test-target.cir, but the
// classification is injected via a function attribute rather than computed
// by the test target. This is the driver mode that lets tests verify
// rewriter behavior against arbitrary classifications without depending on
// any real ABI target.
!s32i = !cir.int<s, 32>
!s64i = !cir.int<s, 64>
#all_direct_two_args = {
return = { kind = "direct" },
args = [ { kind = "direct" }, { kind = "direct" } ]
}
module attributes {
dlti.dl_spec = #dlti.dl_spec<
#dlti.dl_entry<i32, dense<32>: vector<2xi64>>,
#dlti.dl_entry<i64, dense<64>: vector<2xi64>>>
} {
cir.func @passthrough(%arg0: !s32i, %arg1: !s64i) -> !s32i
attributes { test_classify = #all_direct_two_args } {
cir.return %arg0 : !s32i
}
// CHECK: cir.func{{.*}} @passthrough(%arg0: !s32i, %arg1: !s64i) -> !s32i
// CHECK: cir.return %arg0 : !s32i
cir.func @caller(%arg0: !s32i, %arg1: !s64i) -> !s32i
attributes { test_classify = #all_direct_two_args } {
%0 = cir.call @passthrough(%arg0, %arg1) : (!s32i, !s64i) -> !s32i
cir.return %0 : !s32i
}
// CHECK: cir.func{{.*}} @caller(%arg0: !s32i, %arg1: !s64i) -> !s32i
// CHECK: %[[RES:.*]] = cir.call @passthrough(%arg0, %arg1) : (!s32i, !s64i) -> !s32i
// CHECK: cir.return %[[RES]] : !s32i
}
@@ -0,0 +1,35 @@
// RUN: cir-opt %s -cir-call-conv-lowering=target=test | FileCheck %s
!s32i = !cir.int<s, 32>
!s64i = !cir.int<s, 64>
module attributes {
dlti.dl_spec = #dlti.dl_spec<
#dlti.dl_entry<i1, dense<8>: vector<2xi64>>,
#dlti.dl_entry<i8, dense<8>: vector<2xi64>>,
#dlti.dl_entry<i16, dense<16>: vector<2xi64>>,
#dlti.dl_entry<i32, dense<32>: vector<2xi64>>,
#dlti.dl_entry<i64, dense<64>: vector<2xi64>>>
} {
// Register-sized integer args/returns are Direct under the test target.
// Direct (without coercion) is a true pass-through — function signature
// and call sites are unchanged.
cir.func @passthrough(%arg0: !s32i, %arg1: !s64i) -> !s32i {
cir.return %arg0 : !s32i
}
// CHECK: cir.func{{.*}} @passthrough(%arg0: !s32i, %arg1: !s64i) -> !s32i
// CHECK-NEXT: cir.return %arg0 : !s32i
cir.func @caller(%arg0: !s32i, %arg1: !s64i) -> !s32i {
%0 = cir.call @passthrough(%arg0, %arg1) : (!s32i, !s64i) -> !s32i
cir.return %0 : !s32i
}
// CHECK: cir.func{{.*}} @caller(%arg0: !s32i, %arg1: !s64i) -> !s32i
// CHECK-NEXT: %[[RES:.*]] = cir.call @passthrough(%arg0, %arg1) : (!s32i, !s64i) -> !s32i
// CHECK-NEXT: cir.return %[[RES]] : !s32i
}
@@ -0,0 +1,40 @@
// RUN: cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \
// RUN: | FileCheck %s
// Ignore'd argument: the rewriter drops it from both the function signature
// and call sites. Body uses of the arg (if any) get replaced with a
// poison constant so the IR stays well-typed without pretending we have a
// real value at the ABI boundary.
!s32i = !cir.int<s, 32>
#ignore_first_arg = {
return = { kind = "direct" },
args = [ { kind = "ignore" }, { kind = "direct" } ]
}
module attributes {
dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>>
} {
cir.func @callee(%arg0: !s32i, %arg1: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
cir.return %arg1 : !s32i
}
// The first arg is dropped; second arg is preserved.
// CHECK: cir.func{{.*}} @callee(%arg0: !s32i) -> !s32i
// CHECK-NEXT: cir.return %arg0 : !s32i
cir.func @caller(%arg0: !s32i, %arg1: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
%0 = cir.call @callee(%arg0, %arg1) : (!s32i, !s32i) -> !s32i
cir.return %0 : !s32i
}
// The call site drops the first arg too.
// CHECK: cir.func{{.*}} @caller(%arg0: !s32i) -> !s32i
// CHECK-NEXT: %[[R:.*]] = cir.call @callee(%arg0) : (!s32i) -> !s32i
// CHECK-NEXT: cir.return %[[R]] : !s32i
}
@@ -0,0 +1,49 @@
// RUN: cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \
// RUN: | FileCheck %s
// Ignore'd return: the rewriter changes the function's return type to void,
// drops the operand from every cir.return, and at call sites issues a void
// call. Any remaining uses of the original (non-void) call result get
// replaced with a poison constant of the original return type so the IR
// stays well-typed without pretending we have a real value at the ABI
// boundary.
!s32i = !cir.int<s, 32>
#ignore_return = {
return = { kind = "ignore" },
args = [ ]
}
module attributes {
dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>>
} {
cir.func @callee() -> !s32i
attributes { test_classify = #ignore_return } {
%0 = cir.const #cir.int<42> : !s32i
cir.return %0 : !s32i
}
// Return type becomes void (dropped from the func type), and the cir.return
// becomes operand-free.
// CHECK: cir.func{{.*}} @callee()
// CHECK-NOT: -> !s32i
// CHECK: cir.return
// CHECK-NOT: : !s32i
cir.func @caller_uses_result() -> !s32i
attributes { test_classify = #ignore_return } {
%0 = cir.call @callee() : () -> !s32i
cir.return %0 : !s32i
}
// Caller's own return is also classified Ignore, so its return type is
// also void. The use of @callee's result becomes a poison constant (the
// returned-but-ignored value is never observable).
// CHECK: cir.func{{.*}} @caller_uses_result()
// CHECK: cir.call @callee() : () -> ()
// CHECK: cir.const #cir.poison : !s32i
// CHECK: cir.return
}
@@ -0,0 +1,35 @@
// RUN: not cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \
// RUN: 2>&1 | FileCheck %s
!s32i = !cir.int<s, 32>
#ignore_first_arg = {
return = { kind = "direct" },
args = [ { kind = "ignore" }, { kind = "direct" } ]
}
#passthrough = {
return = { kind = "direct" },
args = [ { kind = "direct" }, { kind = "direct" }, { kind = "direct" } ]
}
module attributes {
dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>>
} {
cir.func @callee(%arg0: !s32i, %arg1: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
cir.return %arg1 : !s32i
}
cir.func @caller(%fp: !cir.ptr<!cir.func<(!s32i, !s32i) -> !s32i>>,
%arg0: !s32i, %arg1: !s32i) -> !s32i
attributes { test_classify = #passthrough } {
%0 = cir.call %fp(%arg0, %arg1)
: (!cir.ptr<!cir.func<(!s32i, !s32i) -> !s32i>>, !s32i, !s32i) -> !s32i
cir.return %0 : !s32i
}
}
// CHECK: error: 'cir.call' op indirect call not yet implemented in CallConvLowering
@@ -0,0 +1,31 @@
// RUN: not cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \
// RUN: 2>&1 | FileCheck %s
!s32i = !cir.int<s, 32>
#ignore_first_arg = {
return = { kind = "direct" },
args = [ { kind = "ignore" }, { kind = "direct" } ]
}
module attributes {
dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>>
} {
cir.func @callee(%arg0: !s32i, %arg1: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
cir.return %arg1 : !s32i
}
cir.func @caller(%arg0: !s32i, %arg1: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
cir.try_call @callee(%arg0, %arg1) ^bb1, ^bb2 : (!s32i, !s32i) -> !s32i
^bb1(%res: !s32i):
cir.return %res : !s32i
^bb2:
cir.return %arg1 : !s32i
}
}
// CHECK: error: 'cir.try_call' op TryCallOp not yet implemented in CallConvLowering
@@ -0,0 +1,29 @@
// RUN: not cir-opt %s -cir-call-conv-lowering="classification-attr=test_classify" \
// RUN: 2>&1 | FileCheck %s
!s32i = !cir.int<s, 32>
#ignore_first_arg = {
return = { kind = "direct" },
args = [ { kind = "ignore" }, { kind = "direct" } ]
}
module attributes {
dlti.dl_spec = #dlti.dl_spec<#dlti.dl_entry<i32, dense<32>: vector<2xi64>>>
} {
cir.func @callee(%arg0: !s32i, %arg1: !s32i, %arg2: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
cir.return %arg2 : !s32i
}
cir.func @caller(%arg0: !s32i, %arg1: !s32i, %arg2: !s32i) -> !s32i
attributes { test_classify = #ignore_first_arg } {
%0 = cir.call @callee(%arg0, %arg1, %arg2)
: (!s32i, !s32i, !s32i) -> !s32i
cir.return %0 : !s32i
}
}
// CHECK: error: 'cir.call' op variadic arguments not yet implemented in CallConvLowering
+4
View File
@@ -74,6 +74,10 @@ int main(int argc, char **argv) {
return mlir::createCXXABILoweringPass();
});
::mlir::registerPass([]() -> std::unique_ptr<::mlir::Pass> {
return mlir::createCallConvLoweringPass();
});
mlir::omp::registerOpenMPPasses();
mlir::registerTransformsPasses();
+4 -4
View File
@@ -128,12 +128,12 @@ public:
///
/// \param funcOp The function to rewrite (via FunctionOpInterface).
/// \param fc The ABI classification for this function.
/// \param rewriter The pattern rewriter to use for modifications.
/// \param builder The OpBuilder to use for modifications.
/// \returns success() if the function was rewritten.
virtual LogicalResult
rewriteFunctionDefinition(FunctionOpInterface funcOp,
const FunctionClassification &fc,
OpBuilder &rewriter) = 0;
OpBuilder &builder) = 0;
/// Rewrite a call operation to match the callee's ABI-lowered
/// signature.
@@ -143,11 +143,11 @@ public:
///
/// \param callOp The call operation to rewrite.
/// \param fc The ABI classification for the callee.
/// \param rewriter The pattern rewriter to use for modifications.
/// \param builder The OpBuilder to use for modifications.
/// \returns success() if the call was rewritten.
virtual LogicalResult rewriteCallSite(Operation *callOp,
const FunctionClassification &fc,
OpBuilder &rewriter) = 0;
OpBuilder &builder) = 0;
/// Return the dialect namespace this context handles (e.g. "cir").
virtual StringRef getDialectNamespace() const = 0;