Commit Graph
361 Commits
Author SHA1 Message Date
Matthias Braun 168c288af1 JumpThreading: Propagate branch weights in tryToUnfoldSelectInCurrBB (#66116)
Propagate "branch_weights" metadata whe turning a select into a
conditional branch in tryToUnfoldSelectInCurrBB
2023-09-12 13:36:49 -07:00
DianQK 7ded71b1e4 [JumpThreading] Invalidate LVI after combineMetadataForCSE. 2023-09-04 11:50:14 +08:00
DianQK 5855a4be9c [JumpThreading][NFC] Pre-commit for invalid LVI. 2023-09-04 11:50:07 +08:00
Matt Arsenault 6f2e943de6 InstSimplify: Handle folding fcmp with literal nans without a context instruction
Fixes reported assert after ddb3f12c42
2023-09-02 10:22:09 -04:00
Nikita Popov 4eafc9b6ff [IR] Treat callbr as special terminator (PR64215)
isLegalToHoistInto() currently return true for callbr instructions.
That means that a callbr with one successor will be considered a
proper loop preheader, which may result in instructions that use
the callbr return value being hoisted past it.

Fix this by adding callbr to isExceptionTerminator (with a rename
to isSpecialTerminator), which also fixes similar assumptions in
other places.

Fixes https://github.com/llvm/llvm-project/issues/64215.

Differential Revision: https://reviews.llvm.org/D158609
2023-08-25 09:20:18 +02:00
Noah Goldstein 39e9862e6b [ValueTracking] Use predicates for incoming phi-edges to deduce non-zero
This is basically a copy and paste of the same logic we do in
`computeKnownBits` but adapts it for just `isKnownNonZero`.

Differential Revision: https://reviews.llvm.org/D157801
2023-08-22 10:59:02 -05:00
Noah Goldstein 99d99eb3b0 [JumpThreading] Regenerate checks for test; NFC
Differential Revision: https://reviews.llvm.org/D157815
2023-08-22 10:59:02 -05:00
Matt Arsenault fa90f6b9d0 TTI: Pass function to hasBranchDivergence in a few passes
https://reviews.llvm.org/D152033
2023-07-07 09:49:38 -04:00
Nikita Popov 1379127481 [MemoryBuiltins] Handle phi nodes without operands (PR63013)
Conservatively return unknown in this degenerate case. This is
hard to hit in practice, because such phis are usually optimized
away before they reach a getObjectSize() call.

Fixes https://github.com/llvm/llvm-project/issues/63013.
2023-05-31 10:47:15 +02:00
Nikita Popov 92dc4fff87 [ValueTracking] Check for known bits conflict for shl nsw (PR62908)
I removed the conflict check from computeKnownBitsFromShiftOperator()
in D150648 assuming that this is now handled on the KnownBits side.
However, the nsw handling is still inside ValueTracking, so we
still need to handle conflicts there. Restore the check closer to
where it is relevant.

Fixes https://github.com/llvm/llvm-project/issues/62908.
2023-05-24 10:54:10 +02:00
luxufan f44c7dec67 [Local] Use most generic range if K does not dominate J or K doesn't have a !noundef
Since D141386 has changed the return value of !range from IUB to poison,
metadata !range shouldn't be preserved even if K dominates J.

If this patch was accepted, I plan to adjust metadata !nonnull as well.
BTW, I found that metadata !noundef is not handled in combineMetadata,
is this intentional?

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D142687
2023-03-23 19:31:55 +08:00
Evgeniy Brevnov a1b78fb929 [JT][CT] Preserve exisiting BPI/BFI during JumpThreading
Currently, JT creates and updates local instances of BPI\BFI. As a result global ones have to be invalidated if JT made any changes.
In fact, JT doesn't use any information from BPI/BFI for the sake of the transformation itself. It only creates BPI/BFI to keep them up to date. But since it updates local copies (besides cases when it updates profile metadata) it just waste of time.

Current patch is a rework of D124439. D124439 makes one step and replaces local copies with global ones retrieved through AnalysisPassManager. Here we do one more step and don't create BPI/BFI if the only reason of creation is to keep BPI/BFI up to date. Overall logic is the following. If there is cached BPI/BFI then update it along the transformations. If there is no existing BPI/BFI, then create it only if it is required to update profile metadata.

Please note if BPI/BFI exists on exit from JT (either cached or created) it is always up to date and no reason to invalidate it.

Reviewed By: mkazantsev

Differential Revision: https://reviews.llvm.org/D136827
2023-02-16 16:08:34 +07:00
Ben Mudd e0374fb2f4 [DebugInfo] Make debug intrinsics to track cloned values in JumpThreading
This patch causes debug value intrinsics outside of cloned blocks in the
Jump Threading pass to correctly point towards any derived values. If it cannot,
it kills them.

Reviewed By: probinson, StephenTozer

Differential Revision: https://reviews.llvm.org/D140404
2023-02-01 12:52:37 +00:00
Evgeniy Brevnov f7c1982309 Revert "[JT][CT] Preserve exisiting BPI/BFI during JumpThreading"
This reverts commit 26e7cb24cb.
2023-01-27 15:35:32 +07:00
Evgeniy Brevnov 26e7cb24cb [JT][CT] Preserve exisiting BPI/BFI during JumpThreading
Currently, JT creates and updates local instances of BPI\BFI. As a result global ones have to be invalidated if JT made any changes.
In fact, JT doesn't use any information from BPI/BFI for the sake of the transformation itself. It only creates BPI/BFI to keep them up to date. But since it updates local copies (besides cases when it updates profile metadata) it just waste of time.

Current patch is a rework of D124439. D124439 makes one step and replaces local copies with global ones retrieved through AnalysisPassManager. Here we do one more step and don't create BPI/BFI if the only reason of creation is to keep BPI/BFI up to date. Overall logic is the following. If there is cached BPI/BFI then update it along the transformations. If there is no existing BPI/BFI, then create it only if it is required to update profile metadata.

Please note if BPI/BFI exists on exit from JT (either cached or created) it is always up to date and no reason to invalidate it.

Reviewed By: mkazantsev

Differential Revision: https://reviews.llvm.org/D136827
2023-01-27 15:00:16 +07:00
Max Kazantsev 82cee24e3d [JumpThreading] Preserve profile metadata during select unfolding, take 2
Jump threading can replace select and unconditional branch with
conditional branch, but when doing so loses profile information.

This destructive transform can eventually lead to a performance
degradation due to folding of branches in
shouldFoldCondBranchesToCommonDestination as branch probabilities
are no longer known.

The first version was reverted due to assert caused by i32 overflow,
fixed in this version.

Patch by Roman Paukner!

Differential Revision: https://reviews.llvm.org/D138132
Reviewed By: mkazantsev
2023-01-16 19:04:23 +07:00
Dmitri Gribenko 0e9956204d Revert "[JumpThreading] Preserve profile metadata during select unfolding"
This reverts commit 957952dbf2.

Addition in the newly added code can overflow.  As a result, the
constructor of `BranchProbability()` can trigger an assertion. See
the discussion on https://reviews.llvm.org/D138132 for more details.
2023-01-10 11:54:50 +01:00
Alex Brachet 230dba0ad3 [JumpThreading] Fix test to require asserts 2023-01-09 22:51:08 +00:00
Ben Mudd 1f11d1bd12 [DebugInfo] Fix jump threading failing to update cloned dbg.values
This is a patch to fix duplicated dbg.values in the JumpThreading pass not
pointing towards their local value, and instead towards the variable in the
original block.
JumpThreadingPass::cloneInstructions is the changed function to target metadata
as well as normal cloned values.

Reviewed By: jmorse, StephenTozer

Differential Revision: https://reviews.llvm.org/D140006
2023-01-09 11:42:33 +00:00
Max Kazantsev 957952dbf2 [JumpThreading] Preserve profile metadata during select unfolding
Jump threading can replace select and unconditional branch with
conditional branch, but when doing so loses profile information.

This destructive transform can eventually lead to a performance
degradation due to folding of branches in
shouldFoldCondBranchesToCommonDestination as branch probabilities
are no longer known.

Patch by Roman Paukner!

Differential Revision: https://reviews.llvm.org/D138132
Reviewed By: mkazantsev
2023-01-09 16:14:58 +07:00
Yingchi Long 84733b0f17 [JT] check xor operand is exactly the same in processBranchOnXOR
Reproducer:

    ; RUN: opt -S -jump-threading < %s
    define void @test() {
    entry:
    br i1 false, label %loop, label %exit

    loop:
    %bool = phi i1 [ %xor, %loop.latch ], [ false, %entry ]
    %cmp = icmp eq i16 0, 1
    %xor = xor i1 %cmp, %bool
    br i1 %bool, label %loop.latch, label %exit

    loop.latch:
    %dummy = phi i16 [ 0, %loop ]
    br label %loop

    exit:
    ret void
    }

On this occassion, phi node %bool is actually %xor, and doing substitution causes assertion failure.

Fixes: https://github.com/llvm/llvm-project/issues/58812

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D139783
2022-12-21 21:43:55 +08:00
Bjorn Pettersson 3528e63d89 [test] Remove duplicate RUN lines in Transform tests 2022-12-08 11:47:16 +01:00
Roman Lebedev b487f2e96f [NFC] Port all JumpThreading tests to -passes= syntax 2022-12-08 02:38:45 +03:00
Matt Arsenault b6909fe9ca JumpThreading: Modernize some test checking
Stop using tests with grep, or no output checks at all. Just use
FileCheck and generate checks. Also remove unnecessary requires
asserts.
2022-11-27 12:41:28 -05:00
Matt Arsenault 07e34d2de5 JumpThreading: Convert tests to -passes 2022-11-27 11:19:28 -05:00
Matt Arsenault 83d5052768 JumpThreading: Convert tests to opaque pointers
phi-known.ll:test2 required deleting one manual check for a bitcast

Also strip trailing whitespace while we're touching everything.
2022-11-27 11:19:28 -05:00
Evgeniy Brevnov 50f8eb05af Revert "[JT] Preserve exisiting BPI/BFI during JumpThreading"
This reverts commit 52a4018506.
2022-11-17 17:11:47 +07:00
Evgeniy Brevnov 52a4018506 [JT] Preserve exisiting BPI/BFI during JumpThreading
Currently, JT creates and updates local instances of BPI\BFI. As a result global ones have to be invalidated if JT made any changes.
In fact, JT doesn't use any information from BPI/BFI for the sake of the transformation itself. It only creates BPI/BFI to keep them up to date. But since it updates local copies (besides cases when it updates profile metadata) it just waste of time.

Current patch is a rework of D124439. D124439 makes one step and replaces local copies with global ones retrieved through AnalysisPassManager. Here we do one more step and don't create BPI/BFI if the only reason of creation is to keep BPI/BFI up to date. Overall logic is the following. If there is cached BPI/BFI then update it along the transformations. If there is no existing BPI/BFI, then create it only if it is required to update profile metadata.

Please note if BPI/BFI exists on exit from JT (either cached or created) it is always up to date and no reason to invalidate it.

Differential Revision: https://reviews.llvm.org/D136827
2022-11-17 17:00:00 +07:00
Sebastian Peryt 46fc75ab28 [NFC][2/n] Remove PrunePH pass
Second patch in the series to remove legacy PM and
associated -enable-new-pm=0 flag targets pass that
has not been ported to new PM - PruneEH.
Discussion about this can be found in D44415.

Reviewed By: aeubanks

Differential Revision: https://reviews.llvm.org/D134686
2022-09-26 18:38:04 -07:00
Sergey Kachkov be37caca00 [JumpThreading] Process range comparisions with non-local cmp instructions
Use getPredicateOnEdge method if value is a non-local
compare-with-a-constant instruction, that can give more precise
results than getConstantOnEdge.

Differential Revision: https://reviews.llvm.org/D131956
2022-09-02 12:22:45 +02:00
Nikita Popov ab6876a40d reland: [Local] Allow creating callbr with duplicate successors
Since D129288, callbr is allowed to have duplicate successors. This patch removes a limitation which prevents optimizations from actually producing such callbrs.

This is probably the riskiest of all the recent callbr changes, because code with incorrect assumptions might be lurking somewhere. I fixed the one case I encountered ahead of time in https://github.com/llvm/llvm-project/commit/8201e3ef5c84561260218bc041209611aac690e3.

Reviewed By: nickdesaulniers

Differential Revision: https://reviews.llvm.org/D129997

Originally landed as
commit 08860f525a ("[Local] Allow creating callbr with duplicate successors")

Reverted in
commit 1cf6b93df1 ("Revert "[Local] Allow creating callbr with duplicate successors"")
2022-08-31 13:23:00 -07:00
Nick Desaulniers 1cf6b93df1 Revert "[Local] Allow creating callbr with duplicate successors"
This reverts commit 08860f525a.

Crashes during PPC64LE linux kernel builds as reported by @nathanchance.
https://reviews.llvm.org/D129997#3663632
2022-07-19 15:03:27 -07:00
Nikita Popov 08860f525a [Local] Allow creating callbr with duplicate successors
Since D129288, callbr is allowed to have duplicate successors. This
patch removes a limitation which prevents optimizations from actually
producing such callbrs.

Differential Revision: https://reviews.llvm.org/D129997
2022-07-19 14:28:22 +02:00
Nikita Popov 2a721374ae [IR] Don't use blockaddresses as callbr arguments
Following some recent discussions, this changes the representation
of callbrs in IR. The current blockaddress arguments are replaced
with `!` label constraints that refer directly to callbr indirect
destinations:

    ; Before:
    %res = callbr i8* asm "", "=r,r,i"(i8* %x, i8* blockaddress(@test8, %foo))
    to label %asm.fallthrough [label %foo]
    ; After:
    %res = callbr i8* asm "", "=r,r,!i"(i8* %x)
    to label %asm.fallthrough [label %foo]

The benefit of this is that we can easily update the successors of
a callbr, without having to worry about also updating blockaddress
references. This should allow us to remove some limitations:

* Allow unrolling/peeling/rotation of callbr, or any other
  clone-based optimizations
  (https://github.com/llvm/llvm-project/issues/41834)
* Allow duplicate successors
  (https://github.com/llvm/llvm-project/issues/45248)

This is just the IR representation change though, I will follow up
with patches to remove limtations in various transformation passes
that are no longer needed.

Differential Revision: https://reviews.llvm.org/D129288
2022-07-15 10:18:17 +02:00
ChenYang Li 6d036b83d1 [JumpThreading] Avoid threadThroughTwoBasicBlocks when PredPred BB ends with indirectbranch
Since we can't change the destination of indirectbr, so when
encounter indirectbr as PredPredBB terminator, we should pass it.

Differential Revision: https://reviews.llvm.org/D129193
2022-07-08 09:29:17 +02:00
Nikita Popov 40a4078e14 [BasicBlockUtils] Allow splitting predecessors with callbr terminators
SplitBlockPredecessors currently asserts if one of the predecessor
terminators is a callbr. This limitation was originally necessary,
because just like with indirectbr, it was not possible to replace
successors of a callbr. However, this is no longer the case since
D67252. As the requirement nowadays is that callbr must reference
all blockaddrs directly in the call arguments, and these get
automatically updated when setSuccessor() is called, we no longer
need this limitation.

The only thing we need to do here is use replaceSuccessorWith()
instead of replaceUsesOfWith(), because only the former does the
necessary blockaddr updating magic.

I believe there's other similar limitations that can be removed,
e.g. related to critical edge splitting.

Differential Revision: https://reviews.llvm.org/D129205
2022-07-07 09:13:25 +02:00
Arthur Eubanks 36096c2b38 [NFC][JumpThreading] Remove InsertFreezeWhenUnfoldingSelect pass parameter
All callers pass true.

select-unfold-freeze.ll is now a subset of select.ll so delete it.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D126501
2022-05-26 16:13:34 -07:00
Nikita Popov 6f0ca6fd23 [JumpThreading] Insert freeze when unfolding select
JumpThreading may convert selects into branch instructions,
in which case the condition needs to be frozen (as branch on
poison is immediate undefined behavior, unlike select on poison).

The necessary code for this is already in place, this just enables
the option.

Differential Revision: https://reviews.llvm.org/D125869
2022-05-21 11:24:27 +02:00
Nikita Popov c9e7049754 [JumpThreading] Look through freeze in getPredicateAt() fold
This code is valid for any icmp, so we can safely look through a
freeze when trying to find one.

A caveat here is that replaceFoldableUses() may not end up replacing
any uses in this case. It might make sense to use the freeze as the
context instruction (rather than the terminator) if there is a
freeze, to ensure that it always gets folded. This would require
some changes to how replaceFoldedUses() works though, as it
currently assumes that the value is valid at the end of the block.
2022-05-18 12:09:59 +02:00
Nikita Popov bdf25477f6 [JumpThreading] Add additional freeze tests (NFC)
These are for the getPredicateAt() codepath.
2022-05-18 12:03:15 +02:00
Juneyoung Lee 3adcf96b4f [JumpThreading] Let ProcessImpliedCondition look into freeze instructions
This patch makes JumpThreading's ProcessImpliedCondition deal with frozen
conditions.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D84941
2022-05-18 10:41:31 +09:00
Juneyoung Lee 85fb9ccfa3 Precommit a test file for D84941 2022-05-18 10:41:25 +09:00
Nikita Popov b3bd732835 [JumpThreading] Regenerate test checks (NFC) 2022-05-17 12:13:02 +02:00
Yevgeny Rouban 4fb8e0b862 [JumpThreading] Add a test for handling zero !prof branch_weights
The test was a part of the revision D81499 and should have been
added with commit 707836ed4e.

Reviewed By: yamauchi, wenlei
Differential Revision: https://reviews.llvm.org/D81499
2022-01-26 13:42:21 +07:00
Jessica Paquette acb8de565e [JumpThreading] Change asserts for WantInteger into actual checks
After e734e8286b, it is possible to end up in
a situation where an `indirectbr` is fed by a cast, which is in turn fed by
an operation which only produces integers.

`indirectbr` expects a block address, however these operations can't produce
that.

There were several asserts in `computeValueKnownInPredecessorsImpl` which check
that we're not looking for a block address if we're walking through something
which can never produce one.

Since it's now possible to hit these asserts, this changes them into actual
checks which return false if `Preference` is not `WantInteger`.

This adds a testcase which verifies that we don't crash anymore in these
situations.

Differential Revision: https://reviews.llvm.org/D99814
2022-01-14 11:15:14 -08:00
Nick Desaulniers 79ebc3b0dd [llvm][test] rewrite callbr to use i rather than X constraint NFC
In D115311, we're looking to modify clang to emit i constraints rather
than X constraints for callbr's indirect destinations. Prior to doing
so, update all of the existing tests in llvm/ to match.

Reviewed By: void, jyknight

Differential Revision: https://reviews.llvm.org/D115410
2022-01-11 11:31:08 -08:00
Bjorn Pettersson 297fb66484 Use a deterministic order when updating the DominatorTree
This solves a problem with non-deterministic output from opt due
to not performing dominator tree updates in a deterministic order.

The problem that was analysed indicated that JumpThreading was using
the DomTreeUpdater via llvm::MergeBasicBlockIntoOnlyPred. When
preparing the list of updates to send to DomTreeUpdater::applyUpdates
we iterated over a SmallPtrSet, which didn't give a well-defined
order of updates to perform.

The added domtree-updates.ll test case is an example that would
result in non-deterministic printouts of the domtree. Semantically
those domtree:s are equivalent, but it show the fact that when we
use the domtree iterator the order in which nodes are visited depend
on the order in which dominator tree updates are performed.

Since some passes (at least EarlyCSE) are iterating over nodes in the
dominator tree in a similar fashion as the domtree printer, then the
order in which transforms are applied by such passes, transitively,
also depend on the order in which dominator tree updates are
performed. And taking EarlyCSE as an example the end result could be
different depending on in which order the transforms are applied.

Reviewed By: nikic, kuhar

Differential Revision: https://reviews.llvm.org/D110292
2021-11-29 13:14:50 +01:00
Philip Reames e64ed3c8df [test] autogen a couple of additional tests 2021-10-05 18:58:08 -07:00
Hans Wennborg 1e9afab875 Re-apply "[JumpThreading] Ignore free instructions"
It seems the crashes we saw wasn't caused by this (see comments on the review).

> This is basically D108837 but for jump threading. Free instructions
> should be ignored for the threading decision. JumpThreading already
> skips some free instructions (like pointer bitcasts), but does not
> skip various free intrinsics -- in fact, it currently gives them a
> fairly large cost of 2.
>
> Differential Revision: https://reviews.llvm.org/D110290

This reverts commit 4604695d7c.
2021-09-24 18:52:30 +02:00
Hans Wennborg 4604695d7c Revert "[JumpThreading] Ignore free instructions"
It caused compiler crashes, see comment on the code review for repro.

> This is basically D108837 but for jump threading. Free instructions
> should be ignored for the threading decision. JumpThreading already
> skips some free instructions (like pointer bitcasts), but does not
> skip various free intrinsics -- in fact, it currently gives them a
> fairly large cost of 2.
>
> Differential Revision: https://reviews.llvm.org/D110290

This reverts commit 1e3c6fc7cb.
2021-09-24 16:14:53 +02:00