Commit Graph
48 Commits
Author SHA1 Message Date
bixia1 555e7835f4 [mlir][sparse] Fix rewriting for convert op and concatenate op.
Fix a problem in convert op rewriting where it used the original index for
ToIndicesOp.

Extend the concatenate op rewriting to handle dense destination and dynamic
shape destination.

Make the concatenate op integration test run on the codegen path.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D138057
2022-11-15 14:45:21 -08:00
Peiming Liu c3821e1684 [mlir][sparse] fix bugs in concatenate rewriter.
Reviewed By: aartbik, bixia

Differential Revision: https://reviews.llvm.org/D138053
2022-11-15 19:49:59 +00:00
Peiming Liu 8ffdcc594e [mlir][sparse] fix memory leak sparse2sparse reshape
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137994
2022-11-15 00:19:51 +00:00
bixia1 fa46de16db [mlir][sparse][NFC] Add comments to tests that are run for with and without runtime libraries.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137869
2022-11-14 08:21:33 -08:00
Peiming Liu 725e0849b7 [mlir][sparse] fix incorrect coordinates ordering computed by the foreach operation.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137877
2022-11-12 04:08:50 +00:00
bixia1 57416d872a [mlir][sparse] Fix a bug in rewriting dense2dense convert op.
Permutation wasn't handled correctly. Add a test for the rewriting.

Extend an integration test to run with enable_runtime_library=false to
also test the rewriting.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137845
2022-11-11 09:11:54 -08:00
bixia1 ccd1a5b9cb [mlir][sparse] Fix a bug in rewriting for the convert op.
The code to retrieve the number of entries isn't correct.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D137795
2022-11-10 14:30:51 -08:00
Aart Bik e6cbb91483 [mlir][sparse] skip zeros during dense2sparse
This enables the full matmul integration test with runtime_lib=true/false!

Background:
https://github.com/llvm/llvm-project/issues/51657

Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D137750
2022-11-09 20:54:27 -08:00
Aart Bik a61a9a700a [mlir][sparse] first end-to-end matmul with codegen
(1) also fixes memory leak in sparse2dense rewriting
(2) still needs fix in dense2sparse by skipping zeros

Reviewed By: wrengr

Differential Revision: https://reviews.llvm.org/D137736
2022-11-09 13:40:05 -08:00
Peiming Liu f45c21da0c [mlir][sparse] fix SSA chain issue in dense2sparse conversion.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137683
2022-11-09 02:03:33 +00:00
Peiming Liu 7175f9dde1 [mlir][sparse] extend foreach operation to iterator over sparse constant.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137679
2022-11-09 01:50:34 +00:00
Peiming Liu 4fa00ce15c [mlir][sparse] extend foreach operation to accept reduction arguments; fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly.
This patch re-commit D137468 and D137463, which were reverted by mistakes.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137579
2022-11-07 21:40:30 +00:00
Stella Stamenova 99171078bb Revert "[mlir][sparse] extend foreach operation to accept reduction arguments."
This reverts commit 53d5d34011.

This is causing a build failure on the windows mlir bot that was previously hidden by another sparse tensor change that caused failures:

https://lab.llvm.org/buildbot/#/builders/13/builds/28006
2022-11-07 09:34:10 -08:00
Stella Stamenova ec224e3b68 Revert "[mlir][sparse] fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly."
This reverts commit 70508b614e.

This change depends on a reverted change that broke the windows mlir buildbot; reverting to bring remaining mlir bots to green
2022-11-07 09:00:08 -08:00
Peiming Liu 70508b614e [mlir][sparse] fix sparse tensor rewriting patterns that do not propagate sparse tensor SSA properly.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137468
2022-11-04 23:43:54 +00:00
Peiming Liu 53d5d34011 [mlir][sparse] extend foreach operation to accept reduction arguments.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137463
2022-11-04 23:34:16 +00:00
bixia1 3e724e529b [mlir][sparse] Fix windows build.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D137210
2022-11-01 17:00:02 -07:00
bixia1 eb877006a6 [mlir][sparse] Add rewriting rule for the convert operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136301
2022-11-01 15:57:34 -07:00
Peiming Liu 63baab8b39 [mlir][sparse] Fold invariant op only when it has only one use.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136990
2022-10-28 23:42:28 +00:00
Alexander Belyaev b4db15a949 [mlir] Rename getInputs->getDpsInputs and getOutputs->getDpsInits in DPS interface.
https://discourse.llvm.org/t/rfc-interface-for-destination-style-ops/64056

Differential Revision: https://reviews.llvm.org/D136943
2022-10-28 15:41:12 +02:00
Peiming Liu b0f8057e4c [mlir][sparse] use loop emitter to generate loop in sparsification
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136185
2022-10-26 00:27:56 +00:00
Jeff Niu dc63ca78f7 [mlir][sparse] Fix build warning (NFC) 2022-10-21 13:13:07 -07:00
bixia1 e445349d2c [mlir][sparse] Add rewrite rule for the sparse_tensor.out operator.
Also fix the rewrite rule for sparse_tensor.new to reflect the recent change of
the runtime C interface and to use utilities for memref.alloca.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135891
2022-10-21 12:20:53 -07:00
Peiming Liu 35b3a0ce8d [mlir][sparse] support foreach on dense tensor.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136384
2022-10-21 00:12:37 +00:00
bixia1 cf7906cbbd [mlir][sparse] Fix getUnorderedCOOFromType for rank 1 tensor.
Previously, it used DimLevelType::SingletonNo to represent an unorder COO
tensor of rank 1 while it should use DimLevelType::CompressedNuNo.

Reviewed By: Peiming, wrengr

Differential Revision: https://reviews.llvm.org/D136387
2022-10-20 16:33:01 -07:00
wren romano 0e77b63bc0 [mlir][sparse] Use the runtime DimLevelType instead of a separate tablegen enum
This differential replaces all uses of SparseTensorEncodingAttr::DimLevelType with DimLevelType.  The next differential will break out a separate library for the DimLevelType enum, so that the Dialect code doesn't need to depend on the rest of the runtime

Depends On D135995

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135996
2022-10-18 13:45:26 -07:00
bixia1 c1864ab953 [mlir][sparse] Add options to sparse-tensor-rewrite to disable rewriting rules for operators foreach and convert.
This is to help simplify FileCheck tests for sparse-tensor-rewrite.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136093
2022-10-18 09:27:32 -07:00
bixia1 933e7e2393 [mlir][sparse] Change getUnorderedCOOFromType to propagate the overhead types.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D136112
2022-10-18 08:01:16 -07:00
Alexander Belyaev a7cccb9cbb [mlir] Simplify DestinationStyleOpInterface.
Differential Revision: https://reviews.llvm.org/D135348
2022-10-17 12:43:41 +02:00
Oleg Shyshkov c38d9cf20e [mlir] Remove iterator_types() method from LinalgStructuredInterface.
`getIteratorTypesArray` should be used instead. It's a better substitute for all the current usages of the interface.

The current `ArrayAttr iterator_types()` has a few problems:
* It creates an assumption operation has iterators types as an attribute, but it's not always the case. Sometime iterator types can be inferred from other attribute, or they're just static.
* ArrayAttr is an obscure contained and required extracting values in the client code.
* Makes it hard to migrate iterator types from strings to enums ([RFC](https://discourse.llvm.org/t/rfc-enumattr-for-iterator-types-in-linalg/64535/9)).

Concrete ops, like `linalg.generic` will still have iterator types as an attribute if needed.

As a side effect, this change helps a bit with migration to prefixed accessors.

Differential Revision: https://reviews.llvm.org/D135765
2022-10-13 07:52:43 +00:00
Kazu Hirata b2674de375 [mlir] Fix a warning
This patch fixes:

  mlir/lib/Dialect/SparseTensor/Transforms/SparseTensorRewriting.cpp:587:27:
  error: comparison of integers of different signs: 'int64_t' (aka
  'long') and 'uint64_t' (aka 'unsigned long')
  [-Werror,-Wsign-compare]
2022-10-12 15:25:17 -07:00
bixia1 67f61b0863 [mlir][sparse] Add rewriting rules for the sparse_tensor.new operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135692
2022-10-11 15:23:11 -07:00
bixia1 330d48c4aa [mlir][sparse] Add rewrite rules for sparse-to-sparse reshape operators.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D135077
2022-10-06 08:50:30 -07:00
Aart Bik c48e90877f [mlir][sparse] introduce a higher-order tensor mapping
This extension to the sparse tensor type system in MLIR
opens up a whole new set of sparse storage schemes, such as
block sparse storage (e.g. BCSR) and ELL (aka jagged diagonals).

This revision merely introduces the type extension and
initial documentation. The actual interpretation of the type
(reading in tensors, lowering to code, etc.) will follow.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D135206
2022-10-05 09:40:51 -07:00
Peiming Liu 00ad065548 [mlir][sparse] Add rewriting rules for concatente using foreach operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134895
2022-09-30 21:56:55 +00:00
Peiming Liu 550288cbc3 [mlir][sparse] Add new utility class to help generates loop structures over sparse tensors; Implement foreach operator.
Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D134782
2022-09-30 21:42:42 +00:00
Oleg Shyshkov 1227b8ab54 [mlir] Rename getTied* methods to getMatching* in LinalgInterface.
Summary:
As mentioned in the comment to https://reviews.llvm.org/D134444, the term `tied`
is a misnomer in this context and `matching` sounds much better.

Differential Revision: https://reviews.llvm.org/D134534
2022-09-30 10:05:45 +00:00
Jakub Kuderski abc362a107 [mlir][arith] Change dialect name from Arithmetic to Arith
Suggested by @lattner in https://discourse.llvm.org/t/rfc-define-precise-arith-semantics/65507/22.

Tested with:
`ninja check-mlir check-mlir-integration check-mlir-mlir-spirv-cpu-runner check-mlir-mlir-vulkan-runner check-mlir-examples`

and `bazel build --config=generic_clang @llvm-project//mlir:all`.

Reviewed By: lattner, Mogball, rriddle, jpienaar, mehdi_amini

Differential Revision: https://reviews.llvm.org/D134762
2022-09-29 11:23:28 -04:00
Peiming Liu eb65327fe9 [mlir][sparse] Add new option (enable-runtime-library) to sparse compiler pipeline
Add new option (enable-runtime-library) to sparse compiler pipeline, it allows us to decide whether we need to rewrite operations (e.g., concatenate, reshape) within sparsification (when using codegen) or convert them after sparsification (when using runtime library).

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D133597
2022-09-09 20:54:47 +00:00
Mehdi Amini 0449b6a07a Apply clang-tidy fixes for llvm-else-after-return in SparseTensorRewriting.cpp (NFC) 2022-09-07 19:30:54 +00:00
Aart Bik 0d4e7fba9a [mlir][sparse] minor zero test refactoring in rewriting
Reviewed By: bixia

Differential Revision: https://reviews.llvm.org/D133382
2022-09-07 10:07:11 -07:00
Peiming Liu ec495b53f8 [mlir][sparse] Folding operations that try to insert zero into an all-zero sparse tensor
The operations to fill zero into newly allocated sparse tensor are redundant, plus it failed
to lowering the test cases provided in the patch as well.

Reviewed By: aartbik

Differential Revision: https://reviews.llvm.org/D132500
2022-08-25 17:00:04 +00:00
Kazu Hirata 06b551c944 Use llvm::is_contained (NFC) 2022-08-20 21:18:27 -07:00
Jacques Pienaar d3b3f7653d [mlir] Flip to prefixed accessors (NFC) 2022-08-07 04:55:58 -07:00
Aart Bik c7bb69bc75 [mlir][sparse] replace zero yield generic op with copy in allocation
This prepares patterns that sometimes are generated by the front-end
and would prohibit fusion of SDDMM flavored kernels.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D131126
2022-08-04 09:33:57 -07:00
Aart Bik ce3d0e87ac [mlir][sparse] enable SDDMM-flavored fusion
This rewriting was no longer functional after recent migration to one shot
bufferization. However, this revision makes it work again, with a CHECK test
to ensure fusion happens. Note that functionality is tested by several
integration tests.

Reviewed By: Peiming

Differential Revision: https://reviews.llvm.org/D130996
2022-08-02 12:40:04 -07:00
Jacques Pienaar d2c0572b2e [mlir] Flip LinAlg dialect to _Both
This one required more changes than ideal due to overlapping generated name
with different return types. Changed getIndexingMaps to getIndexingMapsArray to
move it out of the way/highlight that it returns (more expensively) a
SmallVector and uses the prefixed name for the Attribute.

Differential Revision: https://reviews.llvm.org/D129919
2022-07-19 14:42:58 -07:00
Aart Bik 28ebb0b61d [mlir][sparse] migrate sparse rewriting to sparse transformations pass
The rules in the linalg file were very specific to sparse tensors so will
find a better home under sparse tensor dialect than linalg dialect. Also
moved some rewriting from sparsification into this new "pre-rewriting" file.

Reviewed By: springerm

Differential Revision: https://reviews.llvm.org/D129910
2022-07-18 09:29:22 -07:00