[CIR] Add InlineAsmOp

[el-ev]

• Part of [CIR] Upstream support for inline assembly #153267

Mostly copied from https://github.com/llvm/clangir/blob/main/clang/lib/CIR/Dialect/IR/CIRDialect.cpp and https://github.com/llvm/clangir/blob/main/clang/include/clang/CIR/Dialect/IR/CIROps.td. Fixed a parsing bug in llvm/clangir@3205aab.

[el-ev]

• [CIR] Implement codegen for inline assembly with output operands #154014
• [CIR] Implement codegen for inline assembly without input and output operands #153546
• [CIR] Add InlineAsmOp lowering to LLVM #153387
• [CIR] Add InlineAsmOp #153362 👈 (View in Graphite)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[llvmbot]

@llvm/pr-subscribers-clang

@llvm/pr-subscribers-clangir

Author: Iris Shi (el-ev)

Changes

• Part of #153267

Mostly copied from https://github.com/llvm/clangir/blob/main/clang/lib/CIR/Dialect/IR/CIRDialect.cpp and https://github.com/llvm/clangir/blob/main/clang/include/clang/CIR/Dialect/IR/CIROps.td. Fixed a parsing bug in llvm/clangir@3205aab.

---

Full diff: https://github.com/llvm/llvm-project/pull/153362.diff

3 Files Affected:

• (modified) clang/include/clang/CIR/Dialect/IR/CIROps.td (+83)
• (modified) clang/lib/CIR/Dialect/IR/CIRDialect.cpp (+202)
• (added) clang/test/CIR/IR/inline-asm.cir (+112)

diff --git a/clang/include/clang/CIR/Dialect/IR/CIROps.td b/clang/include/clang/CIR/Dialect/IR/CIROps.td
index 51cef239aeda2..803fb15adcfe7 100644
--- a/clang/include/clang/CIR/Dialect/IR/CIROps.td
+++ b/clang/include/clang/CIR/Dialect/IR/CIROps.td
@@ -2239,6 +2239,89 @@ def CIR_StackRestoreOp : CIR_Op<"stackrestore"> {
   let assemblyFormat = "$ptr attr-dict `:` qualified(type($ptr))";
 }
 
+//===----------------------------------------------------------------------===//
+// InlineAsmOp
+//===----------------------------------------------------------------------===//
+
+def CIR_AsmFlavor : CIR_I32EnumAttr<"AsmFlavor", "ATT or Intel",
+                                    [I32EnumAttrCase<"x86_att", 0>,
+                                     I32EnumAttrCase<"x86_intel", 1>]>;
+
+def CIR_InlineAsmOp : CIR_Op<"asm", [RecursiveMemoryEffects]> {
+  let description = [{
+    The `cir.asm` operation represents C/C++ asm inline.
+
+    CIR constraints strings follow barely the same rules that are established
+    for the C level assembler constraints with several differences caused by
+    clang::AsmStmt processing.
+
+    Thus, numbers that appears in the constraint string may also refer to:
+    - the output variable index referenced by the input operands.
+    - the index of early-clobber operand
+
+    Operand attributes are a storage, where each element corresponds to the
+    operand with the same index. The first index relates to the operation
+    result (if any).
+    Note, the operands themselves are stored as VariadicOfVariadic in the next
+    order: output, input and then in/out operands.
+
+    Note, when several output operands are present, the result type may be
+    represented as an anon record type.
+
+    Example:
+    ```C++
+    __asm__("foo" : : : );
+    __asm__("bar $42 %[val]" : [val] "=r" (x), "+&r"(x));
+    __asm__("baz $42 %[val]" : [val] "=r" (x), "+&r"(x) : "[val]"(y));
+    ```
+
+    ```mlir
+    !rec_22anon2E022 = !cir.record<struct "anon.0" {!cir.int<s, 32>, !cir.int<s, 32>}>
+    !rec_22anon2E122 = !cir.record<struct "anon.1" {!cir.int<s, 32>, !cir.int<s, 32>}>
+    ...
+    %0 = cir.alloca !s32i, !cir.ptr<!s32i>, ["x", init]
+    %1 = cir.alloca !s32i, !cir.ptr<!s32i>, ["y", init]
+    ...
+    %2 = cir.load %0 : !cir.ptr<!s32i>, !s32i
+    %3 = cir.load %1 : !cir.ptr<!s32i>, !s32i
+
+    cir.asm(x86_att,
+      out = [],
+      in = [],
+      in_out = [],
+      {"foo" "~{dirflag},~{fpsr},~{flags}"}) side_effects
+
+    cir.asm(x86_att,
+      out = [],
+      in = [],
+      in_out = [%2 : !s32i],
+      {"bar $$42 $0" "=r,=&r,1,~{dirflag},~{fpsr},~{flags}"}) -> !rec_22anon2E022
+
+    cir.asm(x86_att,
+      out = [],
+      in = [%3 : !s32i],
+      in_out = [%2 : !s32i],
+      {"baz $$42 $0" "=r,=&r,0,1,~{dirflag},~{fpsr},~{flags}"}) -> !rec_22anon2E122
+    ```
+  }];
+
+  let results = (outs Optional<CIR_AnyType>:$res);
+
+  let arguments =
+      (ins VariadicOfVariadic<AnyType, "operands_segments">:$asm_operands,
+          StrAttr:$asm_string, StrAttr:$constraints, UnitAttr:$side_effects,
+          CIR_AsmFlavor:$asm_flavor, ArrayAttr:$operand_attrs,
+          DenseI32ArrayAttr:$operands_segments);
+
+  let builders = [OpBuilder<(ins
+      "llvm::ArrayRef<mlir::ValueRange>":$asmOperands,
+      "llvm::StringRef":$asmString, "llvm::StringRef":$constraints,
+      "bool":$sideEffects, "AsmFlavor":$asmFlavor,
+      "llvm::ArrayRef<mlir::Attribute>":$operandAttrs)>];
+
+  let hasCustomAssemblyFormat = 1;
+}
+
 //===----------------------------------------------------------------------===//
 // UnreachableOp
 //===----------------------------------------------------------------------===//
diff --git a/clang/lib/CIR/Dialect/IR/CIRDialect.cpp b/clang/lib/CIR/Dialect/IR/CIRDialect.cpp
index 7c8429432b0f9..40a0a206f2259 100644
--- a/clang/lib/CIR/Dialect/IR/CIRDialect.cpp
+++ b/clang/lib/CIR/Dialect/IR/CIRDialect.cpp
@@ -2406,6 +2406,208 @@ OpFoldResult RotateOp::fold(FoldAdaptor adaptor) {
   return IntAttr::get(input.getContext(), input.getType(), resultValue);
 }
 
+//===----------------------------------------------------------------------===//
+// InlineAsmOp
+//===----------------------------------------------------------------------===//
+
+void cir::InlineAsmOp::print(OpAsmPrinter &p) {
+  p << '(' << getAsmFlavor() << ", ";
+  p.increaseIndent();
+  p.printNewline();
+
+  llvm::SmallVector<std::string, 3> names{"out", "in", "in_out"};
+  auto *nameIt = names.begin();
+  auto *attrIt = getOperandAttrs().begin();
+
+  for (auto ops : getAsmOperands()) {
+    p << *nameIt << " = ";
+
+    p << '[';
+    llvm::interleaveComma(llvm::make_range(ops.begin(), ops.end()), p,
+                          [&](Value value) {
+                            p.printOperand(value);
+                            p << " : " << value.getType();
+                            if (*attrIt)
+                              p << " (maybe_memory)";
+                            attrIt++;
+                          });
+    p << "],";
+    p.printNewline();
+    ++nameIt;
+  }
+
+  p << "{";
+  p.printString(getAsmString());
+  p << " ";
+  p.printString(getConstraints());
+  p << "}";
+  p.decreaseIndent();
+  p << ')';
+  if (getSideEffects())
+    p << " side_effects";
+
+  llvm::SmallVector<::llvm::StringRef, 2> elidedAttrs;
+  elidedAttrs.push_back("asm_flavor");
+  elidedAttrs.push_back("asm_string");
+  elidedAttrs.push_back("constraints");
+  elidedAttrs.push_back("operand_attrs");
+  elidedAttrs.push_back("operands_segments");
+  elidedAttrs.push_back("side_effects");
+  p.printOptionalAttrDict(getOperation()->getAttrs(), elidedAttrs);
+
+  if (auto v = getRes())
+    p << " -> " << v.getType();
+}
+
+void cir::InlineAsmOp::build(OpBuilder &odsBuilder, OperationState &odsState,
+                             ArrayRef<ValueRange> asmOperands,
+                             StringRef asmString, StringRef constraints,
+                             bool sideEffects, cir::AsmFlavor asmFlavor,
+                             ArrayRef<Attribute> operandAttrs) {
+  // Set up the operands_segments for VariadicOfVariadic
+  SmallVector<int32_t> segments;
+  for (auto operandRange : asmOperands) {
+    segments.push_back(operandRange.size());
+    odsState.addOperands(operandRange);
+  }
+
+  odsState.addAttribute(
+      "operands_segments",
+      DenseI32ArrayAttr::get(odsBuilder.getContext(), segments));
+  odsState.addAttribute("asm_string", odsBuilder.getStringAttr(asmString));
+  odsState.addAttribute("constraints", odsBuilder.getStringAttr(constraints));
+  odsState.addAttribute("asm_flavor",
+                        AsmFlavorAttr::get(odsBuilder.getContext(), asmFlavor));
+
+  if (sideEffects)
+    odsState.addAttribute("side_effects", odsBuilder.getUnitAttr());
+
+  odsState.addAttribute("operand_attrs", odsBuilder.getArrayAttr(operandAttrs));
+}
+
+ParseResult cir::InlineAsmOp::parse(OpAsmParser &parser,
+                                    OperationState &result) {
+  llvm::SmallVector<mlir::Attribute> operandAttrs;
+  llvm::SmallVector<int32_t> operandsGroupSizes;
+  std::string asmString, constraints;
+  Type resType;
+  auto *ctxt = parser.getBuilder().getContext();
+
+  auto error = [&](const Twine &msg) -> LogicalResult {
+    return parser.emitError(parser.getCurrentLocation(), msg);
+  };
+
+  auto expected = [&](const std::string &c) {
+    return error("expected '" + c + "'");
+  };
+
+  if (parser.parseLParen().failed())
+    return expected("(");
+
+  auto flavor = FieldParser<AsmFlavor, AsmFlavor>::parse(parser);
+  if (failed(flavor))
+    return error("Unknown AsmFlavor");
+
+  if (parser.parseComma().failed())
+    return expected(",");
+
+  auto parseValue = [&](Value &v) {
+    OpAsmParser::UnresolvedOperand op;
+
+    if (parser.parseOperand(op) || parser.parseColon())
+      return mlir::failure();
+
+    Type typ;
+    if (parser.parseType(typ).failed())
+      return error("can't parse operand type");
+    llvm::SmallVector<mlir::Value> tmp;
+    if (parser.resolveOperand(op, typ, tmp))
+      return error("can't resolve operand");
+    v = tmp[0];
+    return mlir::success();
+  };
+
+  auto parseOperands = [&](llvm::StringRef name) {
+    if (parser.parseKeyword(name).failed())
+      return error("expected " + name + " operands here");
+    if (parser.parseEqual().failed())
+      return expected("=");
+    if (parser.parseLSquare().failed())
+      return expected("[");
+
+    int size = 0;
+    if (parser.parseOptionalRSquare().succeeded()) {
+      operandsGroupSizes.push_back(size);
+      if (parser.parseComma())
+        return expected(",");
+      return mlir::success();
+    }
+
+    if (parser.parseCommaSeparatedList([&]() {
+          Value val;
+          if (parseValue(val).succeeded()) {
+            result.operands.push_back(val);
+            size++;
+
+            if (parser.parseOptionalLParen().failed()) {
+              operandAttrs.push_back(mlir::Attribute());
+              return mlir::success();
+            }
+
+            if (parser.parseKeyword("maybe_memory").succeeded()) {
+              operandAttrs.push_back(mlir::UnitAttr::get(ctxt));
+              if (parser.parseRParen())
+                return expected(")");
+              return mlir::success();
+            }
+          }
+          return mlir::failure();
+        }))
+      return mlir::failure();
+
+    if (parser.parseRSquare().failed() || parser.parseComma().failed())
+      return expected("]");
+    operandsGroupSizes.push_back(size);
+    return mlir::success();
+  };
+
+  if (parseOperands("out").failed() || parseOperands("in").failed() ||
+      parseOperands("in_out").failed())
+    return error("failed to parse operands");
+
+  if (parser.parseLBrace())
+    return expected("{");
+  if (parser.parseString(&asmString))
+    return error("asm string parsing failed");
+  if (parser.parseString(&constraints))
+    return error("constraints string parsing failed");
+  if (parser.parseRBrace())
+    return expected("}");
+  if (parser.parseRParen())
+    return expected(")");
+
+  if (parser.parseOptionalKeyword("side_effects").succeeded())
+    result.attributes.set("side_effects", UnitAttr::get(ctxt));
+
+  if (parser.parseOptionalArrow().succeeded())
+    if (parser.parseType(resType).failed())
+      return mlir::failure();
+
+  if (parser.parseOptionalAttrDict(result.attributes))
+    return mlir::failure();
+
+  result.attributes.set("asm_flavor", AsmFlavorAttr::get(ctxt, *flavor));
+  result.attributes.set("asm_string", StringAttr::get(ctxt, asmString));
+  result.attributes.set("constraints", StringAttr::get(ctxt, constraints));
+  result.attributes.set("operand_attrs", ArrayAttr::get(ctxt, operandAttrs));
+  result.getOrAddProperties<InlineAsmOp::Properties>().operands_segments =
+      parser.getBuilder().getDenseI32ArrayAttr(operandsGroupSizes);
+  if (resType)
+    result.addTypes(TypeRange{resType});
+
+  return mlir::success();
+}
+
 //===----------------------------------------------------------------------===//
 // TableGen'd op method definitions
 //===----------------------------------------------------------------------===//
diff --git a/clang/test/CIR/IR/inline-asm.cir b/clang/test/CIR/IR/inline-asm.cir
new file mode 100644
index 0000000000000..fb1f6315a7d72
--- /dev/null
+++ b/clang/test/CIR/IR/inline-asm.cir
@@ -0,0 +1,112 @@
+// RUN: cir-opt %s | FileCheck %s
+
+!s32i = !cir.int<s, 32>
+!u32i = !cir.int<u, 32>
+
+module {
+cir.func @f1() {
+  // CHECK: cir.asm(x86_att, 
+  // CHECK:   out = [],
+  // CHECK:   in = [],
+  // CHECK:   in_out = [],
+  // CHECK:   {"" "~{dirflag},~{fpsr},~{flags}"})
+  cir.asm(x86_att, 
+    out = [],
+    in = [],
+    in_out = [],
+    {"" "~{dirflag},~{fpsr},~{flags}"})
+  cir.return
+}
+
+cir.func @f2() {
+  // CHECK: cir.asm(x86_att, 
+  // CHECK:   out = [],
+  // CHECK:   in = [],
+  // CHECK:   in_out = [],
+  // CHECK:   {"" "~{dirflag},~{fpsr},~{flags}"}) side_effects
+  cir.asm(x86_att, 
+    out = [],
+    in = [],
+    in_out = [],
+    {"" "~{dirflag},~{fpsr},~{flags}"}) side_effects
+  cir.return
+}
+
+cir.func @f3() {
+  // CHECK: cir.asm(x86_att, 
+  // CHECK:   out = [],
+  // CHECK:   in = [],
+  // CHECK:   in_out = [],
+  // CHECK:   {"abc" "~{dirflag},~{fpsr},~{flags}"}) side_effects
+  cir.asm(x86_att,
+    out = [],
+    in = [],
+    in_out = [],
+    {"abc" "~{dirflag},~{fpsr},~{flags}"}) side_effects
+  cir.return
+}
+
+cir.func @f4(%arg0: !s32i) {
+  %0 = cir.alloca !s32i, !cir.ptr<!s32i>, ["x", init] {alignment = 4 : i64}
+  cir.store %arg0, %0 : !s32i, !cir.ptr<!s32i>
+  // CHECK: cir.asm(x86_att, 
+  // CHECK:   out = [],
+  // CHECK:   in = [%0 : !cir.ptr<!s32i> (maybe_memory)],
+  // CHECK:   in_out = [],
+  // CHECK:   {"" "*m,~{dirflag},~{fpsr},~{flags}"}) side_effects
+  cir.asm(x86_att, 
+    out = [],
+    in = [%0 : !cir.ptr<!s32i> (maybe_memory)],
+    in_out = [],
+    {"" "*m,~{dirflag},~{fpsr},~{flags}"}) side_effects
+  cir.return
+}
+
+cir.func @f5() {
+  // CHECK: cir.asm(x86_intel, 
+  // CHECK:   out = [],
+  // CHECK:   in = [],
+  // CHECK:   in_out = [],
+  // CHECK:   {"" "~{dirflag},~{fpsr},~{flags}"})
+  cir.asm(x86_intel, 
+    out = [],
+    in = [],
+    in_out = [],
+    {"" "~{dirflag},~{fpsr},~{flags}"})
+  cir.return
+}
+cir.func @f6() -> !s32i {
+  %0 = cir.alloca !s32i, !cir.ptr<!s32i>, ["x", init] {alignment = 4 : i64}
+  // CHECK: %1 = cir.asm(x86_att, 
+  // CHECK:   out = [],
+  // CHECK:   in = [],
+  // CHECK:   in_out = [],
+  // CHECK:   {"movl $$42, $0" "=r,~{dirflag},~{fpsr},~{flags}"}) side_effects -> !s32i
+  %1 = cir.asm(x86_att, 
+    out = [],
+    in = [],
+    in_out = [],
+    {"movl $$42, $0" "=r,~{dirflag},~{fpsr},~{flags}"}) side_effects -> !s32i
+  cir.store align(4) %1, %0 : !s32i, !cir.ptr<!s32i>
+  %3 = cir.load align(4) %0 : !cir.ptr<!s32i>, !s32i
+  cir.return %3 : !s32i
+}
+cir.func @f7(%arg0: !u32i) -> !u32i {
+  %0 = cir.alloca !u32i, !cir.ptr<!u32i>, ["x", init] {alignment = 4 : i64}
+  cir.store %arg0, %0 : !u32i, !cir.ptr<!u32i>
+  %1 = cir.load align(4) %0 : !cir.ptr<!u32i>, !u32i
+  // CHECK: %2 = cir.asm(x86_att, 
+  // CHECK:   out = [],
+  // CHECK:   in = [],
+  // CHECK:   in_out = [%1 : !u32i],
+  // CHECK:   {"addl $$42, $0" "=r,0,~{dirflag},~{fpsr},~{flags}"}) side_effects -> !u32i
+  %2 = cir.asm(x86_att, 
+    out = [],
+    in = [],
+    in_out = [%1 : !u32i],
+    {"addl $$42, $0" "=r,0,~{dirflag},~{fpsr},~{flags}"}) side_effects -> !u32i
+   cir.store align(4) %2, %0 : !u32i, !cir.ptr<!u32i>
+   %3 = cir.load align(4) %0 : !cir.ptr<!u32i>, !u32i
+   cir.return %3 : !u32i
+}
+}

[andykaylor]

Thanks for working on this! I have a bunch of minor comments, but overall this looks good.

[mmha]

LGTM

[el-ev]

Merge activity

• Aug 14, 4:27 PM UTC: Graphite rebased this pull request as part of a merge.
• Aug 14, 4:36 PM UTC: Graphite rebased this pull request as part of a merge.
• Aug 14, 4:39 PM UTC: Graphite rebased this pull request as part of a merge.
• Aug 14, 4:50 PM UTC: Graphite rebased this pull request as part of a merge.
• Aug 14, 4:53 PM UTC: Graphite rebased this pull request as part of a merge.

[andykaylor]

lgtm. I have just one last nit.