Skip to content

[AMDGPU][True16][GISel] Support v_add_f16 codegen. #74122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Dec 4, 2023
Merged

[AMDGPU][True16][GISel] Support v_add_f16 codegen. #74122

merged 2 commits into from
Dec 4, 2023

Conversation

kosarev
Copy link
Collaborator

@kosarev kosarev commented Dec 1, 2023

No description provided.

@llvmbot
Copy link
Member

llvmbot commented Dec 1, 2023

@llvm/pr-subscribers-backend-amdgpu

Author: Ivan Kosarev (kosarev)

Changes

Patch is 31.52 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/74122.diff

2 Files Affected:

  • (modified) llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp (+2-1)
  • (modified) llvm/test/CodeGen/AMDGPU/fadd.f16.ll (+454-123)
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
index abb7c41e3385ab6..a7b0dfda7852aa8 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
@@ -3084,7 +3084,8 @@ SIRegisterInfo::getRegClassForSizeOnBank(unsigned Size,
                                          const RegisterBank &RB) const {
   switch (RB.getID()) {
   case AMDGPU::VGPRRegBankID:
-    return getVGPRClassForBitWidth(std::max(32u, Size));
+    return getVGPRClassForBitWidth(
+        std::max(ST.useRealTrue16Insts() ? 16u : 32u, Size));
   case AMDGPU::VCCRegBankID:
     assert(Size == 1);
     return isWave32 ? &AMDGPU::SReg_32_XM0_XEXECRegClass
diff --git a/llvm/test/CodeGen/AMDGPU/fadd.f16.ll b/llvm/test/CodeGen/AMDGPU/fadd.f16.ll
index 00f7eacaeb55507..444d6122eb7311c 100644
--- a/llvm/test/CodeGen/AMDGPU/fadd.f16.ll
+++ b/llvm/test/CodeGen/AMDGPU/fadd.f16.ll
@@ -1,8 +1,10 @@
 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
 ; RUN: llc -march=amdgcn -mcpu=tahiti -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=SI %s
 ; RUN: llc -march=amdgcn -mcpu=fiji -mattr=-flat-for-global -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=VI %s
-; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=+real-true16,-flat-for-global -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=GFX11 %s
-; RUN: llc -march=amdgcn -mcpu=gfx1100 -mattr=-flat-for-global -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=GFX11-FAKE16 %s
+; RUN: llc -march=amdgcn -mcpu=gfx1100 -global-isel=0 -mattr=+real-true16,-flat-for-global -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=GFX11-SDAG %s
+; RUN: llc -march=amdgcn -mcpu=gfx1100 -global-isel=1 -mattr=+real-true16,-flat-for-global -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=GFX11-GISEL %s
+; RUN: llc -march=amdgcn -mcpu=gfx1100 -global-isel=0 -mattr=-real-true16,-flat-for-global -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=GFX11-FAKE16-SDAG %s
+; RUN: llc -march=amdgcn -mcpu=gfx1100 -global-isel=1 -mattr=-real-true16,-flat-for-global -verify-machineinstrs -enable-misched=false < %s | FileCheck -check-prefixes=GFX11-FAKE16-GISEL %s
 
 define amdgpu_kernel void @fadd_f16(
 ; SI-LABEL: fadd_f16:
@@ -54,6 +56,103 @@ define amdgpu_kernel void @fadd_f16(
 ; VI-NEXT:    buffer_store_short v0, off, s[8:11], 0
 ; VI-NEXT:    s_endpgm
 ;
+; GFX11-SDAG-LABEL: fadd_f16:
+; GFX11-SDAG:       ; %bb.0: ; %entry
+; GFX11-SDAG-NEXT:    s_clause 0x1
+; GFX11-SDAG-NEXT:    s_load_b128 s[4:7], s[0:1], 0x24
+; GFX11-SDAG-NEXT:    s_load_b64 s[0:1], s[0:1], 0x34
+; GFX11-SDAG-NEXT:    s_mov_b32 s11, 0x31016000
+; GFX11-SDAG-NEXT:    s_mov_b32 s10, -1
+; GFX11-SDAG-NEXT:    s_mov_b32 s3, s11
+; GFX11-SDAG-NEXT:    s_mov_b32 s2, s10
+; GFX11-SDAG-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-SDAG-NEXT:    s_mov_b32 s8, s4
+; GFX11-SDAG-NEXT:    s_mov_b32 s9, s5
+; GFX11-SDAG-NEXT:    s_mov_b32 s4, s6
+; GFX11-SDAG-NEXT:    s_mov_b32 s5, s7
+; GFX11-SDAG-NEXT:    s_mov_b32 s6, s10
+; GFX11-SDAG-NEXT:    s_mov_b32 s7, s11
+; GFX11-SDAG-NEXT:    buffer_load_u16 v0, off, s[4:7], 0 glc dlc
+; GFX11-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-SDAG-NEXT:    buffer_load_u16 v1, off, s[0:3], 0 glc dlc
+; GFX11-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-SDAG-NEXT:    v_mov_b16_e32 v0.h, v1.l
+; GFX11-SDAG-NEXT:    s_delay_alu instid0(VALU_DEP_1)
+; GFX11-SDAG-NEXT:    v_add_f16_e32 v0.l, v0.l, v0.h
+; GFX11-SDAG-NEXT:    buffer_store_b16 v0, off, s[8:11], 0
+; GFX11-SDAG-NEXT:    s_nop 0
+; GFX11-SDAG-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-SDAG-NEXT:    s_endpgm
+;
+; GFX11-GISEL-LABEL: fadd_f16:
+; GFX11-GISEL:       ; %bb.0: ; %entry
+; GFX11-GISEL-NEXT:    s_clause 0x1
+; GFX11-GISEL-NEXT:    s_load_b128 s[4:7], s[0:1], 0x24
+; GFX11-GISEL-NEXT:    s_load_b64 s[0:1], s[0:1], 0x34
+; GFX11-GISEL-NEXT:    s_mov_b32 s10, -1
+; GFX11-GISEL-NEXT:    s_mov_b32 s11, 0x31016000
+; GFX11-GISEL-NEXT:    s_delay_alu instid0(SALU_CYCLE_1)
+; GFX11-GISEL-NEXT:    s_mov_b64 s[2:3], s[10:11]
+; GFX11-GISEL-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-GISEL-NEXT:    s_mov_b64 s[8:9], s[6:7]
+; GFX11-GISEL-NEXT:    s_mov_b64 s[6:7], s[10:11]
+; GFX11-GISEL-NEXT:    buffer_load_u16 v0, off, s[8:11], 0 glc dlc
+; GFX11-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-GISEL-NEXT:    buffer_load_u16 v1, off, s[0:3], 0 glc dlc
+; GFX11-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-GISEL-NEXT:    v_add_f16_e32 v0.l, v0.l, v1.l
+; GFX11-GISEL-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
+; GFX11-GISEL-NEXT:    s_nop 0
+; GFX11-GISEL-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-GISEL-NEXT:    s_endpgm
+;
+; GFX11-FAKE16-SDAG-LABEL: fadd_f16:
+; GFX11-FAKE16-SDAG:       ; %bb.0: ; %entry
+; GFX11-FAKE16-SDAG-NEXT:    s_clause 0x1
+; GFX11-FAKE16-SDAG-NEXT:    s_load_b128 s[4:7], s[0:1], 0x24
+; GFX11-FAKE16-SDAG-NEXT:    s_load_b64 s[0:1], s[0:1], 0x34
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s11, 0x31016000
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s10, -1
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s3, s11
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s2, s10
+; GFX11-FAKE16-SDAG-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s8, s4
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s9, s5
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s4, s6
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s5, s7
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s6, s10
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s7, s11
+; GFX11-FAKE16-SDAG-NEXT:    buffer_load_u16 v0, off, s[4:7], 0 glc dlc
+; GFX11-FAKE16-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-SDAG-NEXT:    buffer_load_u16 v1, off, s[0:3], 0 glc dlc
+; GFX11-FAKE16-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-SDAG-NEXT:    v_add_f16_e32 v0, v0, v1
+; GFX11-FAKE16-SDAG-NEXT:    buffer_store_b16 v0, off, s[8:11], 0
+; GFX11-FAKE16-SDAG-NEXT:    s_nop 0
+; GFX11-FAKE16-SDAG-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-FAKE16-SDAG-NEXT:    s_endpgm
+;
+; GFX11-FAKE16-GISEL-LABEL: fadd_f16:
+; GFX11-FAKE16-GISEL:       ; %bb.0: ; %entry
+; GFX11-FAKE16-GISEL-NEXT:    s_clause 0x1
+; GFX11-FAKE16-GISEL-NEXT:    s_load_b128 s[4:7], s[0:1], 0x24
+; GFX11-FAKE16-GISEL-NEXT:    s_load_b64 s[0:1], s[0:1], 0x34
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b32 s10, -1
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b32 s11, 0x31016000
+; GFX11-FAKE16-GISEL-NEXT:    s_delay_alu instid0(SALU_CYCLE_1)
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b64 s[2:3], s[10:11]
+; GFX11-FAKE16-GISEL-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b64 s[8:9], s[6:7]
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b64 s[6:7], s[10:11]
+; GFX11-FAKE16-GISEL-NEXT:    buffer_load_u16 v0, off, s[8:11], 0 glc dlc
+; GFX11-FAKE16-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-GISEL-NEXT:    buffer_load_u16 v1, off, s[0:3], 0 glc dlc
+; GFX11-FAKE16-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-GISEL-NEXT:    v_add_f16_e32 v0, v0, v1
+; GFX11-FAKE16-GISEL-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
+; GFX11-FAKE16-GISEL-NEXT:    s_nop 0
+; GFX11-FAKE16-GISEL-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-FAKE16-GISEL-NEXT:    s_endpgm
 ; GFX11-LABEL: fadd_f16:
 ; GFX11:       ; %bb.0: ; %entry
 ; GFX11-NEXT:    s_clause 0x1
@@ -81,32 +180,6 @@ define amdgpu_kernel void @fadd_f16(
 ; GFX11-NEXT:    s_nop 0
 ; GFX11-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
 ; GFX11-NEXT:    s_endpgm
-;
-; GFX11-FAKE16-LABEL: fadd_f16:
-; GFX11-FAKE16:       ; %bb.0: ; %entry
-; GFX11-FAKE16-NEXT:    s_clause 0x1
-; GFX11-FAKE16-NEXT:    s_load_b128 s[4:7], s[0:1], 0x24
-; GFX11-FAKE16-NEXT:    s_load_b64 s[0:1], s[0:1], 0x34
-; GFX11-FAKE16-NEXT:    s_mov_b32 s11, 0x31016000
-; GFX11-FAKE16-NEXT:    s_mov_b32 s10, -1
-; GFX11-FAKE16-NEXT:    s_mov_b32 s3, s11
-; GFX11-FAKE16-NEXT:    s_mov_b32 s2, s10
-; GFX11-FAKE16-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX11-FAKE16-NEXT:    s_mov_b32 s8, s4
-; GFX11-FAKE16-NEXT:    s_mov_b32 s9, s5
-; GFX11-FAKE16-NEXT:    s_mov_b32 s4, s6
-; GFX11-FAKE16-NEXT:    s_mov_b32 s5, s7
-; GFX11-FAKE16-NEXT:    s_mov_b32 s6, s10
-; GFX11-FAKE16-NEXT:    s_mov_b32 s7, s11
-; GFX11-FAKE16-NEXT:    buffer_load_u16 v0, off, s[4:7], 0 glc dlc
-; GFX11-FAKE16-NEXT:    s_waitcnt vmcnt(0)
-; GFX11-FAKE16-NEXT:    buffer_load_u16 v1, off, s[0:3], 0 glc dlc
-; GFX11-FAKE16-NEXT:    s_waitcnt vmcnt(0)
-; GFX11-FAKE16-NEXT:    v_add_f16_e32 v0, v0, v1
-; GFX11-FAKE16-NEXT:    buffer_store_b16 v0, off, s[8:11], 0
-; GFX11-FAKE16-NEXT:    s_nop 0
-; GFX11-FAKE16-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
-; GFX11-FAKE16-NEXT:    s_endpgm
     ptr addrspace(1) %r,
     ptr addrspace(1) %a,
     ptr addrspace(1) %b) {
@@ -157,6 +230,80 @@ define amdgpu_kernel void @fadd_f16_imm_a(
 ; VI-NEXT:    buffer_store_short v0, off, s[4:7], 0
 ; VI-NEXT:    s_endpgm
 ;
+; GFX11-SDAG-LABEL: fadd_f16_imm_a:
+; GFX11-SDAG:       ; %bb.0: ; %entry
+; GFX11-SDAG-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-SDAG-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-SDAG-NEXT:    s_mov_b32 s6, -1
+; GFX11-SDAG-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-SDAG-NEXT:    s_mov_b32 s4, s0
+; GFX11-SDAG-NEXT:    s_mov_b32 s5, s1
+; GFX11-SDAG-NEXT:    s_mov_b32 s0, s2
+; GFX11-SDAG-NEXT:    s_mov_b32 s1, s3
+; GFX11-SDAG-NEXT:    s_mov_b32 s2, s6
+; GFX11-SDAG-NEXT:    s_mov_b32 s3, s7
+; GFX11-SDAG-NEXT:    buffer_load_u16 v0, off, s[0:3], 0
+; GFX11-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-SDAG-NEXT:    v_mov_b16_e32 v0.h, 0x3c00
+; GFX11-SDAG-NEXT:    s_delay_alu instid0(VALU_DEP_1)
+; GFX11-SDAG-NEXT:    v_add_f16_e32 v0.l, v0.l, v0.h
+; GFX11-SDAG-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
+; GFX11-SDAG-NEXT:    s_nop 0
+; GFX11-SDAG-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-SDAG-NEXT:    s_endpgm
+;
+; GFX11-GISEL-LABEL: fadd_f16_imm_a:
+; GFX11-GISEL:       ; %bb.0: ; %entry
+; GFX11-GISEL-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-GISEL-NEXT:    s_mov_b32 s6, -1
+; GFX11-GISEL-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-GISEL-NEXT:    v_mov_b16_e32 v1.l, 0x3c00
+; GFX11-GISEL-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-GISEL-NEXT:    s_mov_b64 s[4:5], s[2:3]
+; GFX11-GISEL-NEXT:    s_mov_b64 s[2:3], s[6:7]
+; GFX11-GISEL-NEXT:    buffer_load_u16 v0, off, s[4:7], 0
+; GFX11-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-GISEL-NEXT:    v_add_f16_e32 v0.l, v0.l, v1.l
+; GFX11-GISEL-NEXT:    buffer_store_b16 v0, off, s[0:3], 0
+; GFX11-GISEL-NEXT:    s_nop 0
+; GFX11-GISEL-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-GISEL-NEXT:    s_endpgm
+;
+; GFX11-FAKE16-SDAG-LABEL: fadd_f16_imm_a:
+; GFX11-FAKE16-SDAG:       ; %bb.0: ; %entry
+; GFX11-FAKE16-SDAG-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s6, -1
+; GFX11-FAKE16-SDAG-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s4, s0
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s5, s1
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s0, s2
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s1, s3
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s2, s6
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s3, s7
+; GFX11-FAKE16-SDAG-NEXT:    buffer_load_u16 v0, off, s[0:3], 0
+; GFX11-FAKE16-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-SDAG-NEXT:    v_add_f16_e32 v0, 1.0, v0
+; GFX11-FAKE16-SDAG-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
+; GFX11-FAKE16-SDAG-NEXT:    s_nop 0
+; GFX11-FAKE16-SDAG-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-FAKE16-SDAG-NEXT:    s_endpgm
+;
+; GFX11-FAKE16-GISEL-LABEL: fadd_f16_imm_a:
+; GFX11-FAKE16-GISEL:       ; %bb.0: ; %entry
+; GFX11-FAKE16-GISEL-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b32 s6, -1
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-FAKE16-GISEL-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b64 s[4:5], s[2:3]
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b64 s[2:3], s[6:7]
+; GFX11-FAKE16-GISEL-NEXT:    buffer_load_u16 v0, off, s[4:7], 0
+; GFX11-FAKE16-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-GISEL-NEXT:    v_add_f16_e32 v0, 1.0, v0
+; GFX11-FAKE16-GISEL-NEXT:    buffer_store_b16 v0, off, s[0:3], 0
+; GFX11-FAKE16-GISEL-NEXT:    s_nop 0
+; GFX11-FAKE16-GISEL-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-FAKE16-GISEL-NEXT:    s_endpgm
 ; GFX11-LABEL: fadd_f16_imm_a:
 ; GFX11:       ; %bb.0: ; %entry
 ; GFX11-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
@@ -178,26 +325,6 @@ define amdgpu_kernel void @fadd_f16_imm_a(
 ; GFX11-NEXT:    s_nop 0
 ; GFX11-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
 ; GFX11-NEXT:    s_endpgm
-;
-; GFX11-FAKE16-LABEL: fadd_f16_imm_a:
-; GFX11-FAKE16:       ; %bb.0: ; %entry
-; GFX11-FAKE16-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
-; GFX11-FAKE16-NEXT:    s_mov_b32 s7, 0x31016000
-; GFX11-FAKE16-NEXT:    s_mov_b32 s6, -1
-; GFX11-FAKE16-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX11-FAKE16-NEXT:    s_mov_b32 s4, s0
-; GFX11-FAKE16-NEXT:    s_mov_b32 s5, s1
-; GFX11-FAKE16-NEXT:    s_mov_b32 s0, s2
-; GFX11-FAKE16-NEXT:    s_mov_b32 s1, s3
-; GFX11-FAKE16-NEXT:    s_mov_b32 s2, s6
-; GFX11-FAKE16-NEXT:    s_mov_b32 s3, s7
-; GFX11-FAKE16-NEXT:    buffer_load_u16 v0, off, s[0:3], 0
-; GFX11-FAKE16-NEXT:    s_waitcnt vmcnt(0)
-; GFX11-FAKE16-NEXT:    v_add_f16_e32 v0, 1.0, v0
-; GFX11-FAKE16-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
-; GFX11-FAKE16-NEXT:    s_nop 0
-; GFX11-FAKE16-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
-; GFX11-FAKE16-NEXT:    s_endpgm
     ptr addrspace(1) %r,
     ptr addrspace(1) %b) {
 entry:
@@ -246,6 +373,80 @@ define amdgpu_kernel void @fadd_f16_imm_b(
 ; VI-NEXT:    buffer_store_short v0, off, s[4:7], 0
 ; VI-NEXT:    s_endpgm
 ;
+; GFX11-SDAG-LABEL: fadd_f16_imm_b:
+; GFX11-SDAG:       ; %bb.0: ; %entry
+; GFX11-SDAG-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-SDAG-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-SDAG-NEXT:    s_mov_b32 s6, -1
+; GFX11-SDAG-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-SDAG-NEXT:    s_mov_b32 s4, s0
+; GFX11-SDAG-NEXT:    s_mov_b32 s5, s1
+; GFX11-SDAG-NEXT:    s_mov_b32 s0, s2
+; GFX11-SDAG-NEXT:    s_mov_b32 s1, s3
+; GFX11-SDAG-NEXT:    s_mov_b32 s2, s6
+; GFX11-SDAG-NEXT:    s_mov_b32 s3, s7
+; GFX11-SDAG-NEXT:    buffer_load_u16 v0, off, s[0:3], 0
+; GFX11-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-SDAG-NEXT:    v_mov_b16_e32 v0.h, 0x4000
+; GFX11-SDAG-NEXT:    s_delay_alu instid0(VALU_DEP_1)
+; GFX11-SDAG-NEXT:    v_add_f16_e32 v0.l, v0.l, v0.h
+; GFX11-SDAG-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
+; GFX11-SDAG-NEXT:    s_nop 0
+; GFX11-SDAG-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-SDAG-NEXT:    s_endpgm
+;
+; GFX11-GISEL-LABEL: fadd_f16_imm_b:
+; GFX11-GISEL:       ; %bb.0: ; %entry
+; GFX11-GISEL-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-GISEL-NEXT:    s_mov_b32 s6, -1
+; GFX11-GISEL-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-GISEL-NEXT:    v_mov_b16_e32 v1.l, 0x4000
+; GFX11-GISEL-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-GISEL-NEXT:    s_mov_b64 s[4:5], s[2:3]
+; GFX11-GISEL-NEXT:    s_mov_b64 s[2:3], s[6:7]
+; GFX11-GISEL-NEXT:    buffer_load_u16 v0, off, s[4:7], 0
+; GFX11-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-GISEL-NEXT:    v_add_f16_e32 v0.l, v0.l, v1.l
+; GFX11-GISEL-NEXT:    buffer_store_b16 v0, off, s[0:3], 0
+; GFX11-GISEL-NEXT:    s_nop 0
+; GFX11-GISEL-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-GISEL-NEXT:    s_endpgm
+;
+; GFX11-FAKE16-SDAG-LABEL: fadd_f16_imm_b:
+; GFX11-FAKE16-SDAG:       ; %bb.0: ; %entry
+; GFX11-FAKE16-SDAG-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s6, -1
+; GFX11-FAKE16-SDAG-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s4, s0
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s5, s1
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s0, s2
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s1, s3
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s2, s6
+; GFX11-FAKE16-SDAG-NEXT:    s_mov_b32 s3, s7
+; GFX11-FAKE16-SDAG-NEXT:    buffer_load_u16 v0, off, s[0:3], 0
+; GFX11-FAKE16-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-SDAG-NEXT:    v_add_f16_e32 v0, 2.0, v0
+; GFX11-FAKE16-SDAG-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
+; GFX11-FAKE16-SDAG-NEXT:    s_nop 0
+; GFX11-FAKE16-SDAG-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-FAKE16-SDAG-NEXT:    s_endpgm
+;
+; GFX11-FAKE16-GISEL-LABEL: fadd_f16_imm_b:
+; GFX11-FAKE16-GISEL:       ; %bb.0: ; %entry
+; GFX11-FAKE16-GISEL-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b32 s6, -1
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-FAKE16-GISEL-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b64 s[4:5], s[2:3]
+; GFX11-FAKE16-GISEL-NEXT:    s_mov_b64 s[2:3], s[6:7]
+; GFX11-FAKE16-GISEL-NEXT:    buffer_load_u16 v0, off, s[4:7], 0
+; GFX11-FAKE16-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-FAKE16-GISEL-NEXT:    v_add_f16_e32 v0, 2.0, v0
+; GFX11-FAKE16-GISEL-NEXT:    buffer_store_b16 v0, off, s[0:3], 0
+; GFX11-FAKE16-GISEL-NEXT:    s_nop 0
+; GFX11-FAKE16-GISEL-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-FAKE16-GISEL-NEXT:    s_endpgm
 ; GFX11-LABEL: fadd_f16_imm_b:
 ; GFX11:       ; %bb.0: ; %entry
 ; GFX11-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
@@ -267,26 +468,6 @@ define amdgpu_kernel void @fadd_f16_imm_b(
 ; GFX11-NEXT:    s_nop 0
 ; GFX11-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
 ; GFX11-NEXT:    s_endpgm
-;
-; GFX11-FAKE16-LABEL: fadd_f16_imm_b:
-; GFX11-FAKE16:       ; %bb.0: ; %entry
-; GFX11-FAKE16-NEXT:    s_load_b128 s[0:3], s[0:1], 0x24
-; GFX11-FAKE16-NEXT:    s_mov_b32 s7, 0x31016000
-; GFX11-FAKE16-NEXT:    s_mov_b32 s6, -1
-; GFX11-FAKE16-NEXT:    s_waitcnt lgkmcnt(0)
-; GFX11-FAKE16-NEXT:    s_mov_b32 s4, s0
-; GFX11-FAKE16-NEXT:    s_mov_b32 s5, s1
-; GFX11-FAKE16-NEXT:    s_mov_b32 s0, s2
-; GFX11-FAKE16-NEXT:    s_mov_b32 s1, s3
-; GFX11-FAKE16-NEXT:    s_mov_b32 s2, s6
-; GFX11-FAKE16-NEXT:    s_mov_b32 s3, s7
-; GFX11-FAKE16-NEXT:    buffer_load_u16 v0, off, s[0:3], 0
-; GFX11-FAKE16-NEXT:    s_waitcnt vmcnt(0)
-; GFX11-FAKE16-NEXT:    v_add_f16_e32 v0, 2.0, v0
-; GFX11-FAKE16-NEXT:    buffer_store_b16 v0, off, s[4:7], 0
-; GFX11-FAKE16-NEXT:    s_nop 0
-; GFX11-FAKE16-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
-; GFX11-FAKE16-NEXT:    s_endpgm
     ptr addrspace(1) %r,
     ptr addrspace(1) %a) {
 entry:
@@ -356,6 +537,85 @@ define amdgpu_kernel void @fadd_v2f16(
 ; VI-NEXT:    buffer_store_dword v0, off, s[0:3], 0
 ; VI-NEXT:    s_endpgm
 ;
+; GFX11-SDAG-LABEL: fadd_v2f16:
+; GFX11-SDAG:       ; %bb.0: ; %entry
+; GFX11-SDAG-NEXT:    s_clause 0x1
+; GFX11-SDAG-NEXT:    s_load_b128 s[4:7], s[0:1], 0x24
+; GFX11-SDAG-NEXT:    s_load_b64 s[8:9], s[0:1], 0x34
+; GFX11-SDAG-NEXT:    v_lshlrev_b32_e32 v0, 2, v0
+; GFX11-SDAG-NEXT:    s_mov_b32 s3, 0x31016000
+; GFX11-SDAG-NEXT:    s_mov_b32 s2, -1
+; GFX11-SDAG-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-SDAG-NEXT:    s_clause 0x1
+; GFX11-SDAG-NEXT:    global_load_b32 v1, v0, s[6:7]
+; GFX11-SDAG-NEXT:    global_load_b32 v0, v0, s[8:9]
+; GFX11-SDAG-NEXT:    s_mov_b32 s0, s4
+; GFX11-SDAG-NEXT:    s_mov_b32 s1, s5
+; GFX11-SDAG-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-SDAG-NEXT:    v_pk_add_f16 v0, v1, v0
+; GFX11-SDAG-NEXT:    buffer_store_b32 v0, off, s[0:3], 0
+; GFX11-SDAG-NEXT:    s_nop 0
+; GFX11-SDAG-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-SDAG-NEXT:    s_endpgm
+;
+; GFX11-GISEL-LABEL: fadd_v2f16:
+; GFX11-GISEL:       ; %bb.0: ; %entry
+; GFX11-GISEL-NEXT:    s_clause 0x1
+; GFX11-GISEL-NEXT:    s_load_b128 s[4:7], s[0:1], 0x24
+; GFX11-GISEL-NEXT:    s_load_b64 s[0:1], s[0:1], 0x34
+; GFX11-GISEL-NEXT:    v_lshlrev_b32_e32 v0, 2, v0
+; GFX11-GISEL-NEXT:    s_waitcnt lgkmcnt(0)
+; GFX11-GISEL-NEXT:    s_clause 0x1
+; GFX11-GISEL-NEXT:    global_load_b32 v1, v0, s[6:7]
+; GFX11-GISEL-NEXT:    global_load_b32 v0, v0, s[0:1]
+; GFX11-GISEL-NEXT:    s_mov_b32 s6, -1
+; GFX11-GISEL-NEXT:    s_mov_b32 s7, 0x31016000
+; GFX11-GISEL-NEXT:    s_waitcnt vmcnt(0)
+; GFX11-GISEL-NEXT:    v_pk_add_f16 v0, v1, v0
+; GFX11-GISEL-NEXT:    buffer_store_b32 v0, off, s[4:7], 0
+; GFX11-GISEL-NEXT:    s_nop 0
+; GFX11-GISEL-NEXT:    s_sendmsg sendmsg(MSG_DEALLOC_VGPRS)
+; GFX11-GISEL-NEXT:    s_endpgm
+;
+; GFX11-FAKE16-SDAG-LABEL: ...
[truncated]

Sisyph
Sisyph approved these changes Dec 1, 2023
View reviewed changes
Copy link
Contributor

@Sisyph Sisyph left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

@kosarev kosarev merged commit 2b7191c into llvm:main Dec 4, 2023
@kosarev kosarev deleted the t16-vop2 branch December 4, 2023 14:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Sisyph Sisyph Sisyph approved these changes

@jayfoad jayfoad Awaiting requested review from jayfoad

@arsenm arsenm Awaiting requested review from arsenm

@rampitec rampitec Awaiting requested review from rampitec

Assignees
No one assigned
Labels
backend:AMDGPU
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kosarev @llvmbot @Sisyph