[WIP][IR][Constants] Change the semantic of ConstantPointerNull to represent an actual nullptr instead of a zero-value pointer

The value of a `nullptr` is not always `0`. For example, on AMDGPU, the `nullptr` in address spaces 3 and 5 is `0xffffffff`. Currently, there is no target-independent way to get this information, making it difficult and error-prone to handle null pointers in target-agnostic code.

We do have `ConstantPointerNull`, but it might be a little confusing and misleading. It represents a pointer with an all-zero value rather than necessarily a real `nullptr`. Therefore, to represent a real `nullptr` in address space `N`, we need to use `addrspacecast ptr null to ptr addrspace(N)` and it can't be folded.

In this PR, we change the semantic of `ConstantPointerNull` to represent an actual `nullptr` instead of a zero-value pointer. Here is the detailed changes.

* `ptr addrspace(N) null` will represent the actual `nullptr` in address space `N`.
* `ptr addrspace(N) zeroinitializer` will represent a zero-value pointer in address space `N`.

* `Constant::getNullValue` will return a _null_ value. It is same as the current semantics except for the `PointerType`, which will return a real `nullptr` pointer.
* `Constant::getZeroValue` will return a zero value constant. It is completely same as the current semantics. To represent a zero-value pointer, a `ConstantExpr` will be used (effectively `inttoptr i8 0 to ptr addrspace(N)`).
* Correspondingly, there will be both `Constant::isNullValue` and `Constant::isZeroValue`.

The RFC is https://discourse.llvm.org/t/rfc-introduce-sentinel-pointer-value-to-datalayout/85265. It is a little bit old and the title might look different, but everything eventually converges to this change. An early attempt can be found in #131557, which has many valuable discussion as well.

This PR is still WIP but any early feedback is welcome. I'll include as many necessary code changes as possible in this PR, but eventually this needs to be carefully split into multiple PRs, and I'll do it after the changes look good to every one.

Author: shiltian

clang/lib/Basic/Targets/AMDGPU.cpp

@@ -31,7 +31,7 @@ static const char *const DataLayoutStringR600 =
     "-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1";
 
 static const char *const DataLayoutStringAMDGCN =
-    "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32"
+    "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32"
     "-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-"
     "v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-"
     "v2048:2048-n32:64-S32-A5-G1-ni:7:8:9";

clang/test/CodeGen/target-data.c

@@ -160,12 +160,12 @@
 
 // RUN: %clang_cc1 -triple amdgcn-unknown -target-cpu hawaii -o - -emit-llvm %s \
 // RUN: | FileCheck %s -check-prefix=R600SI
-// R600SI: target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9"
+// R600SI: target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9"
 
 // Test default -target-cpu
 // RUN: %clang_cc1 -triple amdgcn-unknown -o - -emit-llvm %s \
 // RUN: | FileCheck %s -check-prefix=R600SIDefault
-// R600SIDefault: target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9"
+// R600SIDefault: target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9"
 
 // RUN: %clang_cc1 -triple arm64-unknown -o - -emit-llvm %s | \
 // RUN: FileCheck %s -check-prefix=AARCH64

clang/test/CodeGenOpenCL/amdgpu-env-amdgcn.cl

@@ -1,5 +1,5 @@
 // RUN: %clang_cc1 %s -O0 -triple amdgcn -emit-llvm -o - | FileCheck %s
 // RUN: %clang_cc1 %s -O0 -triple amdgcn---opencl -emit-llvm -o - | FileCheck %s
 
-// CHECK: target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9"
+// CHECK: target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32-p7:160:256:256:32-p8:128:128:128:48-p9:192:256:256:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5-G1-ni:7:8:9"
 void foo(void) {}

lld/test/ELF/lto/amdgcn-oses.ll

@@ -25,7 +25,7 @@
 
 ;--- amdhsa.ll
 target triple = "amdgcn-amd-amdhsa"
-target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
+target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
 
 !llvm.module.flags = !{!0}
 !0 = !{i32 1, !"amdhsa_code_object_version", i32 500}
@@ -36,15 +36,15 @@ define void @_start() {
 
 ;--- amdpal.ll
 target triple = "amdgcn-amd-amdpal"
-target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
+target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
 
 define amdgpu_cs void @_start() {
   ret void
 }
 
 ;--- mesa3d.ll
 target triple = "amdgcn-amd-mesa3d"
-target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
+target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
 
 define void @_start() {
   ret void

lld/test/ELF/lto/amdgcn.ll

@@ -5,7 +5,7 @@
 ; Make sure the amdgcn triple is handled
 
 target triple = "amdgcn-amd-amdhsa"
-target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-p3:32:32-p4:64:64-p5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
+target datalayout = "e-m:e-p:64:64-p1:64:64-p2:32:32-po3:32:32-p4:64:64-po5:32:32-p6:32:32-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-v2048:2048-n32:64-S32-A5"
 
 define void @_start() {
   ret void

llvm/docs/LangRef.rst

@@ -3301,8 +3301,12 @@ as follows:
     default address space 0. The value of ``<as>`` must be in the range [1,2^24).
     The optional ``<flags>`` are used to specify properties of pointers in this
     address space: the character ``u`` marks pointers as having an unstable
-    representation, and ``e`` marks pointers having external state. See
-    :ref:`Non-Integral Pointer Types <nointptrtype>`.
+    representation; ``e`` marks pointers having external state; ``z`` marks the
+    value of the nullptr as all-zeros (default behavior if it is not specified);
+    ``o`` marks the value of the nullptr as all-ones; ``c`` marks the value of
+    the nullptr as custom (neither all-zeros nor all-ones), such that LLVM will
+    not be able to fold various casts involving nullptr.
+    See :ref:`Non-Integral Pointer Types <nointptrtype>`.
 
 ``i<size>:<abi>[:<pref>]``
     This specifies the alignment for an integer type of a given bit

llvm/include/llvm/IR/Constant.h

@@ -189,6 +189,8 @@ class Constant : public User {
 
   LLVM_ABI static Constant *getNullValue(Type *Ty);
 
+  LLVM_ABI static Constant *getZeroValue(Type *Ty);
+
   /// @returns the value for an integer or vector of integer constant of the
   /// given type that has all its bits set to true.
   /// Get the all ones value

llvm/include/llvm/IR/DataLayout.h

@@ -83,6 +83,11 @@ class DataLayout {
     /// for additional metadata (e.g. AMDGPU buffer fat pointers with bounds
     /// and other flags or CHERI capabilities that contain bounds+permissions).
     uint32_t IndexBitWidth;
+    /// The value of the nullptr in this address space. It can be three values:
+    /// all-zeros, all-ones, or std::nullopt. Since we don't have a way to
+    /// represent an arbitrary bit pattern, we use std::nullopt to represent the
+    /// case where the nullptr value is neither 0 nor -1.
+    std::optional<APInt> NullPtrValue;
     /// Pointers in this address space don't have a well-defined bitwise
     /// representation (e.g. they may be relocated by a copying garbage
     /// collector and thus have different addresses at different times).
@@ -158,7 +163,8 @@ class DataLayout {
   /// Sets or updates the specification for pointer in the given address space.
   void setPointerSpec(uint32_t AddrSpace, uint32_t BitWidth, Align ABIAlign,
                       Align PrefAlign, uint32_t IndexBitWidth,
-                      bool HasUnstableRepr, bool HasExternalState);
+                      std::optional<APInt> NullPtrValue, bool HasUnstableRepr,
+                      bool HasExternalState);
 
   /// Internal helper to get alignment for integer of given bitwidth.
   LLVM_ABI Align getIntegerAlignment(uint32_t BitWidth, bool abi_or_pref) const;
@@ -697,6 +703,11 @@ class DataLayout {
   ///
   /// This includes an explicitly requested alignment (if the global has one).
   LLVM_ABI Align getPreferredAlign(const GlobalVariable *GV) const;
+
+  /// Returns the value of the nullptr in the given address space.
+  LLVM_ABI std::optional<APInt> getNullPtrValue(unsigned AddrSpace) const {
+    return getPointerSpec(AddrSpace).NullPtrValue;
+  }
 };
 
 inline DataLayout *unwrap(LLVMTargetDataRef P) {

llvm/lib/Analysis/ConstantFolding.cpp

@@ -1497,6 +1497,19 @@ Constant *llvm::ConstantFoldCastOperand(unsigned Opcode, Constant *C,
     llvm_unreachable("Missing case");
   case Instruction::PtrToAddr:
   case Instruction::PtrToInt:
+    // If the input is a nullptr, we can fold it to the corresponding nullptr
+    // value.
+    if (Opcode == Instruction::PtrToInt && C->isNullValue()) {
+      if (std::optional<APInt> NullPtrValue = DL.getNullPtrValue(
+              C->getType()->getScalarType()->getPointerAddressSpace())) {
+        if (NullPtrValue->isZero())
+          return Constant::getZeroValue(DestTy);
+        else if (NullPtrValue->isAllOnes())
+          return Constant::getAllOnesValue(DestTy);
+        else
+          llvm_unreachable("invalid nullptr value");
+      }
+    }
     if (auto *CE = dyn_cast<ConstantExpr>(C)) {
       Constant *FoldedValue = nullptr;
       // If the input is an inttoptr, eliminate the pair.  This requires knowing
@@ -1543,6 +1556,13 @@ Constant *llvm::ConstantFoldCastOperand(unsigned Opcode, Constant *C,
     }
     break;
   case Instruction::IntToPtr:
+    // We can fold it to a null pointer if the input is the nullptr value.
+    if (std::optional<APInt> NullPtrValue = DL.getNullPtrValue(
+            DestTy->getScalarType()->getPointerAddressSpace())) {
+      if ((NullPtrValue->isZero() && C->isZeroValue()) ||
+          (NullPtrValue->isAllOnes() && C->isAllOnesValue()))
+        return Constant::getNullValue(DestTy);
+    }
     // If the input is a ptrtoint, turn the pair into a ptr to ptr bitcast if
     // the int size is >= the ptr size and the address spaces are the same.
     // This requires knowing the width of a pointer, so it can't be done in
@@ -1561,6 +1581,24 @@ Constant *llvm::ConstantFoldCastOperand(unsigned Opcode, Constant *C,
       }
     }
     break;
+  case Instruction::AddrSpaceCast:
+    // A null pointer (`ptr addrspace(N) null` in IR presentation,
+    // `ConstantPointerNull` in LLVM class, not `nullptr` in C/C++) used to
+    // represent a zero-value pointer in the corresponding address space.
+    // Therefore, we can't simply fold an address space cast of a null pointer
+    // from one address space to another, because on some targets, the nullptr
+    // of an address space could be non-zero.
+    //
+    // Recently, the semantic of `ptr addrspace(N) null` is changed to represent
+    // the actual nullptr in the corresponding address space. It can be zero or
+    // non-zero, depending on the target. Therefore, we can fold an address
+    // space cast of a nullptr from one address space to another.
+
+    // If the input is a nullptr, we can fold it to the corresponding
+    // nullptr in the destination address space.
+    if (C->isNullValue())
+      return Constant::getNullValue(DestTy);
+    [[fallthrough]];
   case Instruction::Trunc:
   case Instruction::ZExt:
   case Instruction::SExt:
@@ -1570,7 +1608,6 @@ Constant *llvm::ConstantFoldCastOperand(unsigned Opcode, Constant *C,
   case Instruction::SIToFP:
   case Instruction::FPToUI:
   case Instruction::FPToSI:
-  case Instruction::AddrSpaceCast:
     break;
   case Instruction::BitCast:
     return FoldBitCast(C, DestTy, DL);

llvm/lib/AsmParser/LLParser.cpp

@@ -6575,7 +6575,7 @@ bool LLParser::convertValIDToValue(Type *Ty, ValID &ID, Value *&V,
     if (auto *TETy = dyn_cast<TargetExtType>(Ty))
       if (!TETy->hasProperty(TargetExtType::HasZeroInit))
         return error(ID.Loc, "invalid type for null constant");
-    V = Constant::getNullValue(Ty);
+    V = Constant::getZeroValue(Ty);
     return false;
   case ValID::t_None:
     if (!Ty->isTokenTy())