# mi300a-xnack

Test HSA\_XNACK behavior on AMD MI300A

XNACK refers to the GPU's ability to retry memory accesses that failed due a page fault (which normally would lead to a memory access error), and instead retrieve the missing page [(source)](https://rocm.docs.amd.com/en/docs-6.2.1/conceptual/gpu-memory.html#xnack)

## Quick Start

```c++
./run.sh
```

## ROCm 6.2.1 Results

### Device Name

| `HSA_XNACK=0` (or unset) | `HSA_XNACK=1` |
|-|-|
| `amdgcn-amd-amdhsa--gfx942:sramecc+:xnack-` | `amdgcn-amd-amdhsa--gfx942:sramecc+:xnack+` |

* `xnack+`: XNACK is available and enabled
* `xnack-`: XNACK is available and disabled [(source)](https://rocm.docs.amd.com/en/docs-6.2.1/conceptual/gpu-memory.html#xnack)


### Compiled with `--offload-arch=gfx942` or no `--offload-arch` provided

*GPU kernels will run regardless of whether XNACK is enabled*

| Allocator | Access | `HSA_XNACK=0` (or unset) | `HSA_XNACK=1` |
|-|-|-|-|
| hipMalloc | GPU Kernel | yes | yes |
| hipMalloc | Host Loop | yes | yes |
| hipMalloc | hipMemset | yes | yes |
| malloc | GPU Kernel | **segfault** | **yes** |
| malloc | Host Loop | yes | yes |
| malloc | hipMemset | HIP runtime error | HIP runtime error |
| hipMallocManaged | GPU Kernel | yes | yes |
| hipMallocManaged | Host Loop | yes | yes |
| hipMallocManaged | hipMemset | yes | yes |

### Compiled with `--offload-arch=gfx942:xnack-`

*GPU kernels will run only if XNACK is disabled*

| Allocator | Access | `HSA_XNACK=0` (or unset) | `HSA_XNACK=1` |
|-|-|-|-|
| hipMalloc | GPU Kernel | yes | *N/A* |
| hipMalloc | Host Loop | yes | yes |
| hipMalloc | hipMemset | yes | yes |
| malloc | GPU Kernel | **segfault** | **N/A** |
| malloc | Host Loop | yes | yes |
| malloc | hipMemset | HIP runtime error | HIP runtime error |
| hipMallocManaged | GPU Kernel | yes | *N/A* |
| hipMallocManaged | Host Loop | yes | yes |
| hipMallocManaged | hipMemset | yes | yes |

### Compiled with `--offload-arch=gfx942:xnack+`

*GPU kernels will run only if XNACK is enabled*

| Allocator | Access | `HSA_XNACK=0` (or unset) | `HSA_XNACK=1` |
|-|-|-|-|
| hipMalloc | GPU Kernel | *N/A* | yes |
| hipMalloc | Host Loop | yes | yes |
| hipMalloc | hipMemset | yes | yes |
| malloc | GPU Kernel | *N/A* | **yes** |
| malloc | Host Loop | yes | yes |
| malloc | hipMemset | HIP runtime error | HIP runtime error |
| hipMallocManaged | GPU Kernel | *N/A* | yes |
| hipMallocManaged | Host Loop | yes | yes |
| hipMallocManaged | hipMemset | yes | yes |