Speedstu/CUDA-for-AMD-Windows

A reproducible Windows CUDA compatibility stack for AMD GPUs, built around ZLUDA + AMD ROCm/HIP.

Run CUDA-targeted Windows applications on AMD hardware, verify that work really reaches the GPU, and distinguish a correct result from a crash, timeout, CPU fallback, or silently wrong tensor.

Install a current AMD GPU driver and the AMD HIP SDK for Windows with HIP Libraries.

The required HIP version depends on the GPU architecture. The project checks the architecture-specific floor recorded in manifests/windows-gpu-profiles.json.

git clone https://github.com/Speedstu/CUDA-for-AMD-Windows.git
cd CUDA-for-AMD-Windows
powershell -ExecutionPolicy Bypass -File .\scripts\install.ps1

The installer detects the GPU, verifies the HIP environment, downloads pinned assets, stages the runtime, runs a runtime smoke test, and performs numerical checks when a compatible Python/PyTorch environment is available.

.\scripts\run-zluda.ps1 -Program C:\path\to\app.exe

.\scripts\doctor.ps1
.\scripts\test-runtime.ps1
.\scripts\test-functional.ps1 -PythonExe C:\path\to\venv\Scripts\python.exe

The original low-level setup path is intentionally kept compatible for existing users and old posts:

.\scripts\setup.ps1 -DownloadZluda -DownloadLibTorch

See scripts/README.md for the stable user-facing commands and maintainer tooling.

Launch CUDA-facing PyTorch through scripts/run-zluda.ps1. On both ZLUDA v6 stable and v7 latest, it now defaults to math-only SDPA (flash=False, mem_efficient=False, math=True). This avoids the finite-but-wrong fused-attention output reported on gfx1100/gfx1150. It is a safe workaround, not proof that the fused kernels work. Use -AllowUnsafeFusedSDPA only for isolated diagnostics.

test-functional.ps1 deliberately probes raw kernels separately and may still report them as incorrect. Running zluda.exe directly bypasses the Python safety hook.

The current prebuilt zluda.exe also splits program arguments containing spaces. A v7 source patch candidate is included for this (#6), built and tested on gfx1200. Upstream prebuilt launchers still need rebuilding with the patch before the fix takes effect.

The launcher argv patch was also independently rebuilt and checked on Radeon 890M / gfx1150 (three passing repeats for the argument round-trip and a multi-word llama.cpp prompt). The patch still needs to be applied to distributed binaries; the separate zluda-launcher-candidate-build workflow creates a candidate-only build, not an automatic replacement for stable/latest.

The conservative llama.cpp smoke now limits its default context to 4096 tokens via -c, rather than inheriting a potentially huge model maximum. Pass -ContextSize to override for intentional larger tests. The other safety flags are unchanged.

A green build is not a hardware correctness certificate. Before considering a ZLUDA driver candidate for general use, maintainers must verify the candidate DLL SHA-256 and run the CPU-vs-GPU numerical tests for sort, cumsum, matmul and math-only SDPA. The fail-closed, read-only promotion gate rejects missing, stale or mismatched test evidence. Compilation-only artifacts remain experimental, and the known-good runtime is not replaced automatically.

Full details, version floors, evidence levels, known limitations, and the CUDA DLL → AMD backend map: docs/COMPATIBILITY.md.

The public reference path uses pinned upstream components and has been tested with:

• ZLUDA v6-preview.69
• AMD HIP SDK 6.4
• LibTorch 2.3.0 + cu118
• Radeon RX 9060 XT / gfx1200
• CUDA-facing driver loading plus cuBLAS, cuBLASLt, cuSPARSE and cuFFT smoke coverage
• numerical CPU-vs-GPU correctness probes
• a real 2,216,347-parameter PPO workload
• a clean validation iteration of 65,536 timesteps

See docs/VALIDATION.md.

The repository also maintains a pinned, source-based ZLUDA v7-preview.10 patch series for newer CUDA-facing behavior.

Validated work on the reference machine includes areas such as:

• CUDA Graph compatibility
• driver metadata and launch probes
• cuFFT and cuSPARSE paths
• NVML compatibility
• experimental cuSOLVER → hipSOLVER bridging
• experimental cuDNN v8 forward/backward convolution → MIOpen bridging on the reference gfx1200 system
• fail-closed handling for unsafe fused SDPA fallbacks
• modern llama.cpp registration and GPU execution

The patch set remains separate from the stable installer so experimental work cannot silently redefine the stable path.

See patches/zluda-v7-preview10/README.md.

This project deliberately tests more than tiny API calls.

The reference workload performs rollout, forward, backward, PPO learning, and optimizer work through the CUDA-facing device.

See docs/VALIDATION.md and docs/BENCHMARKS.md.

A recent llama.cpp b10978 build has completed an end-to-end GPU smoke on the RX 9060 XT experimental v7 path:

• AMD GPU exposed as CUDA0 through ZLUDA
• 6/6 model layers offloaded
• KV cache on GPU
• compute buffer on GPU
• CUDA Graph warmup reached
• prompt processing and generation completed
• clean process exit, reproduced more than once

This is evidence for that tested path, not a claim that every llama.cpp kernel or model is supported.

See docs/LLAMA_CPP.md.

Performance claims are kept same-GPU and reproducible where possible; this is not an AMD-vs-NVIDIA benchmark.

On the experimental v7/TheRock path, a clean four-pair FP32 SGEMM validation measured paired median overhead versus direct HIP/rocBLAS of +1.79% at 1024², +0.45% at 2048², and +2.20% at 4096². The corresponding throughput ratios were 98.2%, 99.6%, and 97.8%.

A real VelocityRL 512 agents × rollout 16 smoke separated first-use compilation from steady state: the warmed updates reached 72,306 SPS and 70,333 SPS, for 71,319.5 SPS median steady-state.

Methodology and raw results: docs/BENCHMARKS.md and benchmarks/.

A CUDA API returning success is not enough to call something supported.

The test suite distinguishes:

PASS · UNSUPPORTED · INCORRECT · TIMEOUT · ERROR · process failure

The project follows a fail-closed rule: an explicit unsupported result is better than a plausible-looking but incorrect tensor.

That policy already caught fused SDPA fallback paths that could return numerically wrong output instead of failing clearly.

For the evidence model and contribution rules, see CONTRIBUTING.md.

Generated runtime files stay under .runtime/ and are ignored by Git.

Start with docs/README.md instead of searching through the repository.

• Compatibility and support levels
• Validation methodology and evidence
• Architecture
• Benchmarks
• llama.cpp validation
• RX 9070 XT external validation
• Troubleshooting
• Experimental v7 patch set
• Experimental cuSOLVER proxy
• Experimental cuDNN v8 → MIOpen convolution bridge

Historical reconstruction material is still preserved, but it is no longer part of the recommended path.

Both successful and failed reports are useful.

.\scripts\gpu-scan.ps1 -OutputPath .\gpu-report.json
.\scripts\test-runtime.ps1
.\scripts\test-functional.ps1 -PythonExe C:\path\to\venv\Scripts\python.exe

Then open a GPU compatibility report.

Please include the GPU model, gfxXXXX target, driver, HIP version, ZLUDA channel/build, exact workload, and proof that the AMD/ZLUDA device actually executed the work rather than falling back to CPU.

• ZLUDA does not implement the entire CUDA ecosystem.
• Passing cuda_check or device detection alone does not establish numerical correctness.
• Windows exposes only part of the full ROCm ecosystem.
• The validated stable Windows HIP path does not provide a complete cuDNN/MIOpen equivalent stack. A separate experimental v7 bridge validates a narrow cuDNN v8 2D forward + backward-data + backward-filter subset on gfx1200; see docs/CUDNN_BRIDGE.md.
• NCCL, TensorRT, unsupported PTX behavior, custom CUDA extensions and architecture-specific kernels may fail.
• Fused Flash/memory-efficient SDPA paths can depend on NVIDIA cubins; unsafe fallbacks are treated as unsupported rather than accepted as correct.
• ZLUDA_CC=8.6 is a CUDA-facing compatibility value, not the native AMD GPU architecture.

More detail: docs/COMPATIBILITY.md.

The project is evidence-driven. New compatibility claims should include a focused reproducer, numerical validation where applicable, exact versions/hashes, and a real workload when possible.

Read CONTRIBUTING.md before changing support claims or promoting experimental patches.

Project-owned scripts and documentation are MIT licensed.

ZLUDA, AMD ROCm/HIP, NVIDIA CUDA components, PyTorch/LibTorch and other third-party projects retain their own licenses. See THIRD_PARTY_NOTICES.md. llama.cpp / gfx1150 issue #6: a new pinned safe-build candidate avoids the missing F32 mul_mat_f specialization through existing MMVF/cuBLAS fallback. It leaves ZLUDA_CC=8.6 and F16/BF16 MMF unchanged; the Radeon 890M community tester confirmed 3/3 runs for all six formerly failing models; native-answer comparison was checked on the 4B model only. See F32 MMF candidate. This does not address speculative top-k or Nemotron copy faults.