WHAT IT IS
CUDA is the parallel computing platform and programming model introduced by NVIDIA in 2007 that lets developers use GPUs for general-purpose computing. It provides language extensions, a compiler, and acceleration libraries such as cuDNN and cuBLAS. It addresses efficiently mapping highly parallel computation such as neural-network training and inference onto GPUs.
Why it matters
It has been the practical programming interface for AI compute for over a decade; the libraries and tools built around it form NVIDIA’s deepest ecosystem moat.
Key specs
- First release
- 2007
- Positioning
- GPU general-purpose parallel computing platform and programming model
- Companion libraries
- cuDNN, cuBLAS and others
Related concepts
Training & Inference Infrastructure
Memory decides how large a model you can train, communication how long it takes — raw compute is rarely the bottleneck
Inference Optimization & Serving
Training happens once; inference happens a billion times a day — and serving is torn between fast first tokens and high throughput, which usually pull against each other