Can the Nvidia RTX A4000 ADA Handle Machine Learning Tasks?

In April, Nvidia launched a new product, the RTX A4000 ADA, a small form factor GPU designed for workstation applications. This processor replaces the A2000 and can be used for complex tasks, including scientific research, engineering calculations, and data visualization.

The RTX A4000 ADA features 6,144 CUDA cores, 192 Tensor and 48 RT cores, and 20GB GDDR6 ECC VRAM. One of the key benefits of the new GPU is its power efficiency: the RTX A4000 ADA consumes only 70W, which lowers both power costs and system heat. The GPU also allows you to drive multiple displays thanks to its 4x Mini-DisplayPort 1.4a connectivity.

When comparing the RTX 4000 SFF ADA GPUs to other devices in the same class, it should be noted that when running in single precision mode, it shows performance similar to the latest generation RTX A4000 GPU, which consumes twice as much power (140W vs. 70W).

The ADA RTX 4000 SFF is built on the ADA Lovelace architecture and 5nm process technology. This enables next-generation Tensor Core and ray tracing cores, which significantly improve performance by providing faster and more efficient ray tracing and Tensor cores than the RTX A4000. In addition, ADA's RTX 4000 SFF comes in a small package - the card is 168mm long and as thick as two expansion slots.

Improved ray tracing kernels allows for efficient performance in environments where the technology is used, such as in 3D design and rendering. Furthermore, the new GPU's 20GB memory capacity enables it to handle large environments.

According to the manufacturer, fourth-generation Tensor cores deliver high AI computational performance - a twofold increase in performance over the previous generation. The new Tensor cores support FP8 acceleration. This innovative feature may work well for those developing and deploying AI models in environments such as genomics and computer vision.

It's also of note that the increase in encoding and decoding mechanisms makes the RTX 4000 SFF ADA a good solution for multimedia workloads such as video among others.

Technical specifications of NVIDIA RTX A4000 and RTX A5000 graphics cards, RTX 3090

	RTX A4000 ADA	NVIDIA RTX A4000	NVIDIA RTX A5000	RTX 3090
Architecture	Ada Lovelace	Ampere	Ampere	Ampere
Tech Process	5 nm	8 nm	8 nm	8 nm
GPU	AD104	GA102	GA104	GA102
Number of transistors (millions)	35,800	17,400	28,300	28,300
Memory bandwidth (Gb/s)	280.0	448	768	936.2
Video memory capacity (bits)	160	256	384	384
GPU memory (GB)	20	16	24	24
Memory type	GDDR6	GDDR6	GDDR6	GDDR6X
CUDA cores	6,144	6 144	8192	10496
Tensor cores	192	192	256	328
RT cores	48	48	64	82
SP perf (teraflops)	19.2	19,2	27,8	35,6
RT core performance (teraflops)	44.3	37,4	54,2	69,5
Tensor performance (teraflops)	306.8	153,4	222,2	285
Maximum power (Watts)	70	140	230	350
Interface	PCIe 4.0 x 16	PCI-E 4.0 x16	PCI-E 4.0 x16	PCIe 4.0 x16
Connectors	4x Mini DisplayPort 1.4a	DP 1.4 (4)	DP 1.4 (4)	DP 1.4 (4)
Form Factor	2 slots	1 slot	2 slots	2-3 slots
The vGPU software	no	no	Yes, unlimited	Yes. with limitations
Nvlink	no	no	2x RTX A5000	yes
CUDA support	11.6	8.6	8.6	8.6
VULKAN support	1.3	yes	yes	yes, 1.2
Price (USD)	1,250	1000	2500	1400

Description of the test environment

	RTX A4000 ADA	RTX A4000
CPU	AMD Ryzen 9 5950X 3.4GHz (16 cores)	OctaCore Intel Xeon E-2288G, 3,5 GHz
RAM	4x 32 Gb DDR4 ECC SO-DIMM	2x 32 GB DDR4-3200 ECC DDR4 SDRAM 1600 MHz
Drive	1Tb NVMe SSD	Samsung SSD 980 PRO 1TB
Motherboard	ASRock X570D4I-2T	Asus P11C-I Series
Operating System	Microsoft Windows 10	Microsoft Windows 10

Test results

V-Ray 5 Benchmark

V-Ray GPU CUDA and RTX tests measure relative GPU rendering performance. The RTX A4000 GPU is slightly behind the RTX A4000 ADA (4% and 11%, respectively).

Machine Learning

"Dogs vs. Cats"

To compare the performance of GPUs for neural networks, we used the "Dogs vs. Cats" dataset - the test analyzes the content of a photo and distinguishes whether the photo shows a cat or a dog. All the necessary raw data can be found . We ran this test on different GPUs and cloud services and got the following results:

In this test, the RTX A4000 ADA slightly outperformed the RTX A4000 by 9%, but keep in mind the small size and low power consumption of the new GPU.

AI-Benchmark allows you to measure the performance of the device during an AI model output task. The unit of measurement may vary according to the test, but usually it is the number of operations per second (OPS) or the number of frames per second (FPS).

	RTX A4000	RTX A4000 ADA
1/19. MobileNet-V2	1.1 — inference | batch=50, size=224x224: 38.5 ± 2.4 ms1.2 — training | batch=50, size=224x224: 109 ± 4 ms	1.1 — inference | batch=50, size=224x224: 53.5 ± 0.7 ms1.2 — training | batch=50, size=224x224: 130.1 ± 0.6 ms
2/19. Inception-V3	2.1 — inference | batch=20, size=346x346: 36.1 ± 1.8 ms2.2 — training | batch=20, size=346x346: 137.4 ± 0.6 ms	2.1 — inference | batch=20, size=346x346: 36.8 ± 1.1 ms2.2 — training | batch=20, size=346x346: 147.5 ± 0.8 ms
3/19. Inception-V4	3.1 — inference | batch=10, size=346x346: 34.0 ± 0.9 ms3.2 — training | batch=10, size=346x346: 139.4 ± 1.0 ms	3.1 — inference | batch=10, size=346x346: 33.0 ± 0.8 ms3.2 — training | batch=10, size=346x346: 135.7 ± 0.9 ms
4/19. Inception-ResNet-V2	4.1 — inference | batch=10, size=346x346: 45.7 ± 0.6 ms4.2 — training | batch=8, size=346x346: 153.4 ± 0.8 ms	4.1 — inference batch=10, size=346x346: 33.6 ± 0.7 ms4.2 — training batch=8, size=346x346: 132 ± 1 ms
5/19. ResNet-V2-50	5.1 — inference | batch=10, size=346x346: 25.3 ± 0.5 ms5.2 — training | batch=10, size=346x346: 91.1 ± 0.8 ms	5.1 — inference | batch=10, size=346x346: 26.1 ± 0.5 ms5.2 — training | batch=10, size=346x346: 92.3 ± 0.6 ms
6/19. ResNet-V2-152	6.1 — inference | batch=10, size=256x256: 32.4 ± 0.5 ms6.2 — training | batch=10, size=256x256: 131.4 ± 0.7 ms	6.1 — inference | batch=10, size=256x256: 23.7 ± 0.6 ms6.2 — training | batch=10, size=256x256: 107.1 ± 0.9 ms
7/19. VGG-16	7.1 — inference | batch=20, size=224x224: 54.9 ± 0.9 ms7.2 — training | batch=2, size=224x224: 83.6 ± 0.7 ms	7.1 — inference | batch=20, size=224x224: 66.3 ± 0.9 ms7.2 — training | batch=2, size=224x224: 109.3 ± 0.8 ms
8/19. SRCNN 9-5-5	8.1 — inference | batch=10, size=512x512: 51.5 ± 0.9 ms8.2 — inference | batch=1, size=1536x1536: 45.7 ± 0.9 ms8.3 — training | batch=10, size=512x512: 183 ± 1 ms	8.1 — inference | batch=10, size=512x512: 59.9 ± 1.6 ms8.2 — inference | batch=1, size=1536x1536: 53.1 ± 0.7 ms8.3 — training | batch=10, size=512x512: 176 ± 2 ms
9/19. VGG-19 Super-Res	9.1 — inference | batch=10, size=256x256: 99.5 ± 0.8 ms9.2 — inference | batch=1, size=1024x1024: 162 ± 1 ms9.3 — training | batch=10, size=224x224: 204 ± 2 ms
10/19. ResNet-SRGAN	10.1 — inference | batch=10, size=512x512: 85.8 ± 0.6 ms10.2 — inference | batch=1, size=1536x1536: 82.4 ± 1.9 ms10.3 — training | batch=5, size=512x512: 133 ± 1 ms	10.1 — inference | batch=10, size=512x512: 98.9 ± 0.8 ms10.2 — inference | batch=1, size=1536x1536: 86.1 ± 0.6 ms10.3 — training | batch=5, size=512x512: 130.9 ± 0.6 ms
11/19. ResNet-DPED	11.1 — inference | batch=10, size=256x256: 114.9 ± 0.6 ms11.2 — inference | batch=1, size=1024x1024: 182 ± 2 ms11.3 — training | batch=15, size=128x128: 178.1 ± 0.8 ms	11.1 — inference | batch=10, size=256x256: 146.4 ± 0.5 ms11.2 — inference | batch=1, size=1024x1024: 234.3 ± 0.5 ms11.3 — training | batch=15, size=128x128: 234.7 ± 0.6 ms
12/19. U-Net	12.1 — inference | batch=4, size=512x512: 180.8 ± 0.7 ms12.2 — inference | batch=1, size=1024x1024: 177.0 ± 0.4 ms12.3 — training | batch=4, size=256x256: 198.6 ± 0.5 ms	12.1 — inference | batch=4, size=512x512: 222.9 ± 0.5 ms12.2 — inference | batch=1, size=1024x1024: 220.4 ± 0.6 ms12.3 — training | batch=4, size=256x256: 229.1 ± 0.7 ms
13/19. Nvidia-SPADE	13.1 — inference | batch=5, size=128x128: 54.5 ± 0.5 ms13.2 — training | batch=1, size=128x128: 103.6 ± 0.6 ms	13.1 — inference | batch=5, size=128x128: 59.6 ± 0.6 ms13.2 — training | batch=1, size=128x128: 94.6 ± 0.6 ms
14/19. ICNet	14.1 — inference | batch=5, size=1024x1536: 126.3 ± 0.8 ms14.2 — training | batch=10, size=1024x1536: 426 ± 9 ms	14.1 — inference | batch=5, size=1024x1536: 144 ± 4 ms14.2 — training | batch=10, size=1024x1536: 475 ± 17 ms
15/19. PSPNet	15.1 — inference | batch=5, size=720x720: 249 ± 12 ms15.2 — training | batch=1, size=512x512: 104.6 ± 0.6 ms	15.1 — inference | batch=5, size=720x720: 291.4 ± 0.5 ms15.2 — training | batch=1, size=512x512: 99.8 ± 0.9 ms
16/19. DeepLab	16.1 — inference | batch=2, size=512x512: 71.7 ± 0.6 ms16.2 — training | batch=1, size=384x384: 84.9 ± 0.5 ms	16.1 — inference | batch=2, size=512x512: 71.5 ± 0.7 ms16.2 — training | batch=1, size=384x384: 69.4 ± 0.6 ms
17/19. Pixel-RNN	17.1 — inference | batch=50, size=64x64: 299 ± 14 ms17.2 — training | batch=10, size=64x64: 1258 ± 64 ms	17.1 — inference | batch=50, size=64x64: 321 ± 30 ms17.2 — training | batch=10, size=64x64: 1278 ± 74 ms
18/19. LSTM-Sentiment	18.1 — inference | batch=100, size=1024x300: 395 ± 11 ms18.2 — training | batch=10, size=1024x300: 676 ± 15 ms	18.1 — inference | batch=100, size=1024x300: 345 ± 10 ms18.2 — training | batch=10, size=1024x300: 774 ± 17 ms
19/19. GNMT-Translation	19.1 — inference | batch=1, size=1x20: 119 ± 2 ms	19.1 — inference | batch=1, size=1x20: 156 ± 1 ms

The results of this test show that the performance of the RTX A4000 is 6% higher than RTX A4000 ADA, however, with the caveat that the test results may vary depending on the specific task and operating conditions employed.

RTX A 4000

Benchmarking	Model average train time (ms)
Training double precision type mnasnet0_5	62.995805740356445
Training double precision type mnasnet0_75	98.39066505432129
Training double precision type mnasnet1_0	126.60405158996582
Training double precision type mnasnet1_3	186.89460277557373
Training double precision type resnet18	428.08079719543457
Training double precision type resnet34	883.5790348052979
Training double precision type resnet50	1016.3950300216675
Training double precision type resnet101	1927.2308254241943
Training double precision type resnet152	2815.663013458252
Training double precision type resnext50_32x4d	1075.4373741149902
Training double precision type resnext101_32x8d	4050.0641918182373
Training double precision type wide_resnet50_2	2615.9953451156616
Training double precision type wide_resnet101_2	5218.524832725525
Training double precision type densenet121	751.9759511947632
Training double precision type densenet169	910.3225564956665
Training double precision type densenet201	1163.036551475525
Training double precision type densenet161	2141.505298614502
Training double precision type squeezenet1_0	203.988
Training double precision type squeezenet1_1	98.04857730865479
Training double precision type vgg11	1697.7
Training double precision type vgg11_bn	1729.2972660064697
Training double precision type vgg13	2491.6
Training double precision type vgg13_bn	2545.34
Training double precision type vgg16	3371.68
Training double precision type vgg16_bn	3423.8639068603516
Training double precision type vgg19_bn	4314.55
Training double precision type vgg19	4249.422650337219
Training double precision type mobilenet_v3_large	105.546
Training double precision type mobilenet_v3_small	37.6680850982666
Training double precision type shufflenet_v2_x0_5	26.51611328125
Training double precision type shufflenet_v2_x1_0	61.260504722595215
Training double precision type shufflenet_v2_x1_5	105.30067920684814
Training double precision type shufflenet_v2_x2_0	181.03694438934326
Inference double precision type mnasnet0_5	17.397074699401855
Inference double precision type mnasnet0_75	28.902697563171387
Inference double precision type mnasnet1_0	38.3877
Inference double precision type mnasnet1_3	58.228821754455566
Inference double precision type resnet18	147.95727252960205
Inference double precision type resnet34	293.5
Inference double precision type resnet50	336.44991874694824
Inference double precision type resnet101	637.9982376098633
Inference double precision type resnet152	948.9351654052734
Inference double precision type resnext50_32x4d	372.80876636505127
Inference double precision type resnext101_32x8d	1385.09
Inference double precision type wide_resnet50_2	873.048791885376
Inference double precision type wide_resnet101_2	1729.2765426635742
Inference double precision type densenet121	270.354
Inference double precision type densenet169	327.06
Inference double precision type densenet201	414.733362197876
Inference double precision type densenet161	766.3542318344116
Inference double precision type squeezenet1_0	74.86292839050293
Inference double precision type squeezenet1_1	34.04905319213867
Inference double precision type vgg11	576.3767147064209
Inference double precision type vgg11_bn	580.5839586257935
Inference double precision type vgg13	853.4365510940552
Inference double precision type vgg13_bn	860.39
Inference double precision type vgg16	1145.091052055359
Inference double precision type vgg16_bn	1152.8028392791748
Inference double precision type vgg19_bn	1444.9562692642212
Inference double precision type vgg19	1437.0987701416016
Inference double precision type mobilenet_v3_large	30.8763
Inference double precision type mobilenet_v3_small	11.234536170959473
Inference double precision type shufflenet_v2_x0_5	7.425284385681152
Inference double precision type shufflenet_v2_x1_0	18.25782299041748
Inference double precision type shufflenet_v2_x1_5	33.34946632385254
Inference double precision type shufflenet_v2_x2_0	57.84676551818848

RTX A4000 ADA

Benchmarking	Model average train time
Training half precision type mnasnet0_5	20.2666
Training half precision type mnasnet0_75	21.445374488830566
Training half precision type mnasnet1_0	26.7625
Training half precision type mnasnet1_3	26.57
Training half precision type resnet18	19.624991416931152
Training half precision type resnet34	32.46446132659912
Training half precision type resnet50	57.332
Training half precision type resnet101	98.209
Training half precision type resnet152	138.967
Training half precision type resnext50_32x4d	75.56005001068115
Training half precision type resnext101_32x8d	228.8706636428833
Training half precision type wide_resnet50_2	113.76442432403564
Training half precision type wide_resnet101_2	204.838
Training half precision type densenet121	68.97401332855225
Training half precision type densenet169	85.957
Training half precision type densenet201	103.299241065979
Training half precision type densenet161	137.54578113555908
Training half precision type squeezenet1_0	16.729
Training half precision type squeezenet1_1	12.906527519226074
Training half precision type vgg11	51.7004919052124
Training half precision type vgg11_bn	57.63327598571777
Training half precision type vgg13	86.809
Training half precision type vgg13_bn	95.86676120758057
Training half precision type vgg16	102.918
Training half precision type vgg16_bn	113.74778270721436
Training half precision type vgg19_bn	131.56734943389893
Training half precision type vgg19	119.706
Training half precision type mobilenet_v3_large	31.30636692047119
Training half precision type mobilenet_v3_small	19.44464683532715
Training half precision type shufflenet_v2_x0_5	13.7766
Training half precision type shufflenet_v2_x1_0	23.608479499816895
Training half precision type shufflenet_v2_x1_5	26.793746948242188
Training half precision type shufflenet_v2_x2_0	24.550962448120117
Inference half precision type mnasnet0_5	4.4934
Inference half precision type mnasnet0_75	4.0253
Inference half precision type mnasnet1_0	4.42598819732666
Inference half precision type mnasnet1_3	4.6809
Inference half precision type resnet18	5.803341865539551
Inference half precision type resnet34	9.756693840026855
Inference half precision type resnet50	15.873079299926758
Inference half precision type resnet101	28.268003463745117
Inference half precision type resnet152	40.04594326019287
Inference half precision type resnext50_32x4d	19.53421115875244
Inference half precision type resnext101_32x8d	62.44826316833496
Inference half precision type wide_resnet50_2	33.533992767333984
Inference half precision type wide_resnet101_2	59.60897445678711
Inference half precision type densenet121	18.052735328674316
Inference half precision type densenet169	21.956982612609863
Inference half precision type densenet201	27.851
Inference half precision type densenet161	37.414
Inference half precision type squeezenet1_0	4.3978
Inference half precision type squeezenet1_1	2.42833
Inference half precision type vgg11	17.623
Inference half precision type vgg11_bn	18.40585231781006
Inference half precision type vgg13	28.4386
Inference half precision type vgg13_bn	30.672597885131836
Inference half precision type vgg16	34.43562984466553
Inference half precision type vgg16_bn	36.929
Inference half precision type vgg19_bn	43.1406
Inference half precision type vgg19	40.5385684967041
Inference half precision type mobilenet_v3_large	5.3507
Inference half precision type mobilenet_v3_small	4.0512
Inference half precision type shufflenet_v2_x0_5	5.0797
Inference half precision type shufflenet_v2_x1_0	5.5935
Inference half precision type shufflenet_v2_x1_5	5.649552345275879
Inference half precision type shufflenet_v2_x2_0	5.355663299560547
Training double precision type mnasnet0_5	50.2386999130249
Training double precision type mnasnet0_75	80.66896915435791
Training double precision type mnasnet1_0	103.32422733306885
Training double precision type mnasnet1_3	154.6230697631836
Training double precision type resnet18	337.94031620025635
Training double precision type resnet34	677.7706575393677
Training double precision type resnet50	789.9243211746216
Training double precision type resnet101	1484.3351316452026
Training double precision type resnet152	2170.570478439331
Training double precision type resnext50_32x4d	877.37
Training double precision type resnext101_32x8d	3652.4944639205933
Training double precision type wide_resnet50_2	2154.6
Training double precision type wide_resnet101_2	4176.522083282471
Training double precision type densenet121	607.8699731826782
Training double precision type densenet169	744.6409797668457
Training double precision type densenet201	962.677731513977
Training double precision type densenet161	1759.772515296936
Training double precision type squeezenet1_0	164.3690824508667
Training double precision type squeezenet1_1	78.70647430419922
Training double precision type vgg11	1362.6095294952393
Training double precision type vgg11_bn	1387.2539138793945
Training double precision type vgg13	2006.0230445861816
Training double precision type vgg13_bn	2047.526364326477
Training double precision type vgg16	2702.2086429595947
Training double precision type vgg16_bn	2747.241234779358
Training double precision type vgg19_bn	3447.34
Training double precision type vgg19	3397.990345954895
Training double precision type mobilenet_v3_large	84.65698719024658
Training double precision type mobilenet_v3_small	29.8617
Training double precision type shufflenet_v2_x0_5	27.4073
Training double precision type shufflenet_v2_x1_0	48.322744369506836
Training double precision type shufflenet_v2_x1_5	82.224
Training double precision type shufflenet_v2_x2_0	141.7021369934082
Inference double precision type mnasnet0_5	12.988653182983398
Inference double precision type mnasnet0_75	22.4228
Inference double precision type mnasnet1_0	30.056486129760742
Inference double precision type mnasnet1_3	46.953935623168945
Inference double precision type resnet18	118.04479122161865
Inference double precision type resnet34	231.52336597442627
Inference double precision type resnet50	268.63497734069824
Inference double precision type resnet101	495.20
Inference double precision type resnet152	726.4922094345093
Inference double precision type resnext50_32x4d	291.47679328918457
Inference double precision type resnext101_32x8d	1055.
Inference double precision type wide_resnet50_2	690.69
Inference double precision type wide_resnet101_2	1347.5529861450195
Inference double precision type densenet121	224.35829639434814
Inference double precision type densenet169	268.94
Inference double precision type densenet201	343.51
Inference double precision type densenet161	635.866231918335
Inference double precision type squeezenet1_0	61.92759037017822
Inference double precision type squeezenet1_1	27.0094
Inference double precision type vgg11	462.3375129699707
Inference double precision type vgg11_bn	468.4495782852173
Inference double precision type vgg13	692.82
Inference double precision type vgg13_bn	703.3538103103638
Inference double precision type vgg16	924.4353818893433
Inference double precision type vgg16_bn	936.5075063705444
Inference double precision type vgg19_bn	1169.098300933838
Inference double precision type vgg19	1156.3771772384644
Inference double precision type mobilenet_v3_large	24.2356014251709
Inference double precision type mobilenet_v3_small	8.85490894317627
Inference double precision type shufflenet_v2_x0_5	6.360034942626953
Inference double precision type shufflenet_v2_x1_0	14.3054
Inference double precision type shufflenet_v2_x1_5	24.863481521606445
Inference double precision type shufflenet_v2_x2_0	43.8505744934082

Conclusion

The new graphics card has proven to be an effective solution for a number of work tasks. Thanks to its compact size, it is ideal for powerful SFF (Small Form Factor) computers. Also, it is notable that the 6,144 CUDA cores and 20GB of memory with a 160-bit bus makes this card one of the most productive on the market. Furthermore, a low TDP of 70W helps to reduce power consumption costs. Four Mini-DisplayPort ports allow the card to be used with multiple monitors or as a multi-channel graphics solution.

The RTX 4000 SFF ADA represents a significant advance over previous generations, delivering performance equivalent to a card with twice the power consumption. With no PCIe power connector, the RTX 4000 SFF ADA is easy to integrate into low-power workstations without sacrificing high performance.