Functions used for testing
Benchmarks for similar workloads among the cloud providers
Hero Image

Hello GPU

Hello GPU is a basic test with no dependencies. It simply boots a machine with a GPU and returns a string.

hello_gpu by replicate on t4
Median19.55s
Average43.91s
Failures0
37.42s 12:01:26 AM
17.31s 12:01:48 AM
65.29s 12:02:27 AM
11.35s 12:01:52 AM
12.35s 12:01:57 AM
24.01s 12:02:37 AM
142.7s 7:49:42 PM
13.18s 12:02:05 AM
20.57s 12:02:10 AM
13.46s 12:03:53 AM
114.63s 12:04:37 AM
91.68s 7:48:54 PM
20.85s 12:02:48 AM
13.22s 12:01:27 AM
106.6s 7:48:52 PM
18.54s 12:01:32 AM
15.83s 12:02:27 AM
95.46s 7:48:51 PM
13.22s 12:02:12 AM
12.23s 12:01:59 AM
96.15s 7:48:45 PM
10.07s 12:02:40 AM
hello_gpu by runpod on 16gb
Median102.04s
Average110.74s
Failures0
147.17s 7:49:47 PM
96.38s 7:48:58 PM
111.08s 7:48:56 PM
97.02s 7:48:53 PM
102.04s 7:48:51 PM
hello_gpu by mystic on t4
Median106.67s
Average135.16s
Failures13
100.08s 12:01:45 AM
100.06s 12:02:08 AM
100.05s 12:02:39 AM
100.04s 12:02:04 AM
100.05s 12:02:15 AM
100.06s 12:02:41 AM
150.64s 7:49:50 PM
100.05s 12:02:15 AM
100.08s 12:02:15 AM
14.12s 12:01:49 AM
100.07s 12:02:13 AM
340.41s 7:53:02 PM
18.25s 12:01:53 AM
12s 12:01:45 AM
262.83s 7:51:27 PM
100.06s 12:01:44 AM
38.3s 12:02:39 AM
273.23s 7:51:49 PM
100.04s 12:02:15 AM
100.06s 12:02:20 AM
106.67s 7:48:56 PM
100.05s 12:02:40 AM
hello_gpu by inferless on t4
Median117.24s
Average122.61s
Failures2
143.11s 7:49:43 PM
112.8s 7:49:14 PM
166.95s 7:49:52 PM
137.78s 7:49:34 PM
117.24s 7:49:06 PM
hello_gpu by modal on t4
Median146.54s
Average155.8s
Failures0
198.3s 7:49:42 PM
148.74s 7:48:54 PM
146.54s 7:48:52 PM
145.04s 7:48:51 PM
140.39s 7:48:45 PM
hello_gpu by beam on t4
MedianNaNs
AverageNaNs
Failures5
142.81s 7:49:42 PM
91.82s 7:48:54 PM
106.82s 7:48:52 PM
95.68s 7:48:51 PM
96.46s 7:48:45 PM

Hello Torch

Hello Torch is a basic test with Torch as a dependency. It loads Torch and runs some simple matrix multiplications on the GPU.

hello_torch by replicate on t4
Median88.11s
Average104.84s
Failures0
80.6s 12:01:26 AM
77.8s 12:01:48 AM
87.9s 12:02:27 AM
85.39s 12:01:52 AM
82.3s 12:01:57 AM
95.89s 12:02:37 AM
118.41s 7:49:42 PM
89.87s 12:02:05 AM
95.66s 12:02:10 AM
227.53s 12:03:53 AM
246.83s 12:04:37 AM
76.33s 7:48:54 PM
154.69s 12:02:48 AM
78.88s 12:01:27 AM
95.61s 7:48:52 PM
84.58s 12:01:32 AM
88.33s 12:02:27 AM
81.1s 7:48:51 PM
97.4s 12:02:12 AM
78.99s 12:01:59 AM
82.31s 7:48:45 PM
99.99s 12:02:40 AM
hello_torch by runpod on 16gb
Median105.81s
Average116.06s
Failures0
152.47s 7:49:52 PM
101.21s 7:49:03 PM
116.06s 7:49:01 PM
105.81s 7:49:01 PM
104.73s 7:48:54 PM
hello_torch by inferless on t4
Median112.76s
Average112.76s
Failures4
143.1s 7:49:43 PM
112.76s 7:49:14 PM
136.94s 7:49:22 PM
125.87s 7:49:21 PM
126.27s 7:49:15 PM
hello_torch by modal on t4
Median143.03s
Average152.16s
Failures0
194.53s 7:49:42 PM
145.32s 7:48:54 PM
143.03s 7:48:52 PM
140.88s 7:48:51 PM
137.02s 7:48:45 PM
hello_torch by mystic on t4
Median233.12s
Average207.59s
Failures16
100.1s 12:01:45 AM
100.07s 12:02:08 AM
100.06s 12:02:39 AM
100.05s 12:02:04 AM
71.68s 12:02:15 AM
44.14s 12:02:41 AM
293.75s 7:52:13 PM
100.07s 12:02:15 AM
100.08s 12:02:15 AM
100.08s 12:01:49 AM
100.07s 12:02:13 AM
391.9s 7:53:54 PM
100.04s 12:01:53 AM
100.05s 12:01:45 AM
236.43s 7:51:01 PM
100.05s 12:01:44 AM
100.09s 12:02:39 AM
229.81s 7:51:05 PM
100.05s 12:02:15 AM
100.06s 12:02:20 AM
369.74s 7:53:19 PM
100.05s 12:02:40 AM
hello_torch by beam on t4
MedianNaNs
AverageNaNs
Failures5
142.87s 7:49:42 PM
91.8s 7:48:54 PM
106.83s 7:48:52 PM
95.66s 7:48:51 PM
96.38s 7:48:45 PM

Hello TTS

Hello TTS is a test that runs the bark text-to-speech model on a single sentence. This test is small, but it's for a real use case that includes loading a big model into memory and running inference.

hello_tts by modal on a10g
Median138.09s
Average145.46s
Failures0
183.84s 7:49:42 PM
126.36s 7:48:54 PM
132.27s 7:48:52 PM
129.93s 7:48:51 PM
126.74s 7:48:45 PM
126.75s 7:48:49 PM
129.68s 7:49:08 PM
127.9s 7:48:48 PM
121.74s 7:48:37 PM
114.29s 7:48:42 PM
125.28s 7:48:45 PM
125.1s 7:48:43 PM
144.21s 7:49:11 PM
127.04s 7:48:51 PM
147.87s 7:49:12 PM
139.28s 7:49:00 PM
179.15s 7:49:37 PM
149.43s 7:49:16 PM
146.72s 7:49:19 PM
132.9s 7:49:00 PM
136.9s 7:49:03 PM
144.65s 7:49:09 PM
195.48s 7:49:58 PM
190.15s 7:49:49 PM
190.53s 7:49:53 PM
154.23s 7:52:52 PM
185.45s 7:49:44 PM
129.86s 7:48:47 PM
145.11s 7:49:05 PM
160.83s 7:49:29 PM
135.72s 7:48:57 PM
149.47s 7:49:09 PM
hello_tts by beam on a10g
MedianNaNs
AverageNaNs
Failures32
142.82s 7:49:42 PM
91.8s 7:48:54 PM
106.81s 7:48:52 PM
95.66s 7:48:51 PM
96.42s 7:48:45 PM
85.77s 7:48:49 PM
95.23s 7:49:08 PM
89.42s 7:48:48 PM
95.59s 7:48:37 PM
90.32s 7:48:42 PM
97.87s 7:48:45 PM
96.87s 7:48:43 PM
97.57s 7:49:11 PM
94.3s 7:48:51 PM
112.14s 7:49:12 PM
100.53s 7:49:00 PM
113.31s 7:49:37 PM
109.38s 7:49:16 PM
111.03s 7:49:19 PM
99.56s 7:49:00 PM
91.32s 7:49:03 PM
103.19s 7:49:09 PM
96.88s 7:49:58 PM
148.43s 7:49:49 PM
155.27s 7:49:53 PM
127.58s 7:52:52 PM
154.18s 7:49:44 PM
95.5s 7:48:47 PM
108.04s 7:49:05 PM
119.06s 7:49:29 PM
105.24s 7:48:57 PM
107.18s 7:49:09 PM

About

See the code used at github.com/viktorfa/serverless-gpu-samples