Functions used for testing
Benchmarks for similar workloads among the cloud providers
Hero Image

Hello GPU

Hello GPU is a basic test with no dependencies. It simply boots a machine with a GPU and returns a string.

hello_gpu by replicate on t4
Median15.47s
Average32.82s
Failures0
11.5s 12:01:34 AM
12.47s 12:02:05 AM
85.64s 12:09:10 AM
14.1s 12:03:49 AM
13.21s 12:01:51 AM
14s 12:02:01 AM
14.54s 12:02:05 AM
35.69s 12:04:50 AM
29.13s 12:01:49 AM
26.02s 12:02:02 AM
14.5s 12:01:55 AM
15.36s 12:02:03 AM
15.73s 12:02:14 AM
11.28s 12:02:11 AM
30.56s 12:02:22 AM
68.56s 12:03:02 AM
17.15s 12:02:23 AM
14.72s 12:01:49 AM
33.64s 12:03:52 AM
30.41s 12:01:30 AM
12.25s 12:01:40 AM
13.4s 12:02:27 AM
204.36s 12:04:09 AM
12.53s 12:01:34 AM
143.55s 12:03:22 AM
11.6s 12:01:57 AM
28.31s 12:02:07 AM
30.6s 12:03:55 AM
12.2s 12:02:47 AM
11.84s 12:01:56 AM
15.57s 12:02:20 AM
45.68s 12:02:08 AM
hello_gpu by mystic on t4
MedianNaNs
AverageNaNs
Failures32
0.1s 12:00:07 AM
0.12s 12:00:13 AM
0.09s 12:00:25 AM
0.09s 12:00:07 AM
0.03s 12:00:20 AM
0.03s 12:00:37 AM
0.12s 12:00:44 AM
0.07s 12:00:36 AM
0.09s 12:00:03 AM
0.12s 12:00:27 AM
0.18s 12:00:20 AM
0.11s 12:00:21 AM
100.08s 12:02:13 AM
100.04s 12:02:30 AM
100.19s 12:02:07 AM
100.04s 12:02:29 AM
100.04s 12:01:59 AM
100.09s 12:01:59 AM
100.04s 12:02:10 AM
100.11s 12:01:48 AM
100.07s 12:01:51 AM
100.08s 12:02:25 AM
100.06s 12:02:28 AM
100.08s 12:01:40 AM
100.08s 12:02:41 AM
100.08s 12:02:11 AM
100.08s 12:02:10 AM
100.13s 12:02:01 AM
100.08s 12:02:12 AM
100.05s 12:02:07 AM
100.14s 12:02:15 AM
100.08s 12:02:07 AM

Hello Torch

Hello Torch is a basic test with Torch as a dependency. It loads Torch and runs some simple matrix multiplications on the GPU.

hello_torch by replicate on t4
Median98.89s
Average127.46s
Failures0
90.09s 12:01:34 AM
116.38s 12:02:05 AM
528.49s 12:09:10 AM
225.71s 12:03:49 AM
94.54s 12:01:51 AM
86.49s 12:02:01 AM
81.81s 12:02:05 AM
254.74s 12:04:50 AM
109.18s 12:01:49 AM
95.44s 12:02:02 AM
95.42s 12:01:55 AM
101.86s 12:02:03 AM
101.96s 12:02:14 AM
81.28s 12:02:11 AM
114.58s 12:02:22 AM
132.44s 12:03:02 AM
124.22s 12:02:23 AM
89.45s 12:01:49 AM
201.82s 12:03:52 AM
81.79s 12:01:30 AM
85.42s 12:01:40 AM
102.7s 12:02:27 AM
83.9s 12:04:09 AM
90.79s 12:01:34 AM
87.3s 12:03:22 AM
83.2s 12:01:57 AM
97.19s 12:02:07 AM
214.22s 12:03:55 AM
135.24s 12:02:47 AM
85.2s 12:01:56 AM
105.14s 12:02:20 AM
100.6s 12:02:08 AM
hello_torch by mystic on t4
MedianNaNs
AverageNaNs
Failures32
0.1s 12:00:07 AM
0.1s 12:00:13 AM
0.09s 12:00:25 AM
0.13s 12:00:07 AM
0.03s 12:00:20 AM
0.03s 12:00:37 AM
0.12s 12:00:44 AM
0.07s 12:00:36 AM
0.08s 12:00:03 AM
0.12s 12:00:27 AM
0.17s 12:00:20 AM
0.11s 12:00:21 AM
100.08s 12:02:13 AM
100.05s 12:02:30 AM
100.14s 12:02:07 AM
100.04s 12:02:29 AM
100.04s 12:01:59 AM
100.08s 12:01:59 AM
100.04s 12:02:10 AM
100.1s 12:01:48 AM
100.08s 12:01:51 AM
100.14s 12:02:25 AM
100.06s 12:02:28 AM
100.08s 12:01:40 AM
100.08s 12:02:41 AM
100.07s 12:02:11 AM
100.08s 12:02:10 AM
100.11s 12:02:01 AM
100.08s 12:02:12 AM
100.04s 12:02:07 AM
100.13s 12:02:15 AM
100.08s 12:02:07 AM

Hello TTS

Hello TTS is a test that runs the bark text-to-speech model on a single sentence. This test is small, but it's for a real use case that includes loading a big model into memory and running inference.

hello_tts by modal on a10g
Median138.09s
Average145.46s
Failures0
183.84s 7:49:42 PM
126.36s 7:48:54 PM
132.27s 7:48:52 PM
129.93s 7:48:51 PM
126.74s 7:48:45 PM
126.75s 7:48:49 PM
129.68s 7:49:08 PM
127.9s 7:48:48 PM
121.74s 7:48:37 PM
114.29s 7:48:42 PM
125.28s 7:48:45 PM
125.1s 7:48:43 PM
144.21s 7:49:11 PM
127.04s 7:48:51 PM
147.87s 7:49:12 PM
139.28s 7:49:00 PM
179.15s 7:49:37 PM
149.43s 7:49:16 PM
146.72s 7:49:19 PM
132.9s 7:49:00 PM
136.9s 7:49:03 PM
144.65s 7:49:09 PM
195.48s 7:49:58 PM
190.15s 7:49:49 PM
190.53s 7:49:53 PM
154.23s 7:52:52 PM
185.45s 7:49:44 PM
129.86s 7:48:47 PM
145.11s 7:49:05 PM
160.83s 7:49:29 PM
135.72s 7:48:57 PM
149.47s 7:49:09 PM
hello_tts by beam on a10g
MedianNaNs
AverageNaNs
Failures32
142.82s 7:49:42 PM
91.8s 7:48:54 PM
106.81s 7:48:52 PM
95.66s 7:48:51 PM
96.42s 7:48:45 PM
85.77s 7:48:49 PM
95.23s 7:49:08 PM
89.42s 7:48:48 PM
95.59s 7:48:37 PM
90.32s 7:48:42 PM
97.87s 7:48:45 PM
96.87s 7:48:43 PM
97.57s 7:49:11 PM
94.3s 7:48:51 PM
112.14s 7:49:12 PM
100.53s 7:49:00 PM
113.31s 7:49:37 PM
109.38s 7:49:16 PM
111.03s 7:49:19 PM
99.56s 7:49:00 PM
91.32s 7:49:03 PM
103.19s 7:49:09 PM
96.88s 7:49:58 PM
148.43s 7:49:49 PM
155.27s 7:49:53 PM
127.58s 7:52:52 PM
154.18s 7:49:44 PM
95.5s 7:48:47 PM
108.04s 7:49:05 PM
119.06s 7:49:29 PM
105.24s 7:48:57 PM
107.18s 7:49:09 PM

About

See the code used at github.com/viktorfa/serverless-gpu-samples