Functions used for testing
Benchmarks for similar workloads among the cloud providers
Hero Image

Hello GPU

Hello GPU is a basic test with no dependencies. It simply boots a machine with a GPU and returns a string.

hello_gpu by replicate on t4
Median15.01s
Average30.76s
Failures0
189.38s 12:03:53 AM
22.06s 12:02:20 AM
11.97s 12:02:25 AM
33.48s 12:03:15 AM
10.58s 12:01:42 AM
24.02s 12:02:06 AM
12.59s 12:02:20 AM
11.39s 12:02:25 AM
17.5s 12:05:20 AM
39.28s 12:01:33 AM
42.09s 12:02:26 AM
20.96s 12:02:21 AM
11.09s 12:01:29 AM
15.44s 12:01:46 AM
47.75s 12:04:21 AM
20.05s 12:02:06 AM
87.29s 12:02:11 AM
12.1s 12:01:37 AM
17.84s 12:02:07 AM
30.64s 12:02:07 AM
12.25s 12:01:40 AM
9.82s 12:02:16 AM
12.04s 12:01:43 AM
14.58s 12:01:23 AM
14.01s 12:02:38 AM
12.06s 12:02:16 AM
21.8s 12:04:40 AM
12.45s 12:02:03 AM
160.09s 12:03:29 AM
11.5s 12:01:50 AM
13.43s 12:01:47 AM
12.94s 12:02:15 AM
hello_gpu by mystic on t4
Median54.76s
Average50.88s
Failures10
100.16s 12:02:23 AM
39.85s 12:02:33 AM
100.05s 12:02:36 AM
100.07s 12:02:34 AM
100.1s 12:01:55 AM
84.48s 12:01:58 AM
100.06s 12:02:24 AM
40.37s 12:02:38 AM
87.04s 12:02:02 AM
6.96s 12:01:43 AM
86.72s 12:01:57 AM
45.21s 12:02:39 AM
10.11s 12:01:39 AM
100.05s 12:01:56 AM
67.17s 12:02:11 AM
75.1s 12:02:15 AM
63.55s 12:02:10 AM
74.96s 12:01:55 AM
57.52s 12:02:24 AM
100.14s 12:02:01 AM
8.13s 12:01:40 AM
47.42s 12:02:19 AM
81.07s 12:01:55 AM
11.13s 12:01:44 AM
100.04s 12:02:36 AM
54.54s 12:02:33 AM
42.46s 12:02:40 AM
100.1s 12:02:20 AM
54.98s 12:02:30 AM
67.09s 12:02:11 AM
13.44s 12:01:49 AM
100.08s 12:02:22 AM

Hello Torch

Hello Torch is a basic test with Torch as a dependency. It loads Torch and runs some simple matrix multiplications on the GPU.

hello_torch by replicate on t4
Median88.96s
Average108.66s
Failures0
82.81s 12:03:53 AM
81.05s 12:02:20 AM
88.72s 12:02:25 AM
144.93s 12:03:15 AM
84.11s 12:01:42 AM
111.26s 12:02:06 AM
95.92s 12:02:20 AM
83.89s 12:02:25 AM
298.31s 12:05:20 AM
82.58s 12:01:33 AM
129.59s 12:02:26 AM
81.41s 12:02:21 AM
86.33s 12:01:29 AM
86.62s 12:01:46 AM
233.06s 12:04:21 AM
90.83s 12:02:06 AM
98.81s 12:02:11 AM
81.92s 12:01:37 AM
83.07s 12:02:07 AM
106.56s 12:02:07 AM
89.2s 12:01:40 AM
92.39s 12:02:16 AM
88.43s 12:01:43 AM
78.54s 12:01:23 AM
104.44s 12:02:38 AM
82.06s 12:02:16 AM
219.8s 12:04:40 AM
82.42s 12:02:03 AM
146.07s 12:03:29 AM
76.69s 12:01:50 AM
95.22s 12:01:47 AM
90.06s 12:02:15 AM
hello_torch by mystic on t4
Median93.94s
Average88.8s
Failures24
100.15s 12:02:23 AM
94.3s 12:02:33 AM
100.06s 12:02:36 AM
100.07s 12:02:34 AM
100.1s 12:01:55 AM
100.2s 12:01:58 AM
53.62s 12:02:24 AM
100.06s 12:02:38 AM
100.1s 12:02:02 AM
92.38s 12:01:43 AM
100.11s 12:01:57 AM
100.07s 12:02:39 AM
95.62s 12:01:39 AM
88.34s 12:01:56 AM
100.05s 12:02:11 AM
100.13s 12:02:15 AM
97.17s 12:02:10 AM
100.05s 12:01:55 AM
100.08s 12:02:24 AM
100.14s 12:02:01 AM
93.59s 12:01:40 AM
95.41s 12:02:19 AM
100.12s 12:01:55 AM
100.05s 12:01:44 AM
100.05s 12:02:36 AM
100.08s 12:02:33 AM
100.06s 12:02:40 AM
100.1s 12:02:20 AM
100.13s 12:02:30 AM
100.14s 12:02:11 AM
100.09s 12:01:49 AM
100.08s 12:02:22 AM

Hello TTS

Hello TTS is a test that runs the bark text-to-speech model on a single sentence. This test is small, but it's for a real use case that includes loading a big model into memory and running inference.

hello_tts by modal on a10g
Median138.09s
Average145.46s
Failures0
183.84s 7:49:42 PM
126.36s 7:48:54 PM
132.27s 7:48:52 PM
129.93s 7:48:51 PM
126.74s 7:48:45 PM
126.75s 7:48:49 PM
129.68s 7:49:08 PM
127.9s 7:48:48 PM
121.74s 7:48:37 PM
114.29s 7:48:42 PM
125.28s 7:48:45 PM
125.1s 7:48:43 PM
144.21s 7:49:11 PM
127.04s 7:48:51 PM
147.87s 7:49:12 PM
139.28s 7:49:00 PM
179.15s 7:49:37 PM
149.43s 7:49:16 PM
146.72s 7:49:19 PM
132.9s 7:49:00 PM
136.9s 7:49:03 PM
144.65s 7:49:09 PM
195.48s 7:49:58 PM
190.15s 7:49:49 PM
190.53s 7:49:53 PM
154.23s 7:52:52 PM
185.45s 7:49:44 PM
129.86s 7:48:47 PM
145.11s 7:49:05 PM
160.83s 7:49:29 PM
135.72s 7:48:57 PM
149.47s 7:49:09 PM
hello_tts by beam on a10g
MedianNaNs
AverageNaNs
Failures32
142.82s 7:49:42 PM
91.8s 7:48:54 PM
106.81s 7:48:52 PM
95.66s 7:48:51 PM
96.42s 7:48:45 PM
85.77s 7:48:49 PM
95.23s 7:49:08 PM
89.42s 7:48:48 PM
95.59s 7:48:37 PM
90.32s 7:48:42 PM
97.87s 7:48:45 PM
96.87s 7:48:43 PM
97.57s 7:49:11 PM
94.3s 7:48:51 PM
112.14s 7:49:12 PM
100.53s 7:49:00 PM
113.31s 7:49:37 PM
109.38s 7:49:16 PM
111.03s 7:49:19 PM
99.56s 7:49:00 PM
91.32s 7:49:03 PM
103.19s 7:49:09 PM
96.88s 7:49:58 PM
148.43s 7:49:49 PM
155.27s 7:49:53 PM
127.58s 7:52:52 PM
154.18s 7:49:44 PM
95.5s 7:48:47 PM
108.04s 7:49:05 PM
119.06s 7:49:29 PM
105.24s 7:48:57 PM
107.18s 7:49:09 PM

About

See the code used at github.com/viktorfa/serverless-gpu-samples