Whisper-large-v3-turbo is a fast and efficient automatic speech recognition model with 809 million parameters, optimized for transcription and translation.
Library | Inference Time | Cold Start Time |
---|---|---|
Transformers | 0.46 sec | 8.14 sec |
model.py
.
def initialize
: In this function, you will initialize your model and define any variable
that you want to use during inference.
def infer
: This function gets called for every request that you send. Here you can define all the steps that are required for the inference. You can also pass custom values for inference and pass it through inputs(dict)
parameter.
def finalize
: This function cleans up all the allocated memory.
input_schema.py
.
inferless remote-run
(installation guide here) command to test your model or any custom Python script in a remote GPU environment directly from your local machine. Make sure that you use Python3.10
for seamless experience.
inferless
library and initialized Cls(gpu="A10")
.initialize
and infer
functions with @app.load
and @app.infer
respectively.inferless remote-run
command. Since @app.load
automatically handles initialization when the model is instantiated, you can directly use the infer
function without explicitly calling initialize
.app.py
and your inferless-runtime-config.yaml
and run:
--exclude
or -e
flag.
Add a custom model
button that you see on the top right. An import wizard will open up.
--gpu A10
: Specifies the GPU type for deployment. Available options include A10
, A100
, and T4
.--runtime inferless-runtime-config.yaml
: Defines the runtime configuration file. If not specified, the default Inferless runtime is used.