Stability AI has released Stable Diffusion 3, an advanced text-to-image generation model with significant improvements over its predecessors. This new version features a range of models from 800M to 8B parameters, providing users with scalable options to suit their needs.
9.9 sec
and an average inference time of 4.4 sec
for 28
steps image generation.
model.py
def initialize
: In this function, you will initialize your model and define any variable
that you want to use during inference. We are using huggingface access token
which will help us to download the gated model.
def infer
: This function gets called for every request that you send. Here you can define all the steps that are required for the inference. You can also pass custom values for inference through the inputs
parameter.
def finalize
: This function cleans up all the allocated memory.
input_schema.py
in the GitHub/Gitlab repository this will help us create the Input parameters. You can checkout our documentation on Input / Output Schema.
For this tutorial, we have defined four parameters prompt
, negative_prompt
, num_inference_steps
and guidance_scale
which are required during the API call. Now lets create the input_schema.py
.
inferless remote-run
(installation guide here) command to test your model or any custom Python script in a remote GPU environment directly from your local machine. Make sure that you use Python3.10
for seamless experience.
inferless
library and initialize Cls(gpu="A10")
. The available GPU options are T4
, A10
and A100
.initialize
and infer
functions with @app.load
and @app.infer
respectively.my_local_entry
) with @inferless.local_entry_point
.
Within this function, instantiate your model class, convert any incoming parameters into a RequestObjects
object, and invoke the model’s infer
method.app.py
and your inferless-runtime-config.yaml
and run:
--exclude
or -e
flag.
Add a custom model
button that you see on the top right. An import wizard will open up.
--gpu A100
: Specifies the GPU type for deployment. Available options include A10
, A100
, and T4
.--runtime inferless-runtime-config.yaml
: Defines the runtime configuration file. If not specified, the default Inferless runtime is used.