Stability AI has released Stable Diffusion 3, an advanced text-to-image generation model with significant improvements over its predecessors. This new version features a range of models from 800M to 8B parameters, providing users with scalable options to suit their needs.
Stable Diffusion 3 sets a new benchmark in image generation, delivering unparalleled image quality with enhanced efficiency. Utilizing a sophisticated Multimodal Diffusion Transformer (MMDiT) architecture, it significantly reduces noise and improves clarity. The model incorporates three advanced text encoders (OpenCLIP-ViT/G, CLIP-ViT/L and T5-xxl) to better understand and execute complex prompts. With innovative sampling methods like Rectified Flow, Stable Diffusion 3 ensures a streamlined path from noise to a detailed image, making it a powerful tool for creating high-fidelity images efficiently.
We have deployed this model using A100 GPU and observed that the model took an average cold start time of 9.9 sec
and an average inference time of 4.4 sec
for 28
steps image generation.
We are using the HuggingFace Diffusers library for the deployment.
Now quickly construct the GitHub/GitLab template, this process is mandatory and make sure you don’t add any file named model.py
You can also add other files to this directory.
In the app.py we will define the class and import all the required functions
def initialize
: In this function, you will initialize your model and define any variable
that you want to use during inference. We are using huggingface access token
which will help us to download the gated model.
def infer
: This function gets called for every request that you send. Here you can define all the steps that are required for the inference. You can also pass custom values for inference through the inputs
parameter.
def finalize
: This function cleans up all the allocated memory.
We have to create a input_schema.py
in the GitHub/Gitlab repository this will help us create the Input parameters. You can checkout our documentation on Input / Output Schema.
For this tutorial, we have defined four parameters prompt
, negative_prompt
, num_inference_steps
and guidance_scale
which are required during the API call. Now lets create the input_schema.py
.
This is a mandatory step where we allow the users to upload their own custom runtime through inferless-runtime-config.yaml.
You can use the inferless remote-run
(installation guide here) command to test your model or any custom Python script in a remote GPU environment directly from your local machine. Make sure that you use Python3.10
for seamless experience.
To enable Remote Run, simply do the following:
inferless
library and initialize Cls(gpu="A10")
. The available GPU options are T4
, A10
and A100
.initialize
and infer
functions with @app.load
and @app.infer
respectively.my_local_entry
) with @inferless.local_entry_point
.
Within this function, instantiate your model class, convert any incoming parameters into a RequestObjects
object, and invoke the model’s infer
method.From your local terminal, navigate to the folder containing your app.py
and your inferless-runtime-config.yaml
and run:
If you want to exclude certain files or directories from being uploaded, use the --exclude
or -e
flag.
Inferless supports multiple ways of importing your model. For this tutorial, we will use GitHub.
Navigate to your desired workspace in Inferless and Click on Add a custom model
button that you see on the top right. An import wizard will open up.
Once the model is in ‘Active’ status you can click on the ‘API’ page to call the model
Inferless allows you to deploy your model using Inferless-CLI. Follow the steps to deploy using Inferless CLI.
Let’s begin by cloning the model repository:
To deploy the model using Inferless CLI, execute the following command:
Explanation of the Command:
--gpu A100
: Specifies the GPU type for deployment. Available options include A10
, A100
, and T4
.--runtime inferless-runtime-config.yaml
: Defines the runtime configuration file. If not specified, the default Inferless runtime is used.
Stability AI has released Stable Diffusion 3, an advanced text-to-image generation model with significant improvements over its predecessors. This new version features a range of models from 800M to 8B parameters, providing users with scalable options to suit their needs.
Stable Diffusion 3 sets a new benchmark in image generation, delivering unparalleled image quality with enhanced efficiency. Utilizing a sophisticated Multimodal Diffusion Transformer (MMDiT) architecture, it significantly reduces noise and improves clarity. The model incorporates three advanced text encoders (OpenCLIP-ViT/G, CLIP-ViT/L and T5-xxl) to better understand and execute complex prompts. With innovative sampling methods like Rectified Flow, Stable Diffusion 3 ensures a streamlined path from noise to a detailed image, making it a powerful tool for creating high-fidelity images efficiently.
We have deployed this model using A100 GPU and observed that the model took an average cold start time of 9.9 sec
and an average inference time of 4.4 sec
for 28
steps image generation.
We are using the HuggingFace Diffusers library for the deployment.
Now quickly construct the GitHub/GitLab template, this process is mandatory and make sure you don’t add any file named model.py
You can also add other files to this directory.
In the app.py we will define the class and import all the required functions
def initialize
: In this function, you will initialize your model and define any variable
that you want to use during inference. We are using huggingface access token
which will help us to download the gated model.
def infer
: This function gets called for every request that you send. Here you can define all the steps that are required for the inference. You can also pass custom values for inference through the inputs
parameter.
def finalize
: This function cleans up all the allocated memory.
We have to create a input_schema.py
in the GitHub/Gitlab repository this will help us create the Input parameters. You can checkout our documentation on Input / Output Schema.
For this tutorial, we have defined four parameters prompt
, negative_prompt
, num_inference_steps
and guidance_scale
which are required during the API call. Now lets create the input_schema.py
.
This is a mandatory step where we allow the users to upload their own custom runtime through inferless-runtime-config.yaml.
You can use the inferless remote-run
(installation guide here) command to test your model or any custom Python script in a remote GPU environment directly from your local machine. Make sure that you use Python3.10
for seamless experience.
To enable Remote Run, simply do the following:
inferless
library and initialize Cls(gpu="A10")
. The available GPU options are T4
, A10
and A100
.initialize
and infer
functions with @app.load
and @app.infer
respectively.my_local_entry
) with @inferless.local_entry_point
.
Within this function, instantiate your model class, convert any incoming parameters into a RequestObjects
object, and invoke the model’s infer
method.From your local terminal, navigate to the folder containing your app.py
and your inferless-runtime-config.yaml
and run:
If you want to exclude certain files or directories from being uploaded, use the --exclude
or -e
flag.
Inferless supports multiple ways of importing your model. For this tutorial, we will use GitHub.
Navigate to your desired workspace in Inferless and Click on Add a custom model
button that you see on the top right. An import wizard will open up.
Once the model is in ‘Active’ status you can click on the ‘API’ page to call the model
Inferless allows you to deploy your model using Inferless-CLI. Follow the steps to deploy using Inferless CLI.
Let’s begin by cloning the model repository:
To deploy the model using Inferless CLI, execute the following command:
Explanation of the Command:
--gpu A100
: Specifies the GPU type for deployment. Available options include A10
, A100
, and T4
.--runtime inferless-runtime-config.yaml
: Defines the runtime configuration file. If not specified, the default Inferless runtime is used.