Stable Cascade distinguishes itself by operating within a significantly smaller latent space, offering faster inference and cost-effective training.
Stable Cascade is based on the Würstchen architecture, differs from others like Stable Diffusion by operating in a smaller latent space. A smaller latent space means faster inference and cheaper training. For instance, Stable Cascade achieves a compression factor of 42, allowing encoding of a 1024x1024 image to 24x24 while retaining clear reconstructions. The text-conditional model is trained in this highly compressed latent space.
We have deployed this model using A100 GPU and observed that the model took an average cold start time of 9.64 sec
and an average inference time of 3.02 sec
for image generation.
We are using the HuggingFace Diffusers library for the deployment.
Now quickly construct the GitHub/GitLab template, this process is mandatory and make sure you don’t add any file named model.py
You can also add other files to this directory.
In the app.py we will define the class and import all the required functions
def initialize
: In this function, you will initialize your model and define any variable
that you want to use during inference.
def infer
: This function gets called for every request that you send. Here you can define all the steps that are required for the inference. You can also pass custom values for inference through the inputs
parameter.
def finalize
: This function cleans up all the allocated memory.
We have to create a input_schema.py
in your GitHub/Gitlab repository this will help us create the Input parameters. You can checkout our documentation on Input / Output Schema.
For this tutorial, we have defined two parameters prompt
and negative_prompt
which is required during the API call. Now lets create the input_schema.py
.
This is a mandatory step where we allow the users to upload their own custom runtime through inferless-runtime-config.yaml.
You can use the inferless remote-run
(installation guide here) command to test your model or any custom Python script in a remote GPU environment directly from your local machine. Make sure that you use Python3.10
for seamless experience.
To enable Remote Run, simply do the following:
inferless
library and initialize Cls(gpu="A100")
. The available GPU options are T4
, A10
and A100
.initialize
and infer
functions with @app.load
and @app.infer
respectively.my_local_entry
) with @inferless.local_entry_point
.
Within this function, instantiate your model class, convert any incoming parameters into a RequestObjects
object, and invoke the model’s infer
method.From your local terminal, navigate to the folder containing your app.py
and your inferless-runtime-config.yaml
and run:
You can pass the other input parameters in the same way as long as your code expects them in the inputs
dictionary.
If you want to exclude certain files or directories from being uploaded, use the --exclude
or -e
flag.
Inferless supports multiple ways of importing your model. For this tutorial, we will use GitHub.
Navigate to your desired workspace in Inferless and Click on Add a custom model
button that you see on the top right. An import wizard will open up.
Once the model is in ‘Active’ status you can click on the ‘API’ page to call the model
Inferless allows you to deploy your model using Inferless-CLI. Follow the steps to deploy using Inferless CLI.
Let’s begin by cloning the model repository:
To deploy the model using Inferless CLI, execute the following command:
Explanation of the Command:
--gpu A100
: Specifies the GPU type for deployment. Available options include A10
, A100
, and T4
.--runtime inferless-runtime-config.yaml
: Defines the runtime configuration file. If not specified, the default Inferless runtime is used.
Stable Cascade distinguishes itself by operating within a significantly smaller latent space, offering faster inference and cost-effective training.
Stable Cascade is based on the Würstchen architecture, differs from others like Stable Diffusion by operating in a smaller latent space. A smaller latent space means faster inference and cheaper training. For instance, Stable Cascade achieves a compression factor of 42, allowing encoding of a 1024x1024 image to 24x24 while retaining clear reconstructions. The text-conditional model is trained in this highly compressed latent space.
We have deployed this model using A100 GPU and observed that the model took an average cold start time of 9.64 sec
and an average inference time of 3.02 sec
for image generation.
We are using the HuggingFace Diffusers library for the deployment.
Now quickly construct the GitHub/GitLab template, this process is mandatory and make sure you don’t add any file named model.py
You can also add other files to this directory.
In the app.py we will define the class and import all the required functions
def initialize
: In this function, you will initialize your model and define any variable
that you want to use during inference.
def infer
: This function gets called for every request that you send. Here you can define all the steps that are required for the inference. You can also pass custom values for inference through the inputs
parameter.
def finalize
: This function cleans up all the allocated memory.
We have to create a input_schema.py
in your GitHub/Gitlab repository this will help us create the Input parameters. You can checkout our documentation on Input / Output Schema.
For this tutorial, we have defined two parameters prompt
and negative_prompt
which is required during the API call. Now lets create the input_schema.py
.
This is a mandatory step where we allow the users to upload their own custom runtime through inferless-runtime-config.yaml.
You can use the inferless remote-run
(installation guide here) command to test your model or any custom Python script in a remote GPU environment directly from your local machine. Make sure that you use Python3.10
for seamless experience.
To enable Remote Run, simply do the following:
inferless
library and initialize Cls(gpu="A100")
. The available GPU options are T4
, A10
and A100
.initialize
and infer
functions with @app.load
and @app.infer
respectively.my_local_entry
) with @inferless.local_entry_point
.
Within this function, instantiate your model class, convert any incoming parameters into a RequestObjects
object, and invoke the model’s infer
method.From your local terminal, navigate to the folder containing your app.py
and your inferless-runtime-config.yaml
and run:
You can pass the other input parameters in the same way as long as your code expects them in the inputs
dictionary.
If you want to exclude certain files or directories from being uploaded, use the --exclude
or -e
flag.
Inferless supports multiple ways of importing your model. For this tutorial, we will use GitHub.
Navigate to your desired workspace in Inferless and Click on Add a custom model
button that you see on the top right. An import wizard will open up.
Once the model is in ‘Active’ status you can click on the ‘API’ page to call the model
Inferless allows you to deploy your model using Inferless-CLI. Follow the steps to deploy using Inferless CLI.
Let’s begin by cloning the model repository:
To deploy the model using Inferless CLI, execute the following command:
Explanation of the Command:
--gpu A100
: Specifies the GPU type for deployment. Available options include A10
, A100
, and T4
.--runtime inferless-runtime-config.yaml
: Defines the runtime configuration file. If not specified, the default Inferless runtime is used.