inferless init

Use this command to initialize a new model import.

​

Usage

inferless init

You will be basked for the follwing information:

​

Prompts

Enter the model configuration file name (default: inferless.yaml):

How do you want to import the model? (default: Local) [Local, Git]:

Enter the model name:

Enter the region where you want to deploy the model (region-1, region-2) :

GPU Type ( A100 / A10 / T4 ) :

Do you want to use Dedicated/Shared GPU? (default: Dedicated) [Dedicated, Shared]:

Do you have a custom runtime ? : (default: No) [Yes, No]:

Do you want to use previously created custom runtime ? : (default: No) [yes, No]:

(If no is selected) Generate the runtime with requirements.txt? (default: No) [Yes, No]:

(If yes is selected) Select the custom runtime from the list:

(If yes is selected) Select the custom runtime version: (default: Latest version ie. 0)

​

Output

Once init is complete you will see the below files created

./
├── app.py
├── input_schema.py 
├── inferless-runtime-config.yaml
└── inferless.yaml

• input_schema.py This file defines the structure and validation rules for the input data that a model expects. This file is crucial for ensuring that the data fed into the model is in the correct format and meets all necessary requirements.

• inferless-runtime-config.yamlThis file will have all the software packages and the Python packages required for the model inferencing.

• inferless.yamlThis file will have all the configurations required for the deployment. Users can update this file according to their requirements.

​

Example usage

You can create the below files and then run the command

app.py

from diffusers import StableDiffusionPipeline
import torch
from io import BytesIO
import base64

class InferlessPythonModel:
    def initialize(self):
        self.pipe = StableDiffusionPipeline.from_pretrained(
            "stabilityai/stable-diffusion-2-1",
            use_safetensors=True,
            torch_dtype=torch.float16,
            device_map='auto'
        )

    def infer(self, inputs):
        prompt = inputs["prompt"]
        image = self.pipe(prompt).images[0]
        buff = BytesIO()
        image.save(buff, format="JPEG")
        img_str = base64.b64encode(buff.getvalue()).decode()
        return { "generated_image_base64" : img_str }
        
    def finalize(self):
        self.pipe = None

​

Example input_schema.py

# input_schema.py
INPUT_SCHEMA = {
    "prompt": {
        'datatype': 'STRING',
        'required': True,
        'shape': [1],
        'example': ["There is a fine house in the forest"]
    }
}