inferless init

Use this command to initialize a new model import.

Usage:

$ inferless init [OPTIONS] COMMAND [ARGS]...

Options:

• -n, --name TEXT: Denotes the name of the model.
• -s, --source TEXT: Not needed if local, else provide Github/Gitlab. [default: local]
• -u, --url TEXT: Denotes the URL of the repo. required if source is not local.
• -b, --branch TEXT: Denotes the branch where the model is located. required if source is not local.
• -a, --autobuild: Enable autobuild for the model. will be False for local source.

Commands:

• docker: Initialize with Docker.
• file: Import a PyTorch, ONNX, or TensorFlow file…
• hf: Load a model from Hugging Face.
• pythonic: (Default) Deploy a Python workflow.

​

inferless init

(Default) Deploy a Python workflow.

Usage:

$ inferless init [OPTIONS]

Options:

• -n, --name TEXT: Denotes the name of the model. [required]
• -s, --source TEXT: Not needed if local, else provide Github/Gitlab. [default: local]
• -u, --url TEXT: Denotes the URL of the repo. required if source is not local.
• -b, --branch TEXT: Denotes the branch where the model is located. required if source is not local.
• -a, --autobuild: Enable autobuild for the model. will be False for local source.

​

Example usage

You can run the command

inferless init -n inferless-onboarding

Then create the below files

​

Example app.py

from diffusers import StableDiffusionPipeline
import torch
from io import BytesIO
import base64

class InferlessPythonModel:
    def initialize(self):
        self.pipe = StableDiffusionPipeline.from_pretrained(
            "stabilityai/stable-diffusion-2-1", use_safetensors=True,
            torch_dtype=torch.float16, device_map='auto'
        )

    def infer(self, inputs):
        prompt = inputs["prompt"]
        image = self.pipe(prompt).images[0]
        buff = BytesIO()
        image.save(buff, format="JPEG")
        return { "generated_image_base64" : base64.b64encode(buff.getvalue()).decode() }

​

Example input_schema.py

# input_schema.py
INPUT_SCHEMA = {
    "prompt": {
        'datatype': 'STRING',
        'required': True,
        'shape': [1],
        'example': ["There is a fine house in the forest"]
    }
}

inferless deploy --gpu T4

​

Sub Commands

​

Hugging Face.

This command creates new files called app.py and input_schema.py using the hugging face model name in you active dir

Usage:

$ inferless init hf [OPTIONS]

Options:

• -n, --name TEXT: Denotes the name of the model. [required]
• -m, --hfmodelname TEXT: Name of the Hugging Face repo. [required]
• -t, --modeltype TEXT: Type of the model (transformer/diffuser). [required]
• -k, --tasktype TEXT: Task type of the model (text-generation). [required]

Transformers options:

• audio-classification
• automatic-speech-recognition
• conversational
• depth-estimation
• document-question-answering
• feature-extraction
• fill-mask
• image-classification
• image-segmentation
• image-to-text
• object-detection
• question-answering
• summarization
• table-question-answering
• text-classification
• text-generation
• text2text-generation
• token-classification
• translation
• video-classification
• visual-question-answering
• zero-shot-classification
• zero-shot-image-classification
• zero-shot-object-detection

Diffusers options:

• Depth-to-Image
• Image-Variation
• Image-to-Image
• Inpaint
• InstructPix2Pix
• Stable-Diffusion-Latent-Upscaler

Once init is complete you will see the below files created

./
├── app.py
├── input_schema.py
└── inferless.yaml

• input_schema.py This file defines the structure and validation rules for the input data that a model expects. This file is crucial for ensuring that the data fed into the model is in the correct format and meets all necessary requirements.

• inferless-runtime-config.yamlThis file will have all the software packages and the Python packages required for the model inferencing.

• inferless.yamlThis file will have all the configurations required for the deployment. Users can update this file according to their requirements.

​

Docker

Usage:

$ inferless init docker [OPTIONS]

Options:

• -n, --name TEXT: Denotes the name of the model. [required]
• -t, --type TEXT: Type for import: dockerimage/dockerfile. [required]
• -p, --provider TEXT: Provider for the model dockerimage = (dockerhub/ecr) dockerfile = (github/gitlab). [required]
• -u, --url TEXT: Docker image URL or GitHub/GitLab URL. [required]
• -b, --branch TEXT: Branch for Dockerfile import (GitHub/GitLab). required if type is dockerfile.
• -d, --dockerfilepath TEXT: Path to the Dockerfile. required if type is dockerfile.
• -h, --healthapi TEXT: Health check API endpoint. [required]
• -i, --inferapi TEXT: Inference API endpoint. [required]
• -s, --serverport INTEGER: Server port. [required]
• -a, --autobuild: Enable autobuild for the model.

​

File ( PyTorch/ ONNX /TensorFlow ) inference with Triton server.

The folder structure for the zip file should be as follows:

.

├── config.pbtxt (optional)

├── input.json

├── output.json

├── 1/

│ ├── model.xxx (pt/onnx/savedmodel)

Usage:

$ inferless init file [OPTIONS]

Options:

• -n, --name TEXT: Denotes the name of the model. [required]
• -f, --framework TEXT: Framework of the model. [pytorch, onnx, tensorflow] [default: pytorch]
• -p, --provider TEXT: Provider for the model (local/gcs/s3). [default: local]
• --url TEXT: Provider URL. required if provider is not local.