Use this command to initialize a new model import. Usage: 
$ inferless init [OPTIONS] COMMAND [ARGS]...
Options:
  • -n, --name TEXT: Denotes the name of the model.
  • -s, --source TEXT: Not needed if local, else provide Github/Gitlab. [default: local]
  • -u, --url TEXT: Denotes the URL of the repo. required if source is not local.
  • -b, --branch TEXT: Denotes the branch where the model is located. required if source is not local.
  • -a, --autobuild: Enable autobuild for the model. will be False for local source.
Commands:
  • docker: Initialize with Docker.
  • file: Import a PyTorch, ONNX, or TensorFlow file…
  • hf: Load a model from Hugging Face.
  • pythonic: (Default) Deploy a Python workflow.

​
inferless init

(Default) Deploy a Python workflow. Usage: 
$ inferless init [OPTIONS]
Options:
  • -n, --name TEXT: Denotes the name of the model. [required]
  • -s, --source TEXT: Not needed if local, else provide Github/Gitlab. [default: local]
  • -u, --url TEXT: Denotes the URL of the repo. required if source is not local.
  • -b, --branch TEXT: Denotes the branch where the model is located. required if source is not local.
  • -a, --autobuild: Enable autobuild for the model. will be False for local source.

​
Example usage

You can run the command 
inferless init -n inferless-onboarding
Then create the below files

​
Example app.py


from diffusers import StableDiffusionPipeline
import torch
from io import BytesIO
import base64

class InferlessPythonModel:
    def initialize(self):
        self.pipe = StableDiffusionPipeline.from_pretrained(
            "stabilityai/stable-diffusion-2-1", use_safetensors=True,
            torch_dtype=torch.float16, device_map='auto'
        )

    def infer(self, inputs):
        prompt = inputs["prompt"]
        image = self.pipe(prompt).images[0]
        buff = BytesIO()
        image.save(buff, format="JPEG")
        return { "generated_image_base64" : base64.b64encode(buff.getvalue()).decode() }

​
Example input_schema.py

# input_schema.py
INPUT_SCHEMA = {
    "prompt": {
        'datatype': 'STRING',
        'required': True,
        'shape': [1],
        'example': ["There is a fine house in the forest"]
    }
}

inferless deploy --gpu T4

​
Sub Commands

​
Hugging Face.

This command creates new files called app.py and input_schema.py using the hugging face model name in you active dir Usage: 
$ inferless init hf [OPTIONS]
Options:
  • -n, --name TEXT: Denotes the name of the model. [required]
  • -m, --hfmodelname TEXT: Name of the Hugging Face repo. [required]
  • -t, --modeltype TEXT: Type of the model (transformer/diffuser). [required]
  • -k, --tasktype TEXT: Task type of the model (text-generation). [required]
Transformers options:
  • audio-classification
  • automatic-speech-recognition
  • conversational
  • depth-estimation
  • document-question-answering
  • feature-extraction
  • fill-mask
  • image-classification
  • image-segmentation
  • image-to-text
  • object-detection
  • question-answering
  • summarization
  • table-question-answering
  • text-classification
  • text-generation
  • text2text-generation
  • token-classification
  • translation
  • video-classification
  • visual-question-answering
  • zero-shot-classification
  • zero-shot-image-classification
  • zero-shot-object-detection
Diffusers options:
  • Depth-to-Image
  • Image-Variation
  • Image-to-Image
  • Inpaint
  • InstructPix2Pix
  • Stable-Diffusion-Latent-Upscaler
Once init is complete you will see the below files created 
./
├── app.py
├── input_schema.py
└── inferless.yaml
  • input_schema.py This file defines the structure and validation rules for the input data that a model expects. This file is crucial for ensuring that the data fed into the model is in the correct format and meets all necessary requirements.
  • inferless-runtime-config.yamlThis file will have all the software packages and the Python packages required for the model inferencing.
  • inferless.yamlThis file will have all the configurations required for the deployment. Users can update this file according to their requirements.

​
Docker

Usage: 
$ inferless init docker [OPTIONS]
Options:
  • -n, --name TEXT: Denotes the name of the model. [required]
  • -t, --type TEXT: Type for import: dockerimage/dockerfile. [required]
  • -p, --provider TEXT: Provider for the model dockerimage = (dockerhub/ecr) dockerfile = (github/gitlab). [required]
  • -u, --url TEXT: Docker image URL or GitHub/GitLab URL. [required]
  • -b, --branch TEXT: Branch for Dockerfile import (GitHub/GitLab). required if type is dockerfile.
  • -d, --dockerfilepath TEXT: Path to the Dockerfile. required if type is dockerfile.
  • -h, --healthapi TEXT: Health check API endpoint. [required]
  • -i, --inferapi TEXT: Inference API endpoint. [required]
  • -s, --serverport INTEGER: Server port. [required]
  • -a, --autobuild: Enable autobuild for the model.

​
File ( PyTorch/ ONNX /TensorFlow ) inference with Triton server.

The folder structure for the zip file should be as follows: . ├── config.pbtxt (optional) ├── input.json ├── output.json ├── 1/ │ ├── model.xxx (pt/onnx/savedmodel) Usage: 
$ inferless init file [OPTIONS]
Options:
  • -n, --name TEXT: Denotes the name of the model. [required]
  • -f, --framework TEXT: Framework of the model. [pytorch, onnx, tensorflow] [default: pytorch]
  • -p, --provider TEXT: Provider for the model (local/gcs/s3). [default: local]
  • --url TEXT: Provider URL. required if provider is not local.