inferless init
Use this command to initialize a new model import.
Usage:
Options:
-n, --name TEXT
: Denotes the name of the model.-s, --source TEXT
: Not needed if local, else provide Github/Gitlab. [default: local]-u, --url TEXT
: Denotes the URL of the repo. required if source is not local.-b, --branch TEXT
: Denotes the branch where the model is located. required if source is not local.-a, --autobuild
: Enable autobuild for the model. will be False for local source.
Commands:
docker
: Initialize with Docker.file
: Import a PyTorch, ONNX, or TensorFlow file…hf
: Load a model from Hugging Face.pythonic
: (Default) Deploy a Python workflow.
inferless init
(Default) Deploy a Python workflow.
Usage:
Options:
-n, --name TEXT
: Denotes the name of the model. [required]-s, --source TEXT
: Not needed if local, else provide Github/Gitlab. [default: local]-u, --url TEXT
: Denotes the URL of the repo. required if source is not local.-b, --branch TEXT
: Denotes the branch where the model is located. required if source is not local.-a, --autobuild
: Enable autobuild for the model. will be False for local source.
Example usage
You can run the command
Then create the below files
Example app.py
Example input_schema.py
Sub Commands
Hugging Face.
This command creates new files called app.py and input_schema.py using the hugging face model name in you active dir
Usage:
Options:
-n, --name TEXT
: Denotes the name of the model. [required]-m, --hfmodelname TEXT
: Name of the Hugging Face repo. [required]-t, --modeltype TEXT
: Type of the model (transformer/diffuser). [required]-k, --tasktype TEXT
: Task type of the model (text-generation). [required]
Transformers options:
- audio-classification
- automatic-speech-recognition
- conversational
- depth-estimation
- document-question-answering
- feature-extraction
- fill-mask
- image-classification
- image-segmentation
- image-to-text
- object-detection
- question-answering
- summarization
- table-question-answering
- text-classification
- text-generation
- text2text-generation
- token-classification
- translation
- video-classification
- visual-question-answering
- zero-shot-classification
- zero-shot-image-classification
- zero-shot-object-detection
Diffusers options:
- Depth-to-Image
- Image-Variation
- Image-to-Image
- Inpaint
- InstructPix2Pix
- Stable-Diffusion-Latent-Upscaler
Once init is complete you will see the below files created
-
input_schema.py
This file defines the structure and validation rules for the input data that a model expects. This file is crucial for ensuring that the data fed into the model is in the correct format and meets all necessary requirements. -
inferless-runtime-config.yaml
This file will have all the software packages and the Python packages required for the model inferencing. -
inferless.yaml
This file will have all the configurations required for the deployment. Users can update this file according to their requirements.
Docker
Usage:
Options:
-n, --name TEXT
: Denotes the name of the model. [required]-t, --type TEXT
: Type for import: dockerimage/dockerfile. [required]-p, --provider TEXT
: Provider for the model dockerimage = (dockerhub/ecr) dockerfile = (github/gitlab). [required]-u, --url TEXT
: Docker image URL or GitHub/GitLab URL. [required]-b, --branch TEXT
: Branch for Dockerfile import (GitHub/GitLab). required if type is dockerfile.-d, --dockerfilepath TEXT
: Path to the Dockerfile. required if type is dockerfile.-h, --healthapi TEXT
: Health check API endpoint. [required]-i, --inferapi TEXT
: Inference API endpoint. [required]-s, --serverport INTEGER
: Server port. [required]-a, --autobuild
: Enable autobuild for the model.
File ( PyTorch/ ONNX /TensorFlow ) inference with Triton server.
The folder structure for the zip file should be as follows:
.
├── config.pbtxt (optional)
├── input.json
├── output.json
├── 1/
│ ├── model.xxx (pt/onnx/savedmodel)
Usage:
Options:
-n, --name TEXT
: Denotes the name of the model. [required]-f, --framework TEXT
: Framework of the model. [pytorch, onnx, tensorflow] [default: pytorch]-p, --provider TEXT
: Provider for the model (local/gcs/s3). [default: local]--url TEXT
: Provider URL. required if provider is not local.