Bring your own docker container images. ( This might have higher coldstarts )
This is depreciated in the new version of Inferless, If you need to import via docker please reach out to us at alerts@inferless.com
You can provide a custom docker image to host your model on the inferless platform. Supported formats for custom docker images.
For Inferless to work with the Image you have to meet the following specification :
--inferapi
)GitHub - inferless/inferless-docker-import-examplesGitHub
"Add a custom model"
button that you see on the top right. An import wizard will open up.Click on Add Model
Since we are using a model from Docker Hub, select Docker Hub
as the method of upload from the Provider
list.
To proceed with the upload, you will need to connect your Docker Hub account
. This is a mandatory step as this helps us download the image from your Docker Hub.
After you have selected Repo
Image URL
, Health check API
, Infer API
and Server Port
Model Name
: The desired name of the model that you wish to giveEnter the details as noted
Automatic rebuild
for your model, enable it
Set runtime and configuration
Once you click “Continue,” you will be able to review the details added for the model.
If you would like to make any changes, you can go back and make the changes.
Once you have reviewed everything, click Deploy
to start the model import process.
Review all the details carefully before proceeding
Once you click submit, the model import process would start.
It may take some time to complete the import process, and during this time, you will be redirected to your workspace and can see the status of the import under "In Progress/Failed"
tab.
View the model under `In-Progress/ Failed`
If you encounter any errors during the model import process or if you want to view the build logs for any reason, you can click on the three dots menu and select “View build logs”. This will show you a detailed log of the import process, which can help you troubleshoot any issues you may encounter.
Post-upload, the model will be available under “My Models”
You can then select the model and go to -> API -> Inference Endpoint details.
Here you would find the API endpoints that can be called. You can click on the copy button on the right and can call your model.
Under the API Tab, you can view the API endpoint details.
Sample for now
You can deploy the model with Inferless using docker image. Just make sure the image is stored either in dockerhub/ ecr
Connect your Dockerhub account using this command(below) and enter the username and access token
Once you’re done with integration. Run this command to initialise the model
Now that your model is initialised.
"Add a custom model"
button that you see on the top right. An import wizard will open up.Click on Add Model
Since we are using a model from Dockerfile, select Dockerfile
as the method of upload from the Provider
list.
To proceed with the upload, you will need to connect your GitHub/GitLab account
. This is a mandatory step as this helps us get the file from your repository.
After you have selected Repo
Github Repository
and the branch
.Health check API
, Infer API
, Server Port
and the Docker File Path
Model Name
: The desired name of the model that you wish to giveEnter the details as noted
Automatic rebuild
for your model, enable it
Set runtime and configuration
Once you click “Continue,” you will be able to review the details added for the model.
If you would like to make any changes, you can go back and make the changes.
Once you have reviewed everything, click Deploy
to start the model import process.
Review all the details carefully before proceeding
It may take some time to complete the import process, and during this time, you will be redirected to your workspace and can see the status of the import under "In Progress/Failed"
tab.
View the model under `In-Progress/ Failed`
If you encounter any errors during the model import process or if you want to view the build logs for any reason, you can click on the three dots menu and select “View build logs”. This will show you a detailed log of the import process, which can help you troubleshoot any issues you may encounter.
Post-upload, the model will be available under “My Models”
You can then select the model and go to -> API -> Inference Endpoint details.
Here you would find the API endpoints that can be called. You can click on the copy button on the right and can call your model.
Under the API Tab, you can view the API endpoint details.
Sample for now
If you’ve a dockerfile with you, we can build the image. Just make sure you’re dockerfile is stored in Github/ Gitlab for us to access it.
Login to Inferless.com and click on three dot menu near the profile picture
Click on Manage integrations and connect with your Github/ Gitlab account were the model is stored.
Once you’re done with integration on console. Open CLI and run this command to initialise the model
Now that your model is initialised.