> ## Documentation Index
> Fetch the complete documentation index at: https://docs.inferless.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Hugging face

### Supported Frameworks

You can use a `transformer` or a `diffuser` the based model from Hugging face.

### Steps to load your model

To import a model from Hugging Face, below are the requirements.

As a next step, we would need to import the Hugging Face model into GitHub before we push it to Inverness. How Inferless works is:`Hugging Face`-> `Copy and create a repo in GitHub` -> `Load the model repo into Inferless.`

You can use the imported GitHub Repo to change the pre-processing and post-processing code.

### Pre Requisite: Note the Model Name, Type, and Framework

* Navigate to the Hugging Face model page of your choice that you want to import into Inferless.

* Take note of the `"Model Name" `(you can also use the copy button), `Task Type`, `Model Framework,` and `Model Type`. These will be required for the next steps.

<Frame caption="The fields to be copied/noted are mentioned in RED.">
  <img src="https://mintcdn.com/inferless-68/QJ9zE2jx7w8an0q6/images/hugging1.png?fit=max&auto=format&n=QJ9zE2jx7w8an0q6&q=85&s=cfe80bca7c36218c146d90d201a7194d" alt="" width="2876" height="1440" data-path="images/hugging1.png" />
</Frame>

### Step 1: Add Model in your workspace.

* Navigate to your desired workspace in Inferless and Click on `"Huggingface" `.  An import wizard will open up.

<Frame caption="Click on Add Model">
  <img src="https://mintcdn.com/inferless-68/gmxryf1AUBjwgBTM/images/step1-common-add-model.png?fit=max&auto=format&n=gmxryf1AUBjwgBTM&q=85&s=335816a9e142a39c2f38ec3f82f7e1d5" alt="" width="3416" height="1868" data-path="images/step1-common-add-model.png" />
</Frame>

### Step 2: Enter the model details

* **Model Details** : In this step, Add your `model name`(The name that you wish to call your model), Choose the `model type`(Eg: Transformer), Choose the `task type` (Eg: Text generation) and `Huggingface model name`.

<Frame caption="Enter the details as noted">
  <img src="https://mintcdn.com/inferless-68/QJ9zE2jx7w8an0q6/images/hf-3-model-details.png?fit=max&auto=format&n=QJ9zE2jx7w8an0q6&q=85&s=992c9c4e0aba1deb109f6d2417f54d08" alt="" width="2734" height="1454" data-path="images/hf-3-model-details.png" />
</Frame>

* In case you would like to set up `Automatic rebuild` for your model, enable it
  * You would need to set up a webhook for this method. Click [here](/model-import/automatic-build-via-webhooks) for more details.

### Step 3: Edit the Inference Code and Input/Output Schema

* **Model Code**c: In this step, you can modify the input params ( by adding to input\_schema.py ) and output params, you can also modify the model load and inference code in app.py

<Frame caption="Enter the details as noted">
  <img src="https://mintcdn.com/inferless-68/QJ9zE2jx7w8an0q6/images/hf-3-model-import.png?fit=max&auto=format&n=QJ9zE2jx7w8an0q6&q=85&s=754dce6a77397ecbd7f9cabde4aea85b" alt="" width="3396" height="1862" data-path="images/hf-3-model-import.png" />
</Frame>

### Step 4: Configure Machine and Environment.

* Choose the type of machine, and specify the minimum and maximum number of replicas for deploying your model.
  * **Min scale** -
  ```
  The number of inference workers to keep on at all times.  
  ```
  * **Max scale -**
  ```
  The maximum number of inference workers to allow at any point of time  
  ```
  * Configure Custom Runtime ( If you have pip or apt packages), choose Volume, Secrets and set Environment variables like Inference Timeout / Container Concurrency / Scale Down Timeout

<Frame caption="Set runtime and configuration">
  <img src="https://mintcdn.com/inferless-68/vUTTbQ74sBpn0RYd/images/common-machine-env.png?fit=max&auto=format&n=vUTTbQ74sBpn0RYd&q=85&s=3eaef88476d1f11ee23700d3a88895c5" alt="" width="2728" height="1436" data-path="images/common-machine-env.png" />
</Frame>

### Step 5: Review your model details

* Once you click "Continue," you will be able to review the details added for the model.

* If you would like to make any changes, you can go back and make the changes.

* Once you have reviewed everything, click `Deploy` to start the model import process.

<Frame caption="Review all the details carefully before proceeding">
  <img src="https://mintcdn.com/inferless-68/vUTTbQ74sBpn0RYd/images/common-deploy.png?fit=max&auto=format&n=vUTTbQ74sBpn0RYd&q=85&s=9d046c6e502d80dd98ffb3b129facf8b" alt="" width="2742" height="1450" data-path="images/common-deploy.png" />
</Frame>

### Step 6 : Run your model

* Once you click submit, the model import process would start.

* It may take some time to complete the import process, and during this time, you will be redirected to your workspace and can see the status of the import under `"In Progress/Failed"` tab.\\
  <Frame caption="View the model under `In-Progress/ Failed`">
    <img src="https://mintcdn.com/inferless-68/vUTTbQ74sBpn0RYd/images/common-in-progress.png?fit=max&auto=format&n=vUTTbQ74sBpn0RYd&q=85&s=528f121c863b5a68dbb485ea97b1405a" alt="" width="2724" height="1436" data-path="images/common-in-progress.png" />
  </Frame>

* If you encounter any errors during the model import process or if you want to view the build logs for any reason, you can click on the three dots menu and select "View build logs". This will show you a detailed log of the import process, which can help you troubleshoot any issues you may encounter.

* Post-upload, the model will be available under "My Models"

* You can then select the model and go to `"My Model" -> API -> Inference Endpoint details. `Here you would find the API endpoints that can be called. You can click on the copy button on the right and can call your model.

<Frame caption="Under the API Tab, you can view the API endpoint details.">
  <img src="https://mintcdn.com/inferless-68/vUTTbQ74sBpn0RYd/images/common-api-call.png?fit=max&auto=format&n=vUTTbQ74sBpn0RYd&q=85&s=4cf7b8f43919c8c509b8c61d1eb3f5bc" alt="" width="2732" height="1454" data-path="images/common-api-call.png" />
</Frame>

### Extra Step: Getting API key details

* You can now call using this from your end. The inference result would be the output for these calls.
* In case you need help with API Keys:
  * Click on settings, available on the top, next to your Workspace Name
  * Click on "Workspace API keys"
  * You can view the details of your key or generate a new one\\
    <Frame caption="Sample for now">
      <img src="https://mintcdn.com/inferless-68/vUTTbQ74sBpn0RYd/images/common-workspace-api.png?fit=max&auto=format&n=vUTTbQ74sBpn0RYd&q=85&s=2f473cf4a12b3697fd10c468a0f1957c" alt="" width="2732" height="1446" data-path="images/common-workspace-api.png" />
    </Frame>

Here is a sample video of the whole process for a 7GB Stable diffusion Model : [Click to view](https://www.youtube.com/watch?v=5vYsmPu9pZM)

### Using CLI

You can use any model you want from Huggingface and deploy with us

1. Connect your [Huggingface](https://huggingface.co/login) account using this command(below) and enter the name and apikey. Make sure your github account is integrated with [Inferless](https://console.inferless.com/user/integration)

   ```bash theme={null}
   inferless integration add HF --name <name> --api-key <apikey>
   ```

2. Once you’re done with integration. Run this command to initialise the model

   ```bash theme={null}
   inferless init hf --name <modelname> --hfmodelname <hfreponame> --modeltype <transfomer/diffuser> --tasktype <type> 
   ```

   Transformer options:

   * audio-classification
   * automatic-speech-recognition
   * conversational
   * depth-estimation
   * document-question-answering
   * feature-extraction
   * fill-mask
   * image-classification
   * image-segmentation
   * image-to-text
   * object-detection
   * question-answering
   * summarization
   * table-question-answering
   * text-classification
   * text-generation
   * text2text-generation
   * token-classification
   * translation
   * video-classification
   * visual-question-answering
   * zero-shot-classification
   * zero-shot-image-classification
   * zero-shot-object-detection

   Diffuser options:

   * Depth-to-Image
   * Image-Variation
   * Image-to-Image
   * Inpaint
   * InstructPix2Pix
   * Stable-Diffusion-Latent-Upscaler

3. Now that your model is initialised.

   * To do default deployment use this command

   ```bash theme={null}
   inferless deploy --gpu t4 
   ```

   * To do customised deployment use this command

   ```bash theme={null}
   inferless deploy --gpu t4 --region <regionname> --runtime <runtimename> --volume <volumename> --fractional  
   ```