Installation
pip install --upgrade inferless
Usage
This client can be used to call Inferless API from your Python code. It supports both synchronous and asynchronous calls.
inferless.call
inferless.call(url: str, workspace_api_key: str, data: dict ) -> Response [dict]
This method sends a synchronous request to the Inferless API endpoint
Parameters
-
url - Inferless Model API URL
-
workspace_api_key - Inferless Workspace API Key
-
data - Model Input Data in Dict Format
-
inputs (Optional) - Model Input Data in Inference Protocol format, data should not be given is this is given
Example
import inferless
URL = "https://<model>.default.model-v2.inferless.com/v2/models/<model>/versions/1/infer"
API_KEY = "76e51......39b57"
data = {"prompt" : "a horse near a beach"}
# This call is synced until the response is returned
result = inferless.call(URL, API_KEY, data)
inferless.call_async
inferless.call_async(URL: str, workspace_api_key: str, data: dict, callback=None)
This method sends a request to the inferless endpoint in the background.
Parameters
-
url - Inferless Model API URL
-
workspace_api_key - Inferless Workspace API Key
-
data - Model Input Data in Dict Format
-
inputs (Optional) - Model Input Data in Inference Protocol format, data should not be given is this is given
-
**callback - The callback function will be called after receiving the response.
- callback function should have two params:
callback(error, response)
- error - any error resulting while calling the inferless endpoint
- response - response from the inferless endpoint
import inferless
URL = "https://<model>.default.model-v2.inferless.com/v2/models/<model>/versions/1/infer"
API_KEY = "76e51......39b57"
data = {"prompt" : "a horse near a beach"}
# callback function which writes the response to a file
def callback_func(e, response):
# e is the error object
# response is the response object
print(response)
# This is not a blocking call and runs in the background and callback function is triggered to write the endpoint response to a file
inferless.call_async(URL, KEY, DATA, callback_func)
import inferless
URL = "https://<model>.default.model-v2.inferless.com/v2/models/<model>/versions/1/infer"
API_KEY = "76e51......39b57"
inputs = {
"inputs": [
{
"name": "prompt",
"shape": [
1
],
"data": [
"a horse near a beach"
],
"datatype": "BYTES"
}
]
}
# This call is synced until the response is returned
result = inferless.call(URL, API_KEY, inputs=inputs)
