Inferless allows you to easily define input and output schemas for APIs following the Inference Protocol v2. You can now define your inputs and outputs directly within your code using Pydantic models or use an input_schema.py
file. This document explains both methods.
With the new approach, you define your input schema directly in your code using Pydantic models. You no longer need to define an external schema file. Simply use the @inferless.request
decorator to define the required inputs. you need to inferless python clinet
is_aws
), you can set it using Optional
.str
, int
, bool
, float
, List
, etc., to define the expected data types.For outputs, instead of manually handling dictionary returns, you can define the output schema directly within the code using the @inferless.response
decorator. This allows you to declare structured outputs in a more maintainable way.
is_aws
), you can set it using Optional
.str
, int
, bool
, float
, List
, etc., to define the expected data types.Here’s an example showing how you can process these input and output schemas inside your API:
infer
function takes a request
object of type RequestObjects
as input.request
object.ResponseObjects
object with the output parameters.For backward compatibility, you can still use the older method of defining an input schema using an external input_schema.py file. In this method, you create a dictionary that defines the required inputs for your model.
For each input, there are 3 fields required
datatype: “STRING”, “BOOL”, “INT8”, “INT16”, “INT32”, “FP16” “FP32”, “UINT8”, “UINT16”, “UINT32”, “UINT64”, “INT64” , “FP64” , “BYTES”, “BF16”
shape: The length of the array, If the shape is [1] you will get the variable, if the array > 1 you will get an array, If the length is variable you can put -1
required: If the parameter is required in all API calls
example( optional ): Sample value for calling the API
In code
In input_schema.py
More example of varrible length array here
You can return any dictionary in the return statement of app.py. You don’t need to provide any configuration. There are some limitations on the dictories that you can return.
Possible return types are
You can’t have nested dictionaries, arrays of arrays, or arrays of dictionaries.
If you have nested Object/Dictionary you can serialise the object to JSON and return the JSON string.
More example of complex outputs here
Inferless allows you to easily define input and output schemas for APIs following the Inference Protocol v2. You can now define your inputs and outputs directly within your code using Pydantic models or use an input_schema.py
file. This document explains both methods.
With the new approach, you define your input schema directly in your code using Pydantic models. You no longer need to define an external schema file. Simply use the @inferless.request
decorator to define the required inputs. you need to inferless python clinet
is_aws
), you can set it using Optional
.str
, int
, bool
, float
, List
, etc., to define the expected data types.For outputs, instead of manually handling dictionary returns, you can define the output schema directly within the code using the @inferless.response
decorator. This allows you to declare structured outputs in a more maintainable way.
is_aws
), you can set it using Optional
.str
, int
, bool
, float
, List
, etc., to define the expected data types.Here’s an example showing how you can process these input and output schemas inside your API:
infer
function takes a request
object of type RequestObjects
as input.request
object.ResponseObjects
object with the output parameters.For backward compatibility, you can still use the older method of defining an input schema using an external input_schema.py file. In this method, you create a dictionary that defines the required inputs for your model.
For each input, there are 3 fields required
datatype: “STRING”, “BOOL”, “INT8”, “INT16”, “INT32”, “FP16” “FP32”, “UINT8”, “UINT16”, “UINT32”, “UINT64”, “INT64” , “FP64” , “BYTES”, “BF16”
shape: The length of the array, If the shape is [1] you will get the variable, if the array > 1 you will get an array, If the length is variable you can put -1
required: If the parameter is required in all API calls
example( optional ): Sample value for calling the API
In code
In input_schema.py
More example of varrible length array here
You can return any dictionary in the return statement of app.py. You don’t need to provide any configuration. There are some limitations on the dictories that you can return.
Possible return types are
You can’t have nested dictionaries, arrays of arrays, or arrays of dictionaries.
If you have nested Object/Dictionary you can serialise the object to JSON and return the JSON string.
More example of complex outputs here