Skip to main content

Input Schema

You have to define the input_schema.py in your GitHub/Gitlab repository this will help us create the Input parameters : For each input, there are 3 fields required
  • datatype: “STRING”, “BOOL”, “INT8”, “INT16”, “INT32”, “FP16” “FP32”, “UINT8”, “UINT16”, “UINT32”, “UINT64”, “INT64” , “FP64” , “BYTES”, “BF16”
  • shape: The length of the array, If the shape is [1] you will get the variable, if the array > 1 you will get an array, If the length is variable you can put -1
  • required: If the parameter is required in all API calls
  • example( optional ): Sample value for calling the API
In code
In input_schema.py
Example

Output Schema

You can return any dictionary in the return statement of app.py. You don’t need to provide any configuration.

Returning Dicts

Returning Variable Length Array

Returning Dictionary with Variable keys

Depreciated - Input / Output Json

Sample Input

The input JSON should contain the following fields:
  1. name - the name should match the name of the input/output that is specified in the model
  2. shape - the shape of the input array for the model. if the shape is variable use -1
  3. datatype - One of the formats is given below:
BOOL, UINT8, UINT16, UINT32, UINT64, INT8, INT16, INT32, INT64, FP16, FP32, FP64, BYTES, BF16.
For more details, you can view the matrix below the page
  1. data - An example of the data.
Note: Since an Array of Inputs and Outputs is expected, you may have to convert the dimension of your array with an additional dimension of no of requests. An example of a model that takes attention_mask and input_ids Tensor Arrays of shape [10] for 1 Request will be
Sample of an inputjson
Example of a Model that takes prompt (as string) as the input ( Stable Diffusion )
Sample of a inputjson
Example of Model using the Param

Sample Output

There is the output field that your model returns, Having a sample helps us validate that the name you are expecting in output is generated by the model.
Sample of Output json

Returning Dicts

Corresponding Output.json for the Python code

Returning Variable Length Array

Corresponding Output.json for the Python code, Making the Shape parameter -1 will allow variable length items

Returning Dictionary with Variable keys

Corresponding Output.json for the Python code

Example

Below is a representation after giving the details during model import.