Following are the details of the interfaces provided by the Perception Capabilities package. A runner that intends to connect with one of these interfaces should support the input and ouput parameters described below.
Input Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| use_device | bool | Whether to use the device's vision capabilities. |
| prompt | string | The prompt with instructions on how to process the image using the LLM. |
Output Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| text | string | The analysis result generated by the LLM for the given image. |
Input Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| use_device | bool | Whether to use the device's audio capabilities. |
| text | string | The text to be synthesized into speech. |
| voice | string | Optional voice identifier passed through to the speech backend. |
| instructions | string | Optional synthesis instructions passed through to the speech backend. |
Output Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| success | bool | Whether the speech synthesis request completed successfully. |
Input Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| use_device | bool | Whether to use the device's audio capabilities. |
| audio_request_window | int | The duration of the requested device-audio window in seconds. |
Output Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| text | string | The transcription result generated for the given audio. |
| success | bool | Whether the transcription request completed successfully. |
Input Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| text | string | The text to be analyzed for sentiment. |
| use_device | bool | Whether to capture device audio for analysis instead of relying only on the provided text. |
| audio_request_window | int | Requested device-audio window in seconds when use_device is enabled. |
Output Parameters:
| Parameter Name | Type | Description |
|---|---|---|
| text | string | The sentiment label returned for the analyzed input. |
| score | double | Confidence score associated with the returned sentiment label. |
| analyzed_text | string | The text that was actually analyzed by the sentiment service. |