Skip to main content

ai

Connect to an AI Agent.

Parameters​

NameTypeDescription
voicestringTTS voice the AI agent will use. Optional. Default is picked by SignalWire.
promptobjectThe initial set of instructions and settings to configure the agent. Optional. Default is not set. See prompt parameters below.
post_promptobjectThe final set of instructions and configuration settings to send to the agent. Optional. Default is not set. See post_prompt parameters below.
post_prompt_urlstringURL to send status callbacks and reports to. Optional. Default is not set. See Callback Parameters for details on the request body.
post_prompt_auth_userstringAuth username for post_url endpoint. Optional. Default is not set.
post_prompt_auth_passwordstringAuth password for post_url endpoint. Optional. Default is not set.
paramsobjectA JSON object containing parameters as key-value pairs. For the full list of params for possible values. Optional. Default is undefined.
SWAIGobject[]An array of JSON objects to create user-defined functions/endpoints that can be executed during the dialogue. Optional. Default is not set.
hintsstring[]An array of hints (as strings) to provide context to the dialogue. Optional. Default is not set.
languagesobject[]An array of JSON objects defining supported languages in the conversation. Optional. Default is "en-us".
pronounceobject[]An array of JSON objects to clarify the AI's pronunciation of words or expressions. Optional. Default is not set. The fields of this object are the three following.
pronounce.replacestringThe expression to replace. Required.
pronounce.withstringThe phonetic spelling of the expression. Required.
pronounce.ignore_casebooleanWhether the pronunciation replacement should ignore case. Optional. Default is true.

Parameters for prompt and post_prompt​

NameTypeDescription
textstringThe instructions to send to the agent. Optional. Default is not set.
temperaturenumberRandomness setting. Float value between 0.0 and 2.0. Closer to 0 will make the output less random. Optional. Default is 1.0.
top_pnumberRandomness setting. Alternative to temperature. Float value between 0.0 and 1.0. Closer to 0 will make the output less random. Optional. Default is 1.0.
confidencenumberThreshold to fire a speech-detect event at the end of the utterance. Float value between 0.0 and 1.0. Decreasing this value will reduce the pause after the user speaks, but may introduce false positives. Optional. Default is 0.6.
presence_penaltynumberAversion to staying on topic. Float value between -2.0 and 2.0. Positive values increase the model's likelihood to talk about new topics. Optional. Default is 0.
frequency_penaltynumberAversion to repeating lines. Float value between -2.0 and 2.0. Positive values decrease the model's likelihood to repeat the same line verbatim. Optional. Default is 0.
resultobject | arraySet up a switch on return_value by using an object or a cond statement by using an array.

Parameters for SWAIG​

NameTypeDescription
defaultsobjectDefault settings for all SWAIG functions. If defaults is not set, settings may be set in each function object. Optional. Default is not set.
defaults.web_hook_urlstringDefault URL to send status callbacks and reports to. Optional. Default is not set. See Callback Parameters for details on the request body.
defaults.web_hook_auth_userstringDefault auth username for web_hook_url endpoint. Optional. Default is not set.
defaults.web_hook_auth_passwordstringDefault auth password for web_hook_url endpoint. Optional. Default is not set.
native_functionsstring[]Prebuilt functions the AI agent is able to call. Valid values are: "check_time" for the AI to return the time in a location specified by the caller and "wait_seconds" for the AI to track a certain amount of time they need to wait, as specified in the prompt or by a caller.
includesobject[]An array of objects to include remote function signatures. The object fields are url to specify where the remote functions are defined and functions which is an array of the function names as strings. See examples below of implementation.
functionsobject[]An array of JSON objects to define functions that can be executed during the interaction with the AI. Optional. Default is not set. The fields of this object are the six following.
functions.activebooleanWhether the function is active. Optional. Default is true.
functions.functionstringA unique name for the function. For example, "get_weather".
functions.meta_dataobjectA JSON object containing any metadata, as a key-value map. Optional. Default is not set.
functions.meta_data_tokenstringScoping token for meta_data. If not supplied, metadata will be scoped to function's web_hook_url.
functions.data_mapobject[]An object containing properties to process or validate the input, perform actions based on the input, or connect to external APIs or services in a serverless fashion.
functions.web_hook_urlstringFunction-specific URL to send status callbacks and reports to. Takes precedence over a default setting. Optional. Default is not set. See Callback Parameters for details on the request body.
functions.web_hook_auth_userstringFunction-specific auth username for web_hook_url endpoint. Takes precedence over a default setting. Optional. Default is not set.
functions.web_hook_auth_passstringFunction-specific auth password for web_hook_url endpoint. Takes precedence over a default setting. Optional. Default is not set.
functions.purposestringA description of the context and purpose of the function, to explain to the agent when to use it.
functions.argumentobjectA JSON object defining the input that should be passed to the function. The fields of this object are the following two parameters.
functions.argument.typestringThe type of argument the AI is passing to the function. Possible values are "string" and "object".
functions.argument.propertiesobjectA JSON object describing the argument. The value is set by the user. For example, "location" for a location-based search or "company" if searching a database of companies. This object has sub-parameters description and type to help define the property. See examples below for implementation.

Parameters for data_map​

NameTypeDescription
data_map.expressionsobject[]An array of objects that define patterns and corresponding actions.
data_map.expressions.stringstringThe actual input or value from the user or system.
data_map.expressions.patternstringA regular expression pattern to validate or match the string.
data_map.expressions.outputobjectDefines the response or action to be taken when the pattern matches.
data_map.expressions.output.responsestringA static response text or message.
data_map.expressions.output.actionobject[]A list of actions to be performed upon matching.
data_map.webhooksobject[]An array of objects that define external API calls.
data_map.webhooks.urlstringThe endpoint for the external service or API.
data_map.webhooks.headersobjectAny necessary headers for the API call.
data_map.webhooks.methodstringThe HTTP method (GET, POST, etc.) for the API call.
data_map.webhooks.outputobjectDefines the format or structure of the response from the API.
data_map.webhooks.output.actionobject[]A list of actions to be performed upon matching.
data_map.webhooks.output.responsestringA static response text or message based on the outcome of the API call.

Valid actions for output.action​

NameTypeDescription
SWMLobjectA SWML object to be executed.
saystringA message to be spoken by the AI agent.
stopbooleanWhether to stop the conversation.
toggle_functionsobject[]Whether to toggle the functions on or off.
toggle_functions.activebooleanWhether to activate or deactivate the functions. Default is true
toggle_functions.functionobject[]A list of functions to toggle.
back_to_back_functionsbooleanWhether to execute functions back to back. Default is false
set_meta_dataobjectA JSON object containing any metadata, as a key-value map.
playback_bgobjectA JSON object containing the audio file to play.
playback_bg.filestringURL or filepath of the audio file to play.
playback_bg.waitbooleanWhether to wait for the audio file to finish playing before continuing. Default is false
stop_playback_bgbooleanWhether to stop the background audio file.
user_inputstringused to inject text into the users queue as if they input the data themselves.

Parameters for languages​

If changing the language from the default, all three parameters are required for a valid entry.

NameTypeDescription
namestringName of the language. For example, "French". Optional. Default is "English".
codestringLanguage code. For example, "fr-FR". Optional. Default is "en-us".
List of language codes can be found here: Google Codes
voicestringVoice to use for the language. For example, "fr-FR-Neural2-B". Optional. Default is picked by SignalWire.
List of language voices can be found here Google Voices
fillersstring[]An array of strings to be used as fillers in the conversation. Optional. Default is not set.

Callback Request Parameters​

Request Parameters for post_prompt_url​

SignalWire will make a request to the post_prompt_url with the following parameters:

NameTypeDescription
actionstringAction that prompted this request. The value will be "post_conversation".
ai_end_datenumberTimestamp indicating when the AI session ended.
ai_session_idstringA unique identifier for the AI session.
ai_start_datenumberTimestamp indicating when the AI session started.
app_namestringName of the application that originated the request.
call_answer_datenumberTimestamp indicating when the call was answered.
call_end_datenumberTimestamp indicating when the call ended.
call_idstringID of the call.
call_logobjectThe complete log of the call, as a JSON object.
call_log.contentstringContent of the call log entry.
call_log.rolestringRole associated with the call log entry (e.g., "system", "assistant", "user").
call_start_datenumberTimestamp indicating when the call started.
caller_id_namestringName associated with the caller ID.
caller_id_numberstringNumber associated with the caller ID.
content_dispositionstringDisposition of the content.
content_typestringType of content. The value will be text/swaig.
conversation_idstringID of the conversation.
post_prompt_dataobjectThe answer from the AI agent to the post_prompt. The object contains the three following fields.
post_prompt_data.parsedobjectIf a JSON object is detected within the answer, it is parsed and provided here.
post_prompt_data.rawstringThe raw data answer from the AI agent.
post_prompt_data.substitutedstringThe answer from the AI agent, excluding any JSON.
project_idstringID of the project.
space_idstringID of the space.
SWMLVarsobjectA collection of variables related to SWML.
swaig_logobjectA log related to SWAIG functions.
total_input_tokensnumberRepresents the total number of input tokens.
total_output_tokensnumberRepresents the total number of output tokens.
versionstringVersion number.

Post Prompt Callback Request Example​

Below is a json example of the callback request that is sent to the post_prompt_url:

{
   "total_output_tokens": 119,
   "caller_id_name": "[CALLER_NAME]",
   "SWMLVars": {
      "ai_result" : "success",
      "answer_result" : "success"
   },
   "call_start_date": 1694541295773508,
   "project_id": "[PROJECT_ID]",
   "call_log": [
      {
         "content": "[AI INITIAL PROMPT/INSTRUCTIONS]",
         "role": "system"
      },
      {

         "content": "[AI RESPONSE]",
         "role": "assistant"
      },
      {
         "content": "[USER RESPONSE]",
         "role": "user"
      }
   ],
   "ai_start_date": 1694541297950440,
   "call_answer_date": 1694541296799504,
   "version": "2.0",
   "content_disposition": "Conversation Log",
   "conversation_id": "[CONVERSATION_ID]",
   "space_id": "[SPACE_ID]",
   "app_name": "swml app",
   "swaig_log": [
      {
         "post_data": {
            "content_disposition": "SWAIG Function",
            "conversation_id": "[CONVERSATION_ID]",
            "space_id": "[SPACE_ID]",
            "meta_data_token": "[META_DATA_TOKEN]",
            "app_name": "swml app",
            "meta_data": {},
            "argument": {
               "raw": "{\n  \"target\": \"[TRANSFER_TARGET]\"\n}",
               "substituted": "",
               "parsed": [
                  {
                     "target": "[TRANSFER_TARGET]"
                  }
               ]
            },
            "call_id": "[CALL_ID]",
            "content_type": "text/swaig",
            "ai_session_id": "[AI_SESSION_ID]",
            "caller_id_num": "[CALLER_NUMBER]",
            "caller_id_name": "[CALLER_NAME]",
            "project_id": "[PROJECT_ID]",
            "purpose": "Use to transfer to a target",
            "argument_desc": {
               "type": "object",
               "properties": {
                  "target": {
                     "description": "the target to transfer to",
                     "type": "string"
                  }
               }
            },
            "function": "transfer",
            "version": "2.0"
         },
         "command_name": "transfer",
         "epoch_time": 1694541334,
         "command_arg": "{\n  \"target\": \"[TRANSFER_TARGET]\"\n}",
         "url": "https://example.com/here",
         "post_response": {
            "action": [
               {
                  "say": "This is a say message!"
               },
               {
                  "SWML": {
                     "sections": {
                        "main": [
                           {
                              "connect": {
                                 "to": "+1XXXXXXXXXX"
                              }
                           }
                        ]
                     },
                     "version": "1.0.0"
                  }
               },
               {
                  "stop": "true"
               }
            ],
            "response": "transferred to [TRANSFER_TARGET], the call has ended"
         }
      }
   ],
   "total_input_tokens": 5627,
   "caller_id_number": "[CALLER_NUMBER]",
   "call_id": "[CALL_ID]",
   "call_end_date": 1694541335435503,
   "content_type": "text/swaig",
   "action": "post_conversation",
   "post_prompt_data": {
      "substituted": "[SUMMARY_MESSAGE_PLACEHOLDER]",
      "parsed": [],
      "raw": "[SUMMARY_MESSAGE_PLACEHOLDER]"
   },
   "ai_end_date": 1694541335425164,
   "ai_session_id": "[AI_SESSION_ID]"
}

Responding to Post Prompt Requests​

The response to the callback request should be a JSON object with the following parameters:

{
  "response": "ok"
}

Request Parameters for web_hook_url​

SignalWire will make a request to the web_hook_url of a SWAIG function with the following parameters:

NameTypeDescription
content_typestringType of content. The value will be Conversation/SWAIG-function.
app_namestringName of the application that originated the request.
functionstringName of the function that was invoked.
meta_dataobjectA JSON object containing any user metadata, as a key-value map.
call_logobjectThe complete log of the call, as a JSON object.
SWMLVarsobjectA collection of variables related to SWML.
purposestringThe purpose of the function being invoked. The value will be the functions.purpose value you provided in the SWAIG parameters.
argument_descstringThe description of the argument being passed. This value comes from the argument you provided in the SWAIG parameters.
argumentobjectThe argument the AI agent is providing to the function. The object contains the three following fields.
argument.parsedobjectIf a JSON object is detected within the argument, it is parsed and provided here.
argument.rawstringThe raw argument provided by the AI agent.
argument.substitutedstringThe argument provided by the AI agent, excluding any JSON.
versionstringVersion number.

Webhook Request Example​

Below is a json example of the callback request that is sent to the web_hook_url:

{
  "content_type": "Conversation/SWAIG-function",
  "call_log": [
    {
      "role": "system",
      "content": "[AI INITIAL PROMPT/INSTRUCTIONS]"
    }
  ],
  "SWMLVars": {
    "answer_result": "success"
  },
  "argument": {
    "substituted": "",
    "parsed": [
      {
        "location": "[USER_INPUT]"
      }
    ],
    "raw": "{\"location\": \"[USER_INPUT]\"\n}"
  },
  "app_name": "swml app",
  "function": "get_weather",
  "version": "2.0",
  "argument_desc": "The location to get the weather for",
  "purpose": "Get weather for a specific location"
}

Responding to Webhook Requests​

The response should be in JSON format, the primary key that the AI will look for is the response key. The value associated with this key will be the content or instruction that the AI will process.

{
  "response": "The weather in [CITY] is [WEATHER]."
}

Variables​

  • ai_result: (out) success | failed
  • return_value: (out) success | failed

Examples​

A Voicemail Bot​

---
version: 1.0.0
sections:
  main:
    - label: ai
      ai:
        voice: en-US-Neural2-F
        post_prompt_url: https://example.com/my-api
        prompt:
          confidence: 0.6
          temperature: 0.2
          text: |
            You are Franklin's assistant, and your job is to collect messages for him over the phone.
            You can reassure that Franklin will get in touch as soon as possible.
            Collect the user's name and number if you do not already know it from the caller id.
            Start by presenting yourself, then let the contact know that Franklin is not available, then offer to collect a message.
            After collecting the message, do not wait for the user to end the conversation: say good bye and hang up the call.
        post_prompt:
          text: |
            Summarize the message as a valid anonymous json object by filling the upper case placeholders in this template:
            { "contact_info": { "name": "CONTACT_NAME", "number": "CONTACT_PHONE" }, "message": "MESSAGE" }
          result:
            case:
              failed:
                - goto:
                    label: retry_ai
                    max: 1

Declaring functions​

version: 1.0.0
sections:
  main:
    - ai:
        post_prompt_url: "https://example.com/my-api"
        prompt:
          text: |
            You are a helpful assistant that can provide information to users about a destination. You can use the appropriate function to get the phone number, address, or weather information.
        post_prompt:
          text: "Summarize the conversation."
        SWAIG:
          includes:
            - functions:
                - get_phone_number
                - get_address
              url: https://example.com/functions
              user: me
              pass: secret
          defaults:
            web_hook_url: https://example.com/my-webhook
            web_hook_auth_user: me
            web_hook_auth_pass: secret
          functions:
            - function: get_weather
              purpose: To determine what the current weather is in a provided location.
              argument:
                properties:
                  location:
                    type: string
                    description: The name of the city to find the weather from.
                type: object

data_map Examples​

Webhook Example for Transfer​

In this example, the system aims to transfer a call based on a provided target by querying an external service via a webhook. This allows for dynamic decision-making based on real-time or externally managed data.

Example:

sections:
  main:
  - ai:
      prompt:
        text: Your name is Frank. You help transfer to the right department.
      post_prompt: Summarize the call as a json object
      post_prompt_url: https://example.com/here
      SWAIG:
        functions:
        - function: transfer
          purpose: use to transfer to a target
          argument:
            type: object
            properties:
              target:
                type: string
                description: The target to transfer to
          data_map:
            webhooks:
            - url: https://example.com/v1/destination?name=${enc:args.target}
              headers:
                X-Api-Key: api-token-here
              method: GET
              output:
                response: Transferring to ${input.args.target}.
                action:
                - SWML:
                    version: 1.0.0
                    sections:
                      main:
                      - connect:
                          to: "${destination_number}"
                  stop: true
Responding to the Webhook request​

You can learn more on how to respond to a webhook request here.

In this example we will pass an additional parameter destination_number which will be used to connect the call to the destination number.

{
  "response": "Transferring to ${input.args.target}.",
  "destination_number": "+1XXXXXXXXXX"
}
Expressions Example for Transfer:​

In this example, the AI aims to transfer a call based on a provided target name. Instead of querying an external service (like the webhook scenario), this approach utilizes a predefined list meta_data to determine the destination of the transfer.

The meta_data.table serves as a directory, linking user-friendly targets (like "support" or "sales") to distinct phone numbers. When a user provides input, the system uses the pattern \\w+ to see if the input corresponds with any keys in this table. If there's a match, the system retrieves the associated phone number and initiates the call transfer to that destination. However, if the input doesn't match any key, the system resorts to the fallback expression ".*" to capture all other inputs, subsequently informing the user that the intended transfer was unsuccessful.

Example:

sections:
  main:
  - ai:
      prompt:
        text: Your name is Frank. You help transfer to the right department.
      post_prompt: Summarize the call as a json object
      post_prompt_url: https://example.com/post_prompt_url
      SWAIG:
        functions:
        - function: transfer
          purpose: use to transfer to a target
          data_map:
            expressions:
            - pattern: "\\w+"
              string: "${meta_data.table.${lc:args.target}}"
              output:
                action:
                - say: Please stand by while I connect your call.
                - SWML:
                    version: 1.0.0
                    sections:
                      main:
                      - connect:
                          from: "+1XXXXXXXXXX"
                          to: "${meta_data.table.${lc:args.target}}"
                - stop: 'true'
                response: transferred, the call has ended.
            - string: "${args.target}"
              pattern: ".*"
              output:
                response: I'm sorry, I was unable to transfer your call to ${input.args.target}.
          argument:
            type: object
            properties:
              target:
                description: the target to transfer to
                type: string
          meta_data:
            table:
              support: "+1XXXXXXXXXX"
              sales: "+1YYYYYYYYYY"

toggle_functions Example​

In this example, the transfer function is toggled off from the start. The AI agent will toggle this function on after the get_joke function is called. This creates a scenario where a user can only be transferred after hearing a joke from the AI. The AI agent will then match the transfer destination based on the user's input, with the meta_data table serving as a directory for the transfer destinations. If no match is found, the AI agent will fall back to the ".*" expression, which will inform the user that the transfer was unsuccessful and requires a valid input.

transfer Function​

      SWAIG:
        functions:
        - function: transfer
          active: 'false'
          purpose: use to transfer to a target
          argument:
            type: object
            properties:
              target:
                description: the target to transfer to
                type: string
          data_map:
            expressions:
            - pattern: "\\w+"
              string: "${meta_data.table.${lc:args.target}}"
              output:
                action:
                - say: Please stand by while I connect your call.
                - SWML:
                    version: 1.0.0
                    sections:
                      main:
                      - connect:
                          to: "${meta_data.table.${lc:args.target}}"
                - stop: 'true'
                response: transferred, the call has ended.
            - string: "${args.target}"
              pattern: ".*"
              output:
                response: I'm sorry, I was unable to transfer your call to ${input.args.target}.
          meta_data:
            table:
              support: "+1XXXXXXXXXX"
              sales: "+1YYYYYYYYYY"

We set the function to being toggled off with the following line:

active: 'false'

get_joke Function​

        - function: get_joke
          purpose: use to get a joke
          data_map:
            webhooks:
            - url: https://example.com/v1/${args.type}
              headers:
                X-Api-Key: api-token-here
              method: GET
              output:
                response: 'Tell the user: ${array[0].joke}.' # array[0] is the first joke in the array
                action:
                - toggle_functions:
                    active: 'true'
                    function: transfer

In the above function, we set the transfer function to be toggled on. Now when a user asks to be transferred, the AI agent will now be able to do so because the transfer function is toggled on.

- toggle_functions:
    active: 'true'
    function: transfer

Full example​

sections:
  main:
  - ai:
      prompt:
        text: Your name is Frank. You help transfer to the right department. Use the 'get_joke' function to get a joke.
      post_prompt: Summarize the call as a json object
      post_prompt_url: https://example.com/post_prompt_url
      SWAIG:
        functions:
        - function: transfer
          active: 'false'
          purpose: use to transfer to a target
          argument:
            type: object
            properties:
              target:
                description: the target to transfer to
                type: string
          data_map:
            expressions:
            - pattern: "\\w+"
              string: "${meta_data.table.${lc:args.target}}"
              output:
                action:
                - say: Please stand by while I connect your call.
                - SWML:
                    version: 1.0.0
                    sections:
                      main:
                      - connect:
                          to: "${meta_data.table.${lc:args.target}}"
                - stop: 'true'
                response: transferred, the call has ended.
            - string: "${args.target}"
              pattern: ".*"
              output:
                response: I'm sorry, I was unable to transfer your call to ${input.args.target}.
          meta_data:
            table:
              support: "+1XXXXXXXXXX"
              sales: "+1YYYYYYYYYY" # These are the numbers that will be used to transfer the call


        - function: get_joke
          purpose: used to get a joke
          argument:
            type: object
            properties:
              type:
                type: string
                description: must either be 'jokes' or 'dadjokes'
          data_map:
            webhooks:
            - url: https://example.com/v1/${args.type}
              headers:
                X-Api-Key: API TOKEN HERE
              method: GET
              output:
                response: 'Tell the user: ${array[0].joke}.' # array[0] is the first joke in the array
                action:
                  - toggle_functions:
                      - active: 'true'
                        function: transfer