Warning: Context Object exceeds the limit for virtual assistant

Announcements
, , ,
1

Explanation:

  1. The purpose of sending these emails is only to inform the users about the upcoming changes. No changes are made to the platform and everything continues to work without any issues now
  2. In the future, Kore.ai will start enforcing the limit of 1024 KB on the context size. This is required to ensure that the bots respond with minimal latency. Large context objects result in slower dialog execution as we have to internally process the full context object for every state change of the dialog
  3. We have not yet decided on when would we start implementing the restrictions. Whenever we choose to implement, we will give adequate time for our customers to implement the required changes. There will be a minimum of 6-8 weeks after we announce a timeline.
  4. We are actively collecting feedback from customers. This feedback will be used for the following
  5. Based on the use cases where the customers are storing large about of data, we will come up with recommendations on how to reduce the context size
  6. We might reconsider the 1024 KB size and might choose to increase it as appropriate
  7. Meanwhile, the customers can do one of the following
  8. Review the bot definition to see
    1. If any service call request payloads can be updated to only retrieve limited information as necessary for the bot execution
    2. If any custom objects created in the context object can be optimized to store only the relevant data
  9. Explore the possibility with the backend systems to return only the required data to the platform

Example:

image
image959Ă—711 29.9 KB

Note -Applicable for On-Premise Customers:
Customers can change the config according to their use case.

The context limit can be changed by overriding the threshold limit in config/KoreConfig.json by adding the bellow block.

“contextLimits”:{
“limit”: 1024
}

Note:
No impact on existing functionality.
No impact on end-user experience.

2

A few useful tips for managing context size -

  1. Any third-party API calls that send a huge payload in response may cause the context object to exceed the desired limit. It is highly recommended that after getting whatever is required from the response, the context.<serviceNodeName> is set to undefined in one of the following script nodes Example - context.<serviceNodeName> = undefined;
  2. Using TTL is highly recommended for any context variable (it can be under any sub-object - BotUserSession, UserContext, BotContext, etc.).
    a. Case Study - In a few recent cases it was observed that for a few users, the UserContext was huge. A VA was found to be storing the accounts owned by a user in the UserContext. Over 200 accounts were found to be loaded.
  3. Any custom context variable should be kept in the context only till the time it is required. We recommend reviewing and clearing/purging context variables either through TTL, at the end of the dialog task within a script node, or in a dialog task that may have been configured in the end-of-conversation event.
  4. It is recommended to keep the dialog tasks minimal. Very long dialog tasks tend to accumulate a lot of information in the context. At the end of the dialog task, the dialog context is cleared.
2 Likes