Issue:
On July 10th 2020, we experienced a degraded performance issue with the V2 Recommendations where some of our clients experienced a slowness in the API response time or a failure of API calls.
Root Cause:
Our data nodes experienced a period where API calls exceeded data size limits, causing server nodes to restart automatically, with some services within the cluster becoming inaccessible.
Mitigation Steps:
An approach is being investigated to limit the size of individual calls to prevent this from happening in the future.