Panopto Service Disruption 22nd Dec, 2020

On Dec 22nd 2020 between 22:05 and 22:25PM GMT, the Panopto EU Cloud experienced a site outage.

Panopto’s cloud operations team has resolved the issue.

The cause of the incident was a misconfiguration on a database server. During Panopto’s most recent downtime release on Dec 19, they updated the configuration of their database servers to better utilize physical memory. The new configuration contained an error that caused the database server to reach an out of memory condition. Panopto resolved the outage by fixing the misconfiguration on the server that ran out of memory. To prevent a recurrence of this issue, their engineering team audited and updated the memory configuration on all database servers.

Panopto Service Disruption 20-10-20

Incident Status

Service Disruption

Components

Website, Video playback, Search, Recording / Uploading, Processing / Encoding, Integrations

Locations

Europe (EU)


October 20, 2020 11:41AM UTC
[Investigating] We are investigating increased error rates on requests to the Panopto EU Cloud.
October 20, 2020 12:15PM UTC
[Investigating] We are continuing to investigate the cause of increased error rates on requests on the EU cloud.
October 20, 2020 12:31PM UTC
[Monitoring] The source of the increased error rates has been mitigated. Our cloud operations team is monitoring performance metrics to ensure the issue is resolved.