There have been no further issues and the episode releaser system is fully caught up and running normally.
At ~6:28AM (EST) we began receiving alert notifications from our monitoring services. An investigation lead to discovery of a failed service on a primary node in our caching layer that didn't properly failover to backup nodes. This triggered high load on backend database systems causing severe performance issues throughout multiple parts of our systems.
The failed service was manually failed over to backup nodes at 6:41AM and we began to see immediate recovery of our systems. By ~6:44AM all services were fully restored and functioning normally.
We are continuing to monitor the situation. The issue did cause the system responsible for releasing episodes to get backed up and it is now in the process of catching up. We'll post an update when it's fully caught back up.
We’ll find your subscription and send you a link to login to manage your preferences.
We’ve found your existing subscription and have emailed you a secure link to manage your preferences.
We’ll use your email to save your preferences so you can update them later.
Subscribe to other services using the bell icon on the subscribe button on the status page.
You’ll no long receive any status updates from Liberated Syndication, are you sure?
{{ error }}
We’ll no longer send you any status updates about Liberated Syndication.