21 February 2020 Outage

Dan Uncategorized

Dear Friends of Adventure Box,

On Friday 21 February evening, we experienced our first serious outage in over a year; the problem was resolved by Saturday morning. This outage was triggered when we upgraded our database, which should have caused no outage at all. However, database drivers in our servers failed to reconnect to the new database until all our servers were rebooted. The system was then fine.

The Adventure Box system is currently stable, handling a tremendous amount of traffic, and we have identified three fixes, any of which will prevent this situation from happening in the future. Those fixes will be put in place this week after thorough testing.

Our goal is to provide uninterrupted service as we add new features and fix bugs; our development group is improving its journey toward “continuous integration, continuous deployment,” with build, test and deployment automation. In addition, we have made many changes over the past year toward high-availability. This experience will motivate greater attention to load-testing and failure recovery processes.

Thank you for your ongoing support of Adventure Box. We are on a journey to become the preferred platform to play, make, and share games. Sometimes there are bumps in the road, and we will learn from each one.

Dan Greening, COO and VP Engineering
Adventure Box Technology AB

Share: