Info & Sign Up | Log In | Community & Help

Wed 18 Feb 2009

Stubmatic Downtime

Firstly we would like to apologise to all our customers for the two periods of downtime experienced over the last 24 hours. Our hosting partner is confident they have resolved the problem and all services should be back up.

Yesterday we had a failure at our London datacenter which resulted in the service going down shortly at 15:00 GMT. Unfortunately this highlighted an issue and single point of failure that we had overseen with our disaster recovery strategy and immediately contacted our hosting partner as to how we could avoid the problem in the future.

We decided to replicate our entire infrastructure in London creating a stand-by setup in Dallas where by we could fail-over silently to this setup in the result of a failure at our London datacenter in the future. Unfortunately as part of the database replication setup our database became read-only which resulted in the Stubmatic.com service becoming unavailable at 05:00 GMT this morning.

We have now resolved this issue and our replication should now be stable and operational.

No data was lost in the incident and we now have a far more resilient infrastructure to prevent an occurance of this happening again in the future.

Best Regards,

Jon Baker

CEO Vibrant Apps Ltd

One Response to “Stubmatic Downtime”




Cameron Hacault
February 4th, 2009 at 10:19 pm

Thanks for posting. It’s good to know you found the problem and dealt with it appropriately. It’s also great to know what happened.

Thanks again,

Cam

 

Leave a Reply

Comment