When things go wrong. The story behind the Thumsters Android app crash.

What a week. We started out this week working extremely hard to get a new version of Thumsters app out for both Android and IOS. This app release was to be a major update that would stabilize our subscription system and address a number of annoying issues we had been seeing on both platforms where due to the way the App Store and Play Store deal with subscriptions, some users were getting false notifications saying they were not subscribed. All was going well and our engineering team believed they were ready and finished up all their internal testing using the debug versions of the app had passed. There was much celebration as we released the apps to the stores, to wait on their rollout to the public. Due to COVID-19, the app stores are understandably taking much longer to sign off apps before they are released.


Little did we know that our celebration would be short lived.

Both app stores deal with releases quite differently. Apple physically test the apps and if they feel they are not up to scratch, e.g. crash on start up, they will reject it, but on the other hand, Play Store do not. Play Store just look at the code for anything that would violate their terms but that is about it. The app took around 48 hours to release to both stores. It as at this point we instantly realized their was a problem on the Android edition of Thumsters as we instantly saw a spike in the crashes. We quickly realized that our debug versions of the app were performing differently to the release version the public was seeing and we identified the problem almost instantly. Our hearts sank. We were kicking ourselves as to how this major issue had made it into production?


The team fixed the issue and the had fixed version, 1.6.3, ready to release again almost instantly, however the roll-out of the release which would usually only take around 2 hours, as explained earlier was heavily delayed and ended up taking 40 hours for the Play Store to sign off and release. The fix was done, but the damage was also done, and our faithful Android users had already downloaded the faulty app update which was crashing instantly on startup. Our usual 5 star reviews were now coming in as 1 star reviews. This is not the service we promise. It is not the experience we wanted to have for our users. This was our lowest point.

Our amazing team understanding the massive inconvenience that had been caused to almost 50% of our users didn't panic, they set a game plan, lead with listening and empathy, and responded with honesty and integrity. They worked around the clock communicating the issue to users who reached out to support with their frustrations. It wasn't the users fault, it was ours. We were the ones that had let them down. Although we would love to blame COVID-19, it was not cause of the issue, it just caused a delays that compounded the issue.

We are now back to normal operations. We are certain we have lost some Android customers who dismissed the app as soon as it crashed, and we may never see them come back, but on the flip-side we are so thankful for all our loyal Android users who have stuck by us through this difficult time with patience and understanding, and we hope that we will never face a similar problem again. To you we send you a massive Thumbs up!

Thumbs up!