How HotelTonight Prioritizes Finding and Fixing Critical Bugs While Maintaining a 1% Error Budget

CHALLENGE

HotelTonight’s previous crash reporting solution was unreliable, buggy, and bloated with unnecessary features. This created inefficiencies for their engineering team and made debugging a difficult, time-consuming process.

RESULTS

HotelTonight now has the stability score, an at-a-glance metric that immediately informs them of the health of their application. They set a clear stability target and hold their team accountable to it, which helps them deliver a reliable user experience that helps them maintain high ratings in the app store.

---

HotelTonight’s zero tolerance policy for mobile app crashes

HotelTonight is a popular mobile app that lets travelers find hotel accommodation at discounted rates. They help millions of travelers find and book great hotel deals, so any issues in the booking flow of their application are considered high priority to ensure users have a smooth experience planning their travel.

Because they generate revenue primarily through their iOS, Android, and mobile web applications, they operate with a 1% error budget with zero tolerance for overages as app crashes are directly tied to their success as a company. The error budget is a clear metric their entire engineering team is held accountable to, that determines a threshold for errors in their service at any given time.

• iOS, Android, mobile web focus
• Stability is vital as they generate revenue directly through their apps

Maintaining high app stability requires the right tooling and metrics

To help them maintain their error budget, the team at HotelTonight used an error monitoring tool for alerts to new app crashes. However, they found their previous error monitoring tool to be buggy and unreliable and also filled with unnecessary features that caused a negative user experience, so they knew they needed to replace it.

“The stability score is extremely important at HotelTonight because it is a reliable, at-a-glance metric that immediately tells us the health of the application.” Doug Suriano, iOS Manager.

“As a mobile-app only business, crash rates are enormously important to us because when our app crashes, so does our means of doing business,” said Doug Suriano, lead iOS engineer at HotelTonight. “We needed a more dependable and simple solution backed by a team we could trust to help maintain our 1 percent error budget threshold.”

Maintaining their highly stable applications became a challenge without the proper metrics to monitor and tools to help them achieve their goals.

Bugsnag gives them a definitive metric for the health of their applications

HotelTonight chose Bugsnag as their stability monitoring solution for their full stack, including mobile and backend services. Bugsnag provides them with the insight they need into their application’s health, plus a superior user experience from their previous solution. It also gives both the technical and business teams visibility into the business outcomes related to buggy software.

Since moving to Bugsnag, they’ve improved many processes:

• Find and fix bugs 50-60% faster than before
• Maintain at least a 1% error budget consistently
• Save at least 40 hours per month finding and fixing bugs

  
  

“Over time, having this KPI has really motivated and incentivized our team to take a second look when coding new features, to think through potential bugs, and code defensively against them.”

— Doug Suriano, iOS Manager

The stability score, in particular, has made it possible for their team to measure their application’s stability and maintain their team’s error budget of at least 1% at all times. “The stability score is extremely important at HotelTonight because it is a reliable, at-a-glance metric that immediately tells us the health of the application,” says Doug Suriano, iOS Manager.

HotelTonight chose to set stability as a KPI for their team due to its direct impact on revenue and the business bottom line. With Bugsnag, they are able to track this metric and prioritize when to fix bugs versus build features.

“Over time, having this KPI has really motivated and incentivized our team to take a second look when coding new features, to think through potential bugs, and code defensively against them,” explains Doug.

• Stability score is an at-a-glance metric for application health
• Stability is a KPI for their engineering team

HotelTonight prioritizes critical bugs with Bugsnag to maintain high application stability

Whenever the team at HotelTonight works on debugging to maintain their stability targets, they are able to use Bugsnag’s prioritization features to focus on bugs that appear in business-critical sections of code, like the booking section of their application. Bugs that appear in these sections of their code get a priority fix which are scheduled in JIRA through the two-way sync integration. As Doug explains, “it’s not just about fixing crashes; it’s about fixing crashes that have the biggest impact on your customers.”

With Bugsnag’s alerts and error reports, the team at HotelTonight are able to resolve bugs 50-60% faster than before and save 10 hours of work each week.