Summary

This article outlines the procedure to alert (“wake up”) engineers in the event of a trending SERIOUS issue that's WIDESPREAD in our mobile/web app or coach dashboard.

Samples of Serious and Widespread Issues:

• App Outage
• App Crashing
• App Freezing
• Security breaches

Environment

Zendesk (JIRA app integration)

Resolution Process

Please utilize this procedure when there is a Serious and Widespread issue occurring between the following hours:

Monday - Friday, 8pm - 8am PT

Saturday - Sunday, 24 Hours 

To alert an engineer of a widespread issue, you’ll need to create a JIRA ticket of a specific priority.

Open the JIRA app in Zendesk on an affected ticket, then click “Create Issue”

Set the following fields for the JIRA Ticket

• Project: Bug Triage
• Issue Type: Bug 
• Put a thorough Summary & Description
  • For Summary, begin with “Oncall:”
• Click “Configure fields” in the top right, then check “Priority”
• Priority: Production Blocker 
• Click “Create Issue”

The ticket should look like this:

Note the JIRA ticket number for the next step.

Respond to Tickets

It’s likely that you’ll see multiple tickets regarding this same issue. For each related ticket, you’ll want to:

1. Link the JIRA ticket to the Zendesk Ticket
2. Respond letting them know it’s a known issue under investigation.
3. Keep the ticket pending until a resolution is in place.
4. Once a resolution is in place, reply to the tickets to confirm and relay any steps they need to take to restore the app on their end.

Cause

This process will send a notification to our On-Call System (Opsgenie) and based on whichever engineer is on call, it will “page” them to look at the issue.