Our entire data sync went down for 12 hours because someone changed a single API permission in Salesforce and didn't tell us.

3 comments

3 Comments

Shouldn't your system have checks for that kind of change? A single permission shouldn't take down everything for half a day. Maybe the real issue is a lack of built-in alerts.

john_fisher17d ago

We added alerts after our last outage, fixed it fast.

jakewhite17d ago

Seriously, that's the whole point. Alerts are just noise if the system lets you break it that easy in the first place. You need to stop the bad change from going through, not just yell about it after. Building in those blocks is basic stuff, not some fancy feature.