Salesforce cloud outage caused by security change

By

Goes public with post-mortem.

A widespread Salesforce outage last week was caused by the rollout of a new permissions policy as part of defence-in-depth efforts that mistakenly locked users out of their accounts.

Salesforce cloud outage caused by security change

The four-hour outage on September 21, took out a number of Salesforce 'Clouds', along with Tableau and MuleSoft.

In a root cause analysis, published late last week, Salesforce revealed the outage was caused by an access permissions change.

"The change had unintended consequences," Salesforce wrote.

"While it was designed to add defence-in-depth, it inadvertently blocked access to other legitimate and necessary resources beyond its intended scope.

"The end result was a breakdown in communication between our services due to a lack of access permissions, causing failures within our systems.

"This restricted some of our customers from logging in and using the services."

The company explained that its "standard operating procedure mandates ongoing reviews and updates to security controls.

The change, the company admitted in its analysis, “was not identified in validation testing”, because whatever it changed couldn’t be pre-tested: “the standard change deployment pipeline could not be used to deploy this change.”

That meant a particular characteristic of the change wasn’t caught: that it was “applied at a level that impacted multiple systems”. 

Had the change fitted within Salesforce’s “automated deployment process … the change would only have affected a small number of services”.

Salesforce said this will be addressed with a new deployment and testing pipeline later this year.

Got a news tip for our journalists? Share it with us anonymously here.
Copyright © iTnews.com.au . All rights reserved.
Tags:

Most Read Articles

Macquarie Uni to spend up to $700m on 10-year digital transformation

Macquarie Uni to spend up to $700m on 10-year digital transformation

UniSuper's Google cloud deletion traced to "blank parameter" in setup

UniSuper's Google cloud deletion traced to "blank parameter" in setup

Australian Federal Police uses cloud, SASE to upgrade reach and capability

Australian Federal Police uses cloud, SASE to upgrade reach and capability

Western Sydney Uni discloses January "IT network" breach

Western Sydney Uni discloses January "IT network" breach

Log In

  |  Forgot your password?