Worldwide Internet Outage Caused By Single Configuration Error

The June 8th Internet outage which affected websites in dozens of countries across the Americas, Europe and Asia, as well as South Africa has been traced to a single configuration error at a little known but important IT infrastructure company, Fastly

Amazon, eBay and Boots were among retailers whose websites disappeared during the outage. Amazon and Boots alone report a combined £25bn in annual sales in the UK, meaning they would typically earn nearly £3m between them in the course of an average hour. Payment provider PayPal which processed $936bn of transactions in 2020, approximately $106m for every hour, was also temporarily unavailable.

Fastly said it had identified a service configuration that triggered disruptions across its servers and has now disabled that configuration.

The network run by Fastly had the outage for more than an hour and in most cases was occurring mid-morning London time. Investigation reveals the incident was caused by a cloud configuration fault affecting governments and businesses around the world in sectors ranging from media to online retail and telecoms. “We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. We detected the disruption within one minute, then identified and isolated the cause, and disabled the configuration.... Within 49 minutes, 95% of our network was operating as normal”, said Fastly’s SVP  of Engineering & Infrastructure in a statement. 

The basic issue appears to be that Fastly had mitakenly taken down its own network with a bad software update, resulting in a blockage affecting millions of dollars in revenue for numerous corporations around the world.

All UK government websites using the gov.uk domain were swept up in the outage, which temporarily knocked out crucial services such as the online system by which taxpayers can file annual returns with HM Revenue and Customs. The British  Government said it was investigating reports that users were unable to book Covid-19 tests online as a “matter of urgency”.
Visitors to the official website of the White House were also greeted with a message likely to have been seen by hundreds of millions of internet users: “Error 503 service unavailable”.

According to analysis of data from Google’s AdSense platform, outlets lost approximately $300,000 in revenue during the period, as they missed out on clicks that would usually translate into payments from advertisers. The calculation, made by marketing agency connective3, covers lost revenue at affected advertising-reliant media sites including the The BBC, The Guardian New York Times, the Financial Times, CNN, Al Jazeera, Bloomberg, the Independent and the Evening Standard., were all among the websites that crashed. 

Fastly is a content delivery network (CDN), which maintains a network of servers that transfer content quickly from websites to users. It provides a layer of support between Internet companies and customers trying to access the various online platforms it services, when it goes down, access to those platforms can be blocked entirely. 

One of the reasons that the outage was so wide is that cloud computing service companies like Fastly are consolidating, leaving websites dependent on a shrinking number of providers, a critical vulnerability demonstrated recently by a similar large scale outage affecting Amazon Web Services in December 2020.

Fastly:      CNN:     Vox:     Computing:   Guardian:

You Might Also Read:

SANS Institute book: Practical Guide To Security In The AWS Cloud:

 

« Beware Of Credentials Phishing
AI Dominance On The Battlefield »

CyberSecurity Jobsite
Check Point

Directory of Suppliers

DigitalStakeout

DigitalStakeout

DigitalStakeout enables cyber security professionals to reduce cyber risk to their organization with proactive security solutions, providing immediate improvement in security posture and ROI.

Alvacomm

Alvacomm

Alvacomm offers holistic VIP cybersecurity services, providing comprehensive protection against cyber threats. Our solutions include risk assessment, threat detection, incident response.

Tines

Tines

The Tines security automation platform helps security teams automate manual tasks, making them more effective and efficient.

Directory of Cyber Security Suppliers

Directory of Cyber Security Suppliers

Our Supplier Directory lists 8,000+ specialist cyber security service providers in 128 countries worldwide. IS YOUR ORGANISATION LISTED?

MIRACL

MIRACL

MIRACL provides the world’s only single step Multi-Factor Authentication (MFA) which can replace passwords on 100% of mobiles, desktops or even Smart TVs.

Center for a New American Security (CNAS)

Center for a New American Security (CNAS)

CNAS is the nation's leading research institution focused on defense and national security policy. Cyber security issues are an intrinsic element of the national security debate.

HANDD Business Solutions

HANDD Business Solutions

HANDD are independent specialists in data protection with expertise at every stage of the Protect, Detect and Respond cycle, from consultancy and design, right through to installation.

Green Hills Software

Green Hills Software

Green Hills Software is the largest independent vendor of embedded secure software solutions for applications including the Internet of Things.

Skkynet Cloud Systems

Skkynet Cloud Systems

Skkynet is a leader in real-time data systems for the secure management and control of industrial processes (SCADA) and embedded devices (M2M).

Atomicorp

Atomicorp

Atomicorp, the leader in Secure Linux, is a developer of solutions for the protection and support of cloud, virtual, shared, and dedicated web hosting environments.

XTN Cognitive Security

XTN Cognitive Security

XTN is focused on the development of security, Fraud and Mobile Threat Prevention advanced behaviour-based solutions.

Improsec

Improsec

Improsec is a fully independent Cyber Security advisory company - we provide knowledge, experience and both strategic and deep technical expertise to our clients.

X-Ways Software Technology

X-Ways Software Technology

X-Ways provide software for computer forensics, electronic discovery, data recovery, low-level data processing, and IT security.

InsightCyber

InsightCyber

InsightCyber is on a mission to keep the world’s critical infrastructure, supply chains, and manufacturing operations cyber-safe, helping to prevent attacks that can have catastrophic impacts.

Netragard

Netragard

Netragard has an established reputation for providing high-quality offensive and defensive security services.

Netsurion

Netsurion

Netsurion powers secure and agile networks for highly distributed and small-to-medium enterprises and the IT providers that serve them.

FortiGuard Labs

FortiGuard Labs

FortiGuard Labs is the threat intelligence and research organization at Fortinet. Its mission is to provide Fortinet customers with the industry’s best threat intelligence.

Security Compliance Associates (SCA)

Security Compliance Associates (SCA)

The sole focus of SCA is safeguarding critical information and complying with information security regulations.

Thoropass

Thoropass

Thoropass (formerly Laika) helps you get and stay compliant with smart software and expert services.

Tyto Athene

Tyto Athene

At Tyto Athene, we harness the power of technology to provide solutions that shape the future.

Attaxion

Attaxion

Attaxion is an External Attack Surface Management (EASM) Platform. We offer attack surface management solutions with #1 asset coverage and laser-focused, actionable intelligence.