Worldwide Internet Outage Caused By Single Configuration Error

The June 8th Internet outage which affected websites in dozens of countries across the Americas, Europe and Asia, as well as South Africa has been traced to a single configuration error at a little known but important IT infrastructure company, Fastly

Amazon, eBay and Boots were among retailers whose websites disappeared during the outage. Amazon and Boots alone report a combined £25bn in annual sales in the UK, meaning they would typically earn nearly £3m between them in the course of an average hour. Payment provider PayPal which processed $936bn of transactions in 2020, approximately $106m for every hour, was also temporarily unavailable.

Fastly said it had identified a service configuration that triggered disruptions across its servers and has now disabled that configuration.

The network run by Fastly had the outage for more than an hour and in most cases was occurring mid-morning London time. Investigation reveals the incident was caused by a cloud configuration fault affecting governments and businesses around the world in sectors ranging from media to online retail and telecoms. “We experienced a global outage due to an undiscovered software bug that surfaced on June 8 when it was triggered by a valid customer configuration change. We detected the disruption within one minute, then identified and isolated the cause, and disabled the configuration.... Within 49 minutes, 95% of our network was operating as normal”, said Fastly’s SVP  of Engineering & Infrastructure in a statement. 

The basic issue appears to be that Fastly had mitakenly taken down its own network with a bad software update, resulting in a blockage affecting millions of dollars in revenue for numerous corporations around the world.

All UK government websites using the gov.uk domain were swept up in the outage, which temporarily knocked out crucial services such as the online system by which taxpayers can file annual returns with HM Revenue and Customs. The British  Government said it was investigating reports that users were unable to book Covid-19 tests online as a “matter of urgency”.
Visitors to the official website of the White House were also greeted with a message likely to have been seen by hundreds of millions of internet users: “Error 503 service unavailable”.

According to analysis of data from Google’s AdSense platform, outlets lost approximately $300,000 in revenue during the period, as they missed out on clicks that would usually translate into payments from advertisers. The calculation, made by marketing agency connective3, covers lost revenue at affected advertising-reliant media sites including the The BBC, The Guardian New York Times, the Financial Times, CNN, Al Jazeera, Bloomberg, the Independent and the Evening Standard., were all among the websites that crashed. 

Fastly is a content delivery network (CDN), which maintains a network of servers that transfer content quickly from websites to users. It provides a layer of support between Internet companies and customers trying to access the various online platforms it services, when it goes down, access to those platforms can be blocked entirely. 

One of the reasons that the outage was so wide is that cloud computing service companies like Fastly are consolidating, leaving websites dependent on a shrinking number of providers, a critical vulnerability demonstrated recently by a similar large scale outage affecting Amazon Web Services in December 2020.

Fastly:      CNN:     Vox:     Computing:   Guardian:

You Might Also Read:

SANS Institute book: Practical Guide To Security In The AWS Cloud:

 

« Beware Of Credentials Phishing
AI Dominance On The Battlefield »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

North Infosec Testing (North IT)

North Infosec Testing (North IT)

North IT (North Infosec Testing) are an award-winning provider of web, software, and application penetration testing.

CSI Consulting Services

CSI Consulting Services

Get Advice From The Experts: * Training * Penetration Testing * Data Governance * GDPR Compliance. Connecting you to the best in the business.

XYPRO Technology

XYPRO Technology

XYPRO is the market leader in HPE Non-Stop Security, Risk Management and Compliance.

BackupVault

BackupVault

BackupVault is a leading provider of automatic cloud backup and critical data protection against ransomware, insider attacks and hackers for businesses and organisations worldwide.

Practice Labs

Practice Labs

Practice Labs is an IT competency hub, where live-lab environments give access to real equipment for hands-on practice of essential cybersecurity skills.

Caldew Consulting

Caldew Consulting

Caldew specialise in providing information assurance and cyber security consultancy, covering the full spectrum of the security life cycle.

DKCERT

DKCERT

DKCERT (Danish Computer Security Incident Response Team) is a service of DeIC (Danish e-Infrastructure Cooperation).

Samsung Knox

Samsung Knox

Samsung Knox brings multi-layered defence-grade security to your business’s smartphones and tablets.

ATIS Systems

ATIS Systems

ATIS Systems offers first-class complete solutions for legal interception, mediation, data retention, and IT forensics.

Sertainty

Sertainty

Sertainty enables developers to mix intelligence into data files for active risk mitigation and data control. Discover the impact of Data: Empowered.

Cyber Security Forum Initiative (CSFI)

Cyber Security Forum Initiative (CSFI)

CSFI is a non-profit organization with a mission to provide Cyber Warfare awareness, guidance, and security solutions through collaboration, education, volunteer work, and training.

Truesec

Truesec

TRUESEC has an exceptional mix of IT specialists. We are true experts in cyber security, advanced IT infrastructure and secure development.

MillenniumIT ESP (MIT ESP)

MillenniumIT ESP (MIT ESP)

MillenniumIT ESP provides solutions and services around Core Infrastructure, Cloud, Cyber Security, Enterprise Applications, Intelligent Automation and Data, Smart Buildings, and Managed Services.

Xperience

Xperience

Xperience solves our clients’ toughest challenges by delivering business efficiency through digital transformation solutions across cloud, managed IT, CRM and ERP.

Comparitech

Comparitech

Comparitech strives to promote cyber security and privacy for all. We are committed to providing detailed information to help our readers become more cyber secure and cyber aware.

LBMC

LBMC

LBMC is a professional services solutions provider in accounting and finance, human resources, technology, risk and information security, and wealth advisory services.

Hayes Connor Solicitors

Hayes Connor Solicitors

Hayes Connor Solicitors is a specialist data breach and cybercrime law firm. We act for clients on individual data breaches and also where a group has been compromised as part of a targeted attack.

Bright Pixel Capital

Bright Pixel Capital

Bright Pixel Capital is a venture capital company with a focus on Cybersecurity, Retail Technologies, Digital Infrastructure and Emerging Technologies.

Cyber Risk International

Cyber Risk International

Cyber Risk International offer CyberPrism, a B2B SaaS solution that empowers businesses to perform a self-assessment of their cyber security program.

PeoplActive

PeoplActive

PeoplActive is an IT consulting and recruitment services organization with leading capabilities in digital, cloud and security.

Roberts & Obradovic Law

Roberts & Obradovic Law

Roberts & Obradovic Law Group is a corporate, privacy, employment and litigation law firm.