TSB's IT Meltdown Was Evident A Year Before

The banking software at the heart of TSB’s troubles this week was doomed to failure from the start, an insider with extensive knowledge of the systems involved has said. With customers locked out of their bank accounts, mortgage accounts vanishing, small businesses reporting that they could not pay their staff and reports of debit cards ceasing to work, the TSB computer crisis has been one of the worst in recent memory. 

The bank it faces a compensation bill likely to run to tens of millions of pounds and CEO Paul Pester said recently that the bank was on its knees.

Just before the bank’s services crumpled, software engineers and Banco Sabadell, TSB’s Spanish owner, were toasting their own efforts with champagne and claiming a job well done. The comments posted below the photo read: “Hell of a team!” and “Champions!” However, the warning signs that a catastrophe of this magnitude might happen were apparent a full year earlier.
When TSB split from Lloyds Banking Group (LBG), a move forced by the EU as a condition of its taxpayer bailout in 2008, a clone of the original group’s computer system was created and rented to TSB for £100m a year.

That banking system was a “bodge of many old systems for TSB, BOS, Halifax, Cheltenham and Gloucester and others” that had resulted from the “nightmare” integration of HBOS with Lloyds as a result of the banking crisis, according to one insider who had extensive access to and intimate knowledge of LBG and TSB’s internal systems over a prolonged period.

“The idea with the IT was to create a mirror copy of the sprawling LBG merged systems and use this to service the much smaller TSB bank. It seemed a bad fit for a smaller bank to inherit all the problems of a bloated mess to service far fewer customers,” the insider said.

Under this arrangement, LBG held all the cards. It controlled the system and offered it as a costly service to TSB when it was spun off from Lloyds in September 2013. 

When Sabadell bought TSB for £1.7bn in March 2015, it put into motion a plan it had successfully executed in the past for several other smaller banks it had acquired: merge the bank’s IT systems with its own Proteo banking software and, in doing so, save millions.

Sabadell was warned in 2015 that its ambitious plan was high risk and that it was likely to cost far more than the £450m Lloyds was contributing to the effort. 

“It is not overly generous as a budget for that scale of migration,” John Harvie, a director of the global consultancy firm Protiviti, told the Financial Times in July 2015. But the Proteo system was designed in 2000 specifically to handle mergers such as that of TSB into the Spanish group, and Sabadell pressed ahead.

By the summer of 2016, work on developing the new system was meant to be well under way and December 2017 was set as a hard-and-fast deadline for delivery.

“The time period to develop the new system and migrate TSB over to it was just 18 months,” the insider said. “I thought this was ridiculous. TSB people were saying that Sabadell had done this many times in Spain. But tiny Spanish local banks are not sprawling LBG legacy systems.”

To make matters worse, the Sabadell development team did not have full control, and therefore a full understanding, of the system they were trying to migrate customer data and systems from because Lloyds Banking Group was still the supplier. 
“This turned what was a super-hard systems job [into] a clusterfuck in the making,” the insider said.

By March 2017, the nightmare for customers that was going to unfold a year later appeared inevitable. “It was unbelievable, hardly even a prototype or proof of concept, yet it was supposed to be fully tested and working by May before the integration work started,” the insider continued. “Senior staff were furious about the state it was in. Even logging in was problematic.”
By the autumn it still was not ready. TSB announced a delay, blaming the possibility of a UK interest rate rise, which did materialise, and the risk that the bank might leave itself unable to offer mortgage quotes over a crucial weekend. 

Sabadell pushed back the switchover to April to try to get the system working. It was an expensive delay because the fees TSB had to pay to LBG to keep using the old IT system were still clocking up: Pester put the bill at £70m.

On 23 April, Sabadell announced that Proteo4UK, the name given to the TSB version of the Spanish bank’s IT system, was complete, and that 5.4m customers had been “successfully” migrated over to the new system. Josep Oliu, the chairman of Sabadell, said: “With this migration, Sabadell has proven its technological management capacity, not only in national migrations but also on an international scale.”

The team behind the development were celebrating. In a LinkedIn post since removed, those involved in the migration were describing themselves as “champions”, a “hell of a team” and were pictured raising glasses of bubbly to cheers of “TSB transfer done and dusted”.

However, only hours after the switch was flicked, systems crumpled and up to 1.9m TSB customers who use internet and mobile banking were locked out. “I could have put money on the rollout being the disaster it has been, with evidence of major code changes on the hoof over last weekend and into this week,” the insider said.

Twitter lit up as customers frustrated by the inability to access their accounts or get through to the bank’s call centres started to vent their anger.

Customers reported receiving texts saying their cards had been used abroad, that they had discovered thousands of pounds in their accounts they did not have, or that mortgage accounts had vanished, multiplied or changed currency. 
One bemused account holder showed his TSB banking app recording a direct debit paid to Sky Digital 81 years from now. Some saw details of other people’s accounts and holidaymakers complained that they had been left unable to pay restaurant and hotel bills. 

TSB, to customers’ fury, at first insisted the problems were only intermittent. At 3.40am on Wednesday 25 April, Pester, tweeted that the system was “up and running”, only to be forced to apologise the next day and admit it was actually only running at 50% capacity. 

Recently he admitted the bank was on its knees, announced that he was personally seizing control of the attempts to fix the problem from his Spanish masters, and had hired a team from IBM to do the job. Sabadell said it would probably be another week before normal service returned.

The financial ombudsman and the Financial Conduct Authority have launched investigations. The bank has been forced to cancel all overdraft fees for April and raise the interest rate it pays on its classic current account in a bid to stop disillusioned customers taking their business elsewhere.

The number of complaints is slowing, but they have not yet ceased. One customer told the Guardian that some of their personal details that have been switched on to the new system were five years out of date. A Twitter user said they had contacted the bank about a text message received relating to an account closed more than five years ago. 

The software Pester had boasted about in September of being 2,500 man-years in the making, with more than 1,000 people involved, has been a customer service disaster that will cost the bank millions and tarnish its reputation for years.

Guardian:

You Might Also Read:

Bank Data Breaches Are Up And It's An Inside Job:

HSBC Appoints A Technology Advisory Board:
 

 

 

« Cambridge Analytica Goes Out Of Business
British Healthcare System Spends £150m Extra On Cybersecurity »

CyberSecurity Jobsite
Check Point

Directory of Suppliers

IT Governance

IT Governance

IT Governance is a leading global provider of information security solutions. Download our free guide and find out how ISO 27001 can help protect your organisation's information.

North Infosec Testing (North IT)

North Infosec Testing (North IT)

North IT (North Infosec Testing) are an award-winning provider of web, software, and application penetration testing.

MIRACL

MIRACL

MIRACL provides the world’s only single step Multi-Factor Authentication (MFA) which can replace passwords on 100% of mobiles, desktops or even Smart TVs.

XYPRO Technology

XYPRO Technology

XYPRO is the market leader in HPE Non-Stop Security, Risk Management and Compliance.

ZenGRC

ZenGRC

ZenGRC (formerly Reciprocity) is a leader in the GRC SaaS landscape, offering robust and intuitive products designed to make compliance straightforward and efficient.

Conscio Technologies

Conscio Technologies

Conscio Technologies is a specialist in IT security awareness. Our solutions allow you to easily manage innovative online IT awareness campaigns.

Cleo

Cleo

Cleo is a leader in secure information integration, enabling both ease and excellence in business data movement and orchestration.

SailPoint

SailPoint

SailPoint provides identity governance solutions with on-premises and cloud-based identity management software for the most complex challenges.

CyberGreen Institute

CyberGreen Institute

The CyberGreen Institute is a global non-profit and collaborative organization conducting activities focused on helping to improve the health of the global Cyber Ecosystem.

Base Cyber Security

Base Cyber Security

Base Cyber Security is an information and cyber security talent service provider and career specialist.

ISA Global Cybersecurity Alliance (ISAGCA)

ISA Global Cybersecurity Alliance (ISAGCA)

Objectives of the ISA Global Cybersecurity Alliance include the acceleration and expansion of standards, certification, education programs, advocacy efforts, and thought leadership.

Expel

Expel

Expel provide transparent managed security services, 24x7 detection, response and resilience.

Diaplous Group

Diaplous Group

Diaplous Group is a leading Maritime Risk Management (MRM) provider, delivering specialized services to an ever-broadening portfolio of shipping, oil & gas, energy and construction industries.

SEIRIM

SEIRIM

SEIRIM delivers cybersecurity solutions in Shanghai China specializing in Web Application Security, Network Security for SME's, Vulnerability Management, and serving as Managed Security as a Service.

evolutionQ

evolutionQ

evolutionQ delivers quantum-risk management strategies and robust cybersecurity tools designed to be safe in an era with quantum computing technologies.

TotalAV

TotalAV

TotalAV Antivirus is a free-to-use app packed with all the essential features to find and remove malware, keeping you safe.

Retruster

Retruster

Protect your users against phishing emails, ransomware & fraud with the most advanced, user-friendly, non-intrusive solution available.

Tailscale

Tailscale

Tailscale is a VPN service that makes the devices and applications you own accessible anywhere in the world, securely and effortlessly.

MicroAge

MicroAge

Powered by five decades of experience, lasting partnerships, client relationships, and the values that guide us daily, MicroAge is here to help you secure, accelerate, and transform your business.

Kolide

Kolide

Kolide ensures that if a device isn't secure, it can't access your apps.

Linx Security

Linx Security

The Linx Identity Security platform enables identity, security, and IT ops teams to finally control the whole identity lifecycle.