Horse Racing

Tabcorp’s IT staff put on notice days before major outage

Some of Tabcorp's most experienced IT and technology staff were handed letters telling them to reapply for their jobs just days before a freak and unforseeable major outage to the betting company's core system wiped out Stakes day.

A source inside the betting agency has told The Age that it could have taken less than the 28 hours it did to reboot the system had a company reorganisation, which left staff disillusioned and hurt, been delayed until after the spring racing carnival, an assertion categorically denied by Tabcorp.

On the Monday before Derby day, Tabcorp's senior managers and heads of IT and technology departments were issued impact letters giving them until 5pm on Stakes day to reapply for their jobs as part of a cost-saving operation led by Tabcorp's new chief information officer Francoise Russo.

Tabcorp experienced IT woes during the carnival.Credit:Getty Images

Five general managers were also made aware in September that they would be leaving as part of a leadership team restructure. Since then, Tabcorp spokesman Nick Tzaferis says, the implementation of the new operating model has progressed.

But in a twist, the company's corporate applications were affected by the same outage that left punters reeling on Stakes day, meaning those staff had to reapply for their jobs on the following Monday.

In its 2019-20 full year results announcement in August, the company mooted an "enterprise-wide optimisation program to deliver significant cost savings", including "data and digitisation improvements and maximising value from our vendor spend", which comes after Tabcorp's merger with Tatts became substantially complete following the migration of Ubet customers to TAB's digital platform earlier this year.

In-sourcing over the past two years has been a result of the merger, with the new company absorbing different methods of operation across Tabcorp and Tatts.

But that cost-cutting exercise left staff confused on the chain of responsibility in racing's biggest week and left Tabcorp without anyone to blame for the Stakes day catastrophe, according to the company source.

"The recent project to re-insource services has really diluted the clarity of those accountabilities so it would be hard for Tabcorp to point the finger directly at a single vendor and say 'it's your fault that we were down for 28 hours'," the TAB source said.

"Whereas it used to be entirely outsourced, services that were delivered regarding the management of the data centre, it's recently become quite convoluted by a re-insourcing of activities that were cost-driven.

"The fact that she [Russo] has run a reorg design during the spring racing carnival seemed like a pretty interesting sort of idea. Everyone's gone, 'Surely she'll wait until after the spring racing carnival' … but they ploughed on with it."

Tabcorp rejects that suggestion.

The cause of the major outage at the third-party managed data centre in Ultimo, which was estimated by Racing Victoria to cost the industry in excess of $2 million, remains under investigation. Fire retardant was said to be used on the equipment, causing the significant outage.

In hindsight those in charge should have decided to direct the data to the company's disaster recovery site in Brisbane, the TAB source said, despite describing the back-up station as under-utilised and adding it potentially wouldn't have been ready for the onslaught of bets laid on the final day of Cup week.

"The level of confidence in failing over [moving data] to that DR site on the day was low, given it was 11.30am and we were trying to get the platform back up hopefully by the start of the quaddie," the source said.

"In hindsight, they should have done it. There was probably not a clear enough of an understanding of how badly the Ultimo data centre had been affected, and then all of the other factors come into play around what's been going on within the organisation that's got us to the position where we're not clear on accountabilities and not able to quickly resolve the issues in Ultimo.

"There's obviously some questions why it took 28 hours to restore service at the data centre."

However, sources within the TAB are disillusioned by the staff cuts led by ex-Toll Group chief Russo and the technology strategy the company has adopted.

"It would be understandable if people were a little bit less switched on or committed to staying in front of the screen until the issue got resolved," the insider said.

"It's reasonable to say it [the reboot] could have been done sooner in a world where you've not got all this reorg activity happening. It's fair to say people are fatigued."

But Tabcorp denied claims the system was vulnerable heading into racing's biggest week.

"We categorically refute any suggestion that the technology restructure in any way compromised our Spring Racing Carnival operations," Tzaferis said.

"We also entirely refute any suggestion that team disengagement impacted the recovery effort. Our teams worked 24/7 with full focus to restore service levels as soon as possible.

"We have about 1000 employees in technology. In September we announced a plan to evolve our structure so that we can deliver more effectively in a changing environment for our business.

"Our technology team is focused on being future fit and digitally focused, and our new operating model has been designed with that in mind."

Most Viewed in Sport

Source: Read Full Article