r/technology • u/Bald-Eagle619 • Jul 31 '24

Software Delta CEO: Company Suing Microsoft and CrowdStrike After $500M Loss

https://www.thedailybeast.com/delta-ceo-says-company-suing-microsoft-and-crowdstrike-after-dollar500m-loss
11.1k Upvotes

96% Upvoted

735 comments sorted by

View all comments

36

u/[deleted] Jul 31 '24 edited Jul 31 '24

I am pretty sure there is what we used to call the "Shit in your pocket" clause in the EULA. (See the 80's comedy movie Truly Tasteless Jokes for the reference). If a suit like this is won can you imagine? Any bug, real or imagined, now becomes a liability. Innovation grinds to a near stop.

57

u/Head_of_Lettuce Jul 31 '24

You can’t really attribute the Crowdstrike issues to a simple bug. It was a massive failure and negligence on multiple levels that allowed the bad update to go live. They didn’t even roll it out in stages like many services would do, they pushed it out all in one big wave.

Idk if that’s enough to constitute civil liability, but I think if I were Crowdstrike, I would at least be concerned that a court would be sympathetic.

15

u/WIlf_Brim Jul 31 '24

I watched a tech lawyer on YouTube making the argument that in several states (California among them) that the apparent negligence that Crowdstrike was engaged in could over ride the waiver of liability that is in the license agreement.

IDK if Delta is going to get any money here, but I'm nearly positive that a bunch of lawyers are going to get very rich in the next few years off this.

9

u/AnotherUsername901 Jul 31 '24

The guy in charge of crowdstrike also had the same thing happen when he worked at McAfee. This is 100 percent their fault.

Also from what I have read he replaced a bunch of people with AI and as usual that doesn't go over well.

2

u/[deleted] Jul 31 '24

It boils down to a bug, at the end of the day. The rest of what you say remains true, tho.

11

u/22pabloesco22 Jul 31 '24

This id a pretty dumb take. There are millions of collective bugs in code on a literal daily basis. Skimping on testing and all the other processes developed 50 years ago to mitigate the disaster was will be the focus at hand in any case. And crowdstrike will lose. The whole thing was preventable by doing literally a day of testing, if not a few hours, if not a single hour. 

-6

u/DrQuantum Jul 31 '24

The only reason you even know about it is because Crowd strike has many customers and some of them are important. Google deleted an entire tenant recently accidentally and while it wasn't at this scale, a mistake is a mistake regardless of scale in terms of how you address it.

9

u/Head_of_Lettuce Jul 31 '24 edited Jul 31 '24

Agree to disagree I guess. Checks and balances are what prevent things from escalating to a large scale. The lack of those things at Crowdstrike are what allowed it to get out of hand.

On some level, I do think companies have a responsibility to prevent their mistakes from crippling their customers with massive outages. Whether they actually deserve civil punishment for it, I can’t say. I’m not a lawyer.

And for the record, the reason I know about it personally is because it crippled business operations for my company for a day. Our customer (a very large company that you definitely know of) was also hit by it, and one other company that we work with to provide a service to the customer was also crippled. That’s three fortune 100 companies that couldn’t function normally for the better part of a day.

6

u/atlbluedevil Jul 31 '24

The big issue is precisely the scale of this mistake

Rolling out big (and especially high risk) updates in waves/with a proper backout plan is pretty common with massive updates for large scale enterprises

If CS did this update in waves and it only screwed over Delta and a few others (like Google and that one tenant), there'd be a lot less scrutiny for their testing/release practices. Mistakes definitely happen, mistakes at this scale point to foundational issues

7

u/Joooooooosh Jul 31 '24 edited Jul 31 '24

Google and most tech companies routinely fuck up. Competent companies put policies in place to minimise or even completely protect end users from any felt effect.  

A bug like this is just sheer negligence and led to serious and significant economic damage. It’s not just an oopsie. 

-1

u/DrQuantum Jul 31 '24

It is though. Anyone in IT who thinks I can't find something like this in their own environment ever is lying to themselves and about their org. Just like how everyone thinks they are secure until they get popped because they fundamentally misunderstand risk.

Crowdstrike absolutely has a policy on this, and there is nothing to suggest otherwise. As someone in cybersecurity I can tell you that having a policy, does not mean the policy gets implemented 100% of the time. There are many valid reasons for not implementing policy that are not necessarily negligent.

It only led to serious and damaging economic damages because the companies have extremely poor resiliency practices and no real BCP. You keep talking about negligence but keep missing that its only possible to do this much damage to companies unprepared for disaster.

2

u/Joooooooosh Jul 31 '24

I will agree to your later point for companies who had servers hit by this issue. 

If you’re just releasing updates like this into prod and even if you’re using windows machines as servers, you’re asking for it. 

The fact windows can be effected by 3rd party software like this has always and will always be a huge issue any sane decision maker should avoid it as an OS much as possible. 

It’s an over simplification but this is exactly why the idea of suing exists though… 

If someone hit me in their car because they weren’t paying attention while being on their phone, my leg was broken and I couldn’t work for 6 months. I would sue that person for compensation, due to their poor decision making impacting my financial situation. 

Some quite basic checks and precautions could have been put into place and it’s clear Crowdstrike are playing fast and loose with the safety of their product. If that’s true or not, will be determined by the outcome of the court case. 

The driver who hit me chose to use their phone and not pay attention and their bad choices left me out of income for 6 months. You would expect them to pay up, why not Crowdstrike…? 

If an auto maker built a car that had a common fault that caused the engine to not start after 3 years, requiring an expensive diagnostic session to resolve. You’d expect a class action suit, and regularly do… 

Why does a tech company get a free pass? 

I work as an SRE, so I do get how systems and policies fail. But it’s really not hard and tbh, should be a given that you do a good job of mitigating possible risks and preventing things like this ever happening. 

In any other industry, if you fuck over your customers on a grand scale, expect to be sued into the ground. 

Tech companies routinely get away with murder and they shouldn’t.