Why are IT people so miserable? Log4j2itis

Why are IT people so miserable? Log4j2itis

The biggest security problem of a customer's life is happening right under their nose. Even if they don't know about it, their IT admins do.

Credit: Dreamstime

Instead of holiday toasts, did your customers hear screams and moans from their server room? Are IT people sobbing inconsolably even when Amazon Web Services (AWS) is running? Did they walk over sleeping system administrators and developers when they returned to the office?

If that's happening to your customers, let me explain what’s happening. IT people — a lot of IT people — are suffering from Log4j2itis.

You may have seen some general news about it over the last couple of weeks, as even general news sources are picking up that it's bad news. As Jen Easterly, director of the the US Cybersecurity and Infrastructure Security Agency (CISA), said: "The Log4j vulnerability is the most serious vulnerability I have seen in my decades-long career."

I've been at it longer than she has and in my never very humble Twitter opinion, "#Log4Shell may, with no exaggeration, be the worst IT #security problem of our generation."

That sounds really scary, because it is really scary. But what is it exactly? For the side of the story that requires you to have words like "security," "system administrator," or "developer" in your title, I’ve got the ugly details in my New Stack post: "Log4Shell: We Are in So Much Trouble."

If you're an ordinary mortal, here's what's going on and why it's such a major pain to deal with.

Apache Log4j2 is an extremely popular open source Java logging library. If a Java program logs, well, pretty much anything, from the user's name to the number of times it calls some other program for help, odds are it uses Log4J2 to do the job.

That was fine. That was dandy. Everyone was happy. But, then last month security investigators found that if users could make it log a line of malicious code, bad things would happen. How bad?  It has a "perfect" Common Vulnerability Scoring System (CVSS) score of 10 out of 10. It's as bad a security vulnerability as there can ever be.

If any of programs contain a vulnerable version of Logj42, they can be blasted with a remote code execution flaw attack. If successful, an attacker can do anything from playing Doom on servers (seriously) to infecting every box on the network with the Mirai botnet to stiffing the business with ransomware. 

Oh, and government-sponsored hackers are now using the Log4j vulnerability as well. Just ask the Belgian Defense Ministry, which was still recovering from an attack just recently.

What might those programs be? Good question. Thousands of widely used commercial programs are attackable. These include Apple iCloud; numerous Cisco programs; Minecraft client and server; Steam; Twitter; and many VMware programs.

And, if end-users or independent software vendors (ISV) wrote programs with such software components as Apache Druid, Dubbo, Flink, Flume, Hadoop, Kafka, Solr, Spark, and Struts, they could be open to attack, too. This is a security hole that just keeps giving and giving.

The good news is there's a fix, three fixes actually, for Log4j2 vulnerabilities. The short version is if customers update every copy of this troubled software library to log4j 2.17.0, all will be well.

Aye, there's the rub. Customers must update every last one of them. And here's the really not-so-good part. Log4j is hidden away in millions of programs. Without a software bill of materials (SBOM) for every application, users can't be sure they’ll find them all. And SBOM is a new concept. No one was making them last year, never mind seven years ago when Logj42 was first released.

So users must look for them. And, because Java programs hide their code in Russian-nesting doll structures such as Java archive files (JAR), finding the one program that needs patching can be a real pain. There are tools, such as the CISA CVE-2021-44228_scanner, that make life easier for security and development teams, but it's still a lot of work.

Imagine if someone asked you or your customer to find every reference you ever made in documents to your CEO since 2014… without easy-to-use text search tools. It would be a nightmare, right? Now, imagine that if you don't find it your company’s IT infrastructure will collapse into a god-awful mess.

So, be kind to IT staffers. Instead of drinking a New Year's Eve glass of champagne, they were likely still tracking down and cleaning up this mess. This is not going to end quickly and there will be many more related attacks to fend off before it's all done.

Happy new year?

Tags cyber securityApache Log4j

Show Comments