Someone with FTP access to your site (you or your developers) has a virus on their workstations. This virus has installed a keylogger that is stealing credentials from your FTP client and sending this information back to the hacker.

The hacker collects hundreds of such credentials and then uses a program to log into each server, download a file, modify it to append an iframe or block of obfuscated JavaScript or PHP, upload the file, download the next file, modify, upload, next, etc. The files downloaded may either match a set of names (such as only index., default., home.* etc) or just any html or PHP file.

The appended code is often either an iframe that is visibility: hidden or of 1×1px size, a <script> sourcing a remote JavaScript file on a dubious domain, a collection of Javascript obfuscated by some clever str.CharCode’ing, or a block of base64_encode’d eval()’d code. Unobfuscating the code, the result is often an iframe. More recently, some clever attackers are inserting remote shells, granting them backdoor access to your server.

Once all the files have been modified, the attacker logs out. Visitors to your site will be subject to malicious code from the domain linked in the iframe with the intention of installing viruses and rootkits. Among other functions, these viruses will install a keylogger to sniff FTP credentials… and the virus continues spreading.

The attacker is using your credentials, so they can only access files that you have access to. Sometimes, they will upload an additional file in certain directories with an encoded shell, allowing them return access to the server (the common ones are _captcha.php in /forums directores and img.php or gifimg.php in /gallery directories). If you host other domains on your server, as long as the user for the affected domain has no access beyond their current domain, others will not be affected.

There are two ways to stop this sort of attack — prevention and proper antivirus. The attacks can be easily deflected by use of a firewall and limiting FTP access to only a few select IPs. The attackers are not attacking from your own workstation (yet), but rather a server elsewhere in the world. Using proper antivirus on all workstations with access to your FTP account — or, better yet, not using Windows XP — will help prevent the original infection from occurring.

If you are infected, it’s fairly easy to clean the messes up using a bit of clever sed, depending how good you are at spotting the injection and making effective regexes. Otherwise, backups backups backups — always have backups! …Oh, and change your FTP password or they’ll be back tomorrow.

Today, I received a piece of spam that contained a very interesting URL.

I own the domain iheartjpgs.info, on which I have an email address or two hosted. The spam is as follows:

Dear user of the iheartjpgs.info mailing service!
We are informing you that because of the security upgrade of the mailing service your mailbox (kale@iheartjpgs.info) settings were changed. In order to apply the new set of settings click on the following link:
http://iheartjpgs.info/owa/service_directory/settings.php?email=kale@iheartjpgs.info&from=iheartjpgs.info&fromname=kale
Best regards, iheartjpgs.info Technical Support.

The URL is intriguing, as this would suggest they had somehow compromised my server (or had broken their mass mailing script). Looking at the source of the message, however, I found that URL actually linked to http://iheartjpgs.info.bertdffm.co.uk/owa/service_directory/settings.php?email=kale@iheartjpgs.info&from=iheartjpgs.info&fromname=kale . This URL is much more curious, as it suggests that zone files were manipulated on bertdffm.co.uk to create a fairly dynamic subdomain solely for distribution of spam on iheartjpgs.info.

This subdomain had an A record pointing to a list of IPs:

iheartjpgs.info.bertdffm.co.uk has address 124.54.222.141
iheartjpgs.info.bertdffm.co.uk has address 125.143.86.51
iheartjpgs.info.bertdffm.co.uk has address 186.18.194.3
iheartjpgs.info.bertdffm.co.uk has address 189.62.171.189
iheartjpgs.info.bertdffm.co.uk has address 190.105.42.124
iheartjpgs.info.bertdffm.co.uk has address 221.161.59.185
iheartjpgs.info.bertdffm.co.uk has address 222.120.6.233
iheartjpgs.info.bertdffm.co.uk has address 81.198.204.182
iheartjpgs.info.bertdffm.co.uk has address 85.85.241.57
iheartjpgs.info.bertdffm.co.uk has address 85.107.80.168
iheartjpgs.info.bertdffm.co.uk has address 89.36.252.143
iheartjpgs.info.bertdffm.co.uk has address 114.205.211.94
iheartjpgs.info.bertdffm.co.uk has address 118.42.94.82
iheartjpgs.info.bertdffm.co.uk has address 121.125.49.20
iheartjpgs.info.bertdffm.co.uk has address 123.195.226.68

which appear to be residential computers from around the world, mostly in eastern Europe, Brazil, and Korea:

17858 | 124.54.222.141 | 124.48.0.0/13 | KR | apnic | 2005-12-21 | KRNIC-ASBLOCK-AP KRNIC
4766 | 125.143.86.51 | 125.128.0.0/11 | KR | apnic | 2005-08-23 | KIXS-AS-KR Korea Telecom
27747 | 186.18.194.3 | 186.18.192.0/22 | AR | lacnic | 2008-11-06 | Telecentro S.A.
28573 | 189.62.171.189 | 189.62.128.0/18 | BR | lacnic | 2007-03-30 | NET Servicos de Comunicao S.A.
27984 | 190.105.42.124 | 190.105.42.0/24 | AR | lacnic | 2008-06-17 | Ver Tv S.A.
4766 | 221.161.59.185 | 221.160.0.0/13 | KR | apnic | 2003-04-17 | KIXS-AS-KR Korea Telecom
4766 | 222.120.6.233 | 222.120.0.0/15 | KR | apnic | 2003-10-27 | KIXS-AS-KR Korea Telecom
12578 | 81.198.204.182 | 81.198.0.0/16 | LV | ripencc | 2002-10-02 | APOLLO-AS LATTELEKOM-APOLLO
12338 | 85.85.241.57 | 85.85.0.0/16 | ES | ripencc | 2004-10-22 | EUSKALTEL Euskaltel Autonomous System
9121 | 85.107.80.168 | 85.107.0.0/17 | TR | ripencc | 2004-09-20 | TTNET TTnet Autonomous System
9050 | 89.36.252.143 | 89.36.252.0/24 | RO | ripencc | 2005-11-29 | RTD ROMTELECOM S.A
9318 | 114.205.211.94 | 114.200.0.0/13 | KR | apnic | 2008-06-18 | HANARO-AS Hanaro Telecom Inc.
4766 | 118.42.94.82 | 118.32.0.0/12 | KR | apnic | 2007-08-03 | KIXS-AS-KR Korea Telecom
9318 | 121.125.49.20 | 121.125.0.0/16 | KR | apnic | 2006-08-02 | HANARO-AS Hanaro Telecom Inc.

Not all of the hosts are alive at the moment, which is unsurprising when dealing with residential computers. A simple portscan on one of them reveals the following services running:

PORT STATE SERVICE
21/tcp open ftp
80/tcp open http
554/tcp open rtsp
7070/tcp open realserver

Checking a few others that were alive, all of them had these four ports open at minimum. Connecting to the ports, I found that while open, only HTTP reacted as expected; the other ports did not follow their suggested protocols.

A simple GET / query on port 80 resulted in:

HTTP/1.1 302 Found
Date: Wed, 14 Oct 2009 15:29:07 GMT
Server: Apache
X-Powered-By: PHP/5.2.11
Location: http://www.microsoft.com/
Content-Type: text/html

Every one of the live servers served the exact same headers.

After manually setting my hosts file to one of the live servers, I visited the page in Safari — who immediately told me that it’s a phishing site. Kudos on the detection, Apple.

The page looks like a Microsoft Outlook Web Access page with all resources hosted locally. There’s no malicious JavaScript or iframes embedded into the page (which simply seems inefficient). The page offers up a “settings file” that I should run on my workstation to update my Exchange settings — the file is called kale-settings-file.exe. Or john-settings-file.exe. Or mary-settings-file.exe. Or whatever I set the $fromname variable to in the URL.

I scanned the file at VirusTotal – here is the report. As you can see, only 6 of the 41 virus scanners found anything: F-Secure, Kaspersky, McAfee+Artemis and McAfee-GW-Edition (both different than standalone McAfee), Microsoft’s built-in (!), and Sophos. Each AV product identifies the file slightly differently, though the common thread is Zbot. The Zbot payload first started showing up around January; the methods used to hide and obfuscate the payload, however, have grown increasingly sophisticated and harder to detect.

A summary of Zbot from ThreatExpert:

Threat characteristics of ZBot - a banking trojan that disables firewall, steals sensitive financial data (credit card numbers, online banking login details), makes screen snapshots, downloads additional components, and provides a hacker with the remote access to the compromised system.

Keylogger, trojan/bot, backdoor.

Trend picked up on this new “customized URL” Zbot spam and blogged about it yesterday. Zbot is also deployed in spam emails referencing the IRS (”Notice of Underreported Income”). It should be noted that the Zbot-infected binary distributed in the two spam campaigns are significantly different, though the payload is the same — the changes are designed to evade anti-virus protection, always staying one step ahead of the AV firms. A fun user-end analysis of the Zbot trojan (technically, the PostZeuS trojan, on which Zbot is heavily based) can be found at this (machine-translated) French website.

Bottom line: never click links in spam. Seriously. Especially if you run Windows.

Addendum: If you have downloaded and run the linked file, please update your virus definitions and immediately run a scan. You can also go to here to run F-Secure’s online virus scanner, which ought be able to detect and assist in cleaning of the virus as well.

Following up on Trend’s whitepaper, Sophos’s Dmitry Samosseiko has a great read about Partnerka, the Russian affiliate virus deployment and spam marketing get-up.

Trend’s got an excellent whitepaper about the Ilomo botnet up on their site. Well worth the read.

No patch from Adobe yet, though you can disable JavaScript execution within Acrobat to protect yourself. A full rootkit is the payload of the exploit, which takes advantage of Acrobat’s JavaScript support.

http://blog.trendmicro.com/new-adobe-zero-day-exploit/

These compromises aren’t really “compromises” at all — the attacker already has a user/pass to log into the system. The password is often gained illegitimately through a rootkit on the end-user’s computer, sniffing authentication credentials without the user’s knowledge, or through phishing scams. With such access, it’s simple for the attacker to launch a mass infection by simply logging in via FTP and systematically downloading, modifying, and re-uploading website content pages. The modifications are often hidden iframes (sometimes obfuscated by javascript) that redirect unsuspecting visitors to another site which pushes viruses (such as the same rootkit that originally compromised the account information). Recently attackers have also been inserting code allowing remote code execution by an attacker, allowing them to weaponize the server for use in DoS attacks and spamming.

(Note: these commands contain a Plesk bias, as I see such compromises overwhelmingly on Plesk for some reason…)

Identify the files that have been modified:

mkdir /home/rack/ticketnumber
cd !!$
cp /usr/local/psa/var/log/xferlog.* .
gunzip xferlog*
awk '$12 != prev {print $9; prev=$12}' xferlog* | egrep "\.php|\.htm|\.shtm|\.js" | sort |uniq > ftp_modified.out

Search through the result set for common changes:

cat ftp_modified.out |while read line; do grep -H iframe $line >> iframe.out ; done

Review this output for suspicious iframes. Often they have the visibility:hidden attribute, are Russian or Chinese, listen on odd ports, or point to a CGI script. Once you’ve identified one of these, review the file it was found in for unmatched malicious content, usually in the form of obfuscated JavaScript. Note each of these “signatures”.

Generate a regex tight enough to match only such malicious content, but be aware of random elements put into the signature so as to thwart cleanup attempts.

Run this regex against the modified files:

cat iframe.out | awk -F\: '{print $1}' | while read line ; do sed -i 's/<iframe src=.*\/in\.cgi\?.*<\/iframe>//g' $line ; done

Immediately change the passwords for the affected domains, and be sure to advise the customer that the infection will recur lest they find the breach on their side — the infected desktop — and clean it up. Most antivirus products can catch and clean the common rootkits and trojans, but be aware that each scans differently and will find different things. If one scan comes up clean, scan again with another AV product, and likely triple-check as well.

For an Apache compromise, note the domain the attack was coming from and review the logs. This can be tedious if you don’t quite know what you’re looking for — a lot of the attacks have gotten quite a bit smarter, using tor and proxies to randomize IPs, and using scripts that redeploy or change names to further confuse things.

Look for hits on the script that was running in the logs; search again for IPs that hit that script. Try to find out where they came in on the site — did they hit the front page first? Did they hit any pages besides the script? Look for common threads and try to trace your way back to the earliest instances of hits on the attack scripts. See what they accessed just before.

Often times, these “first hits” are POSTs against a legit script on the site. For example, a recent ZenCart (shopping cart software) exploit starts out like this:

/admin/record_company.php/ password_forgotten.php?action=insert

This particular exploit is from:

http://www.milw0rm.com/exploits/9004

And the client ought follow the instructions from ZenCart here:

http://www.zen-cart.com/forum/showthread.php?t=130161

Keep abreast of the hacks as they come out — it’s always cat and mouse, but there’s no reason to lag too far behind!

ps auxwww |grep apache

Examine the output for the user apache running anything but the correct apache binary (/usr/sbin/httpd or /usr/local/bin/apachectl -k startssl). Also look for children processes running with extremely high CPU time (if most children are running with 1:20 or so, and one is running at 60:42, note hte pid of the latter).

Closely examine the child process that is unusual with lsof.

lsof -p 19232

Note any network connections:

lsof -p 19232 |grep IPv4

and note the current working directory of the process:

lsof -p 19232 |grep cwd

It is in this directory that we start the search. Review other files in that directory — look specifically for files owned by apache, files with a much newer modification time than other files in the directory, and hidden directories (named ” ” or “…” etc) or unfairly-named files (”å¥∂øƒø∂ç.php” for example). Investigate suspicious files themselves and use discretion as to whether it is legit or not — it helps to know a bit of PHP or coding here. Note the modification times of the files as well. If you’re certain a file is not part of the site and decidedly malicious, move it away:

mkdir /home/rack/ticketnumber/comp
mv c99.php !!$

Often attackers will hide other shells around the web roots if the apache user can access them (777 dirs). Look for apache-owned scripts to aid the search:

cd /home/rack/ticketnumber
find /home/httpd/vhosts/ -user apache -mtime 365 |egrep "\.php|\.cgi|\.pl" >> newfiles.out

You will have to manually review newfiles.out and each file to tell if a file is malicious. Give the code the benefit of the doubt, but look for tell-tale shells, obfuscated code, and vandalism.

Once files are identified, cull out the legitimate-looking files from the list. Move the malicious files out of the webroots:

alias mv='mv'
cat newfiles.out |while read line ; do mv $line comp ; done

Again review newfiles.out and manually review the directories that contained the malicious scripts. Look for hidden directories, suspicious files that didn’t match the filter (c code, binaries, etc). Remove this content as well.

When logging into a server, the first few seconds at the command prompt can offer general health information. Did it take a long time to get a command prompt after authenticating? Are there delays when typing in characters at the prompt? Are you getting disconnected often?

If you have even the inkling of suspicion, there’s no harm in getting a “feel” for your environment. Note anything that feels unusual; look for anything that stands out. Unless it’s root compromised, there are always breadcrumbs.

Document the environment as it is active:

mkdir /home/rack/ticketnumber
cd !!$
script !!$.out
ps auxwww >> psauxwww.out
pstree >> pstree.out
netstat -plan >> netstat-plan.out
lsof -i >> lsof-i.out
lsof >> lsof.out

Here we determine what kind of compromise it is.