Guest Blog Post by Randy Resnick

It started with an alert from the mail queue, way too many returns. I went to look at what was going on and it became apparent that a mailing had been done on our dedicated server. I began collecting evidence. Here’s how:

1. Looking at the server logs, I saw the campaign going out via a php script. The server was tightly controlled by someone whose knowledge I respect. How could this be happening? Simple. It was an inside job, installed by the web agency designing the new sites at a very high cost to the customer.

2. I found the php script recently uploaded inside the web framework and the list of names. I inserted an anonymous email of mine to the list and waited until the next batch. Sure enough, the message was a spammy mailing going out to the list from another customer of the agency. In short, the agency was using the server of one customer to send another customer’s mailing! Why they did something this stupid has never been explained.

3. I spent most of a week gathering rock solid evidence, because there is a great ironic twist to this story. For the past several months, the web agency was “poisoning” our image, telling our customer of over 10 years that the server wasn’t powerful enough, that we weren’t cooperating with them, and several other completely fabricated lies.

When I had a file that clearly incriminated the agency, I presented it to the IT director of our customer. He took one look and knew that the agency’s house of cards had just crumbled.

We were lucky to see the mail queue fill up, but since that time, I have added cron jobs that do the following within areas controlled by third parties like the agency:

– look at file changes every few minutes and send me an alert when new files are uploaded

– look at all sftp logins

We also now use a separate dedicated server for email and that server is running fail2ban which I highly recommend on all servers.

The first thing you always need to consider when you are concerned with security is who has legitimate access to your server, whether they are agencies or VoIP clients. Traffic analysis is important, it’s how I found the agency fraud. Analysis and fraud detection is what Humbug Labs specializes in.

Humbug Labs Adds:

Keep in mind that when talking voice, this is just one type of insider resource fraud. There are several others like internal calls that are not part of your business (cleaning staff, employees, etc making long distance calls). To parallel the case in this blog, there are cases where your PBX can be turned into a telecom company for someone else’s financial benefit at your cost, this can be internal or the result of vendor misconduct (as in Randy’s example) or even hacking.

About Randy Resnick:

Randy is the creator and producer of the weekly VoIP & Tell conference call, where people have been meeting to talk about telephony, security and networks for the past 5 years. Humbug Labs is an active participant of the talks.

Check them out at or and please join them on the weekly call at 12 Noon Eastern time each Friday.

Image Source: