Fighting spam is really an arms race. As technology evolves, new threats emerge and keeping up is an absolute necessity. The Site Integrity team at Facebook works to protect people from spam, scams, account compromises, and other forms of abuse by building real-time classification systems that process millions of interactions per second and investigate and respond to new threats.
At the beginning of 2011, I joined a new sub-team on the very front line of the war against spam—Recon & Response (SI-RAR). This team is constantly fighting against the biggest and most pressing threats out there, and when things are quiet, we build more defenses. Here’s a look at some of the attacks we worked on last year and the new systems we built to combat emerging threats.
Fake accounts
When I first joined SI-RAR, I immediately started battling a fake account attack. Typically, spammers pose as attractive women to have their friend requests accepted and then use this friendship to send spam. Spammers will go to great lengths to create accounts that look as real as possible, generating fake backgrounds, fake photos, and even fake conversations. In this particular case, a website for Russian brides created fake accounts to promote its services. We had already done a considerable amount of work to identify and shut down these accounts before a single friend request could be sent, but this case was different because these accounts would not send friend requests, but rather comment on some public threads in hopes of receiving a friend request. I learned very quickly that it is very common for attackers to react promptly to the protection we put in place—sometimes within hours. So, I cut my SI-RAR teeth by tweaking our systems often and disabling all of these fake accounts.
Social engineering attacks
In May, we faced a new form of an attack now known as “self-XSS.” Security engineers have known for a while that social engineering attacks (where you trick or deceive people) have the potential to be bigger and more dangerous than attacks leveraging an actual security hole. Normally, one of these attackers will use a fraudulent hook to lure people (shocking Justin Bieber video, free iPad 3s, gift cards, free airline tickets, etc.) and then ask them to copy and paste code to access these special items. However, this code is a piece of malicious JavaScript that allows the spammer to post on person’s profile and their friends’ profiles.
Contrary to a typical cross-site scripting (XSS) vulnerability, which is a security vulnerability in Facebook’s code that would have to be fixed, with self-XSS there is nothing to “fix”. We’ve talked to browser teams across the web and they have made progress, but so have the spammers. Right now, the attacker directly asks people to open the JavaScript console, thereby circumventing some of the newer browser protections. Whenever we detect that someone has been tricked, we put them in a special flow to explain what happened to their browser during this attack and at what point their account was compromised. One important lesson I learned here is that you really cannot take for granted that everyone knows what Javascript is. And as a result, security cannot be an afterthought, it has to be carefully designed within the product.
Monitoring
As you can imagine, attacks do not spike on Wednesdays at 2:00 p.m. when everyone is at their desks, ready for the smallest incident—that would be too simple. They usually start on Friday night, and by the time you open your computer on Saturday, the attack is well under way. Spammers are located all over the world, especially in countries where it is hard to prosecute them. They have day jobs, very often in computer security, and they are always planning strikes around times we are less likely to be at work. Data and automation are our best friends in this world and we use these tools to constantly monitor all negative interactions happening on the site (friend requests being declined, messages reported as spam, etc.). On top of that, an important part of my job is to research and predict what is going to come next and plan for the attacks that we don’t know about yet.
Malicious extension
Last month, we noticed a significant change from self-XSS to malicious browser extension. The upside is that we blocked self-XSS well enough that the attackers had to find a new vector for spam and changed to something with a lot more friction. This browser extension is essentially a virus that people install on their computer under the illusion they are installing a video plugin. We have been working with the browser vendors to fight these extensions and are deploying more advanced counter measures all the time.
And so this was 2011. We saw old attacks declining (drive-by download) and new attacks showing up (self-XSS), and we’ve put new protections in place that we will continue to iterate on this year. However, a team of security engineers will never be as effective as millions of users aware of security issues.
My team works hard to build spam protection that will secure everyone’s account, and with more than 800 million people using Facebook, this is a considerable challenge. But everyone on my team is always on-call, ready to fight. I joined Site Integrity because I liked ML problems and thought I could have a big impact working on security, and two years later, every day still brings a new set of challenges.
Want to help us protect the internet? Apply here.
