Watir Makes Waves
It’s early 2009 and I just graduated from Facebook’s engineering bootcamp. Facebook’s test processes are nascent and disorganized, but another new engineer, Brian Rosenthal, is making waves integrating Watir testing into our systems. Watir is a slick library written in Ruby designed to write integration tests in the browser. Watir automates the browser with JavaScript and allows you to ensure your Web pages are loading and behaving appropriately. As a Ruby enthusiast and a lover of automated testing, I dive in and try to help define some of our first reusable abstractions for testing pages on facebook.com. It isn’t long before I’m pulled into different projects, but Watir continues to make a lot of progress and soon it is part of our “push blocking” routine, wherein a barrage of Watir tests, mostly focused on our core privacy logic, must pass before new code can go live. Later, Adam Simpkins develops “Watirworld,” which allows you to easily farm out your Watir test to dedicated machines that report to you the results. The system also lets you VNC in and watch your tests while they are running in the browser, and it even sends screencasts of failures to easily diagnose what went wrong.
In the meantime, Steven Grimm leads an effort to improve our PHP unit test framework. Accompanying a new system for organizing our PHP code into “flib modules,” it’s easy to ensure specific targeted tests always run whenever an associated module is changed. In addition, all the tests run constantly in various environments, and automatically alert stakeholders any time a test fails. Database “shim” and function “stub” libraries are devised and perfected by Eugene Letuchy to help guarantee the behavior of the functional parts of each test case that they aren’t explicitly testing. Eventually so many tests are running with each new revision that systems are built to farm the tests out to several machines at once so the engineer doesn’t have to wait too long to see them pass. Things are getting pretty nice.
Watir, Watir Everywhere
It’s June 2011 and I’m now working out of our first remote engineering office in Seattle. I’ve spent the last year focusing on developers.facebook.com, and I’m starting to consider what I should choose for my next endeavor. f8 is looming and while I’m pondering my next big project, I decide it’d be helpful to try writing some more Watir tests for our social widgets such as the Comments plugin or the Like button. I figure a series of simple browser-based tests could go a long way to helping ensure these services never fail.
After a week or so of trying things out I find myself frustrated. The Watir we adopted in 2009 has since been improved, integrating a new protocol called WebDriver which allows you to automate the browser more accurately and with more power than could previously be done with straight JavaScript. Michael Stockton of the Mobile team is already at work upgrading our Watir infrastructure to use WebDriver, which he’s dubbed “fatir” (we love to name things by prepending an “f”). But Michael has thus far focused on making it work for mobile environments, which means if I want to get started following his lead I’ve got some work to do getting things working with standard desktop browsers like Firefox.
Unlike Watir, the PHP system for our direct unit tests was much more integrated with our development flow. Watir tests never ran automatically when you made changes, which meant that instead of seeing the test failure immediately you’d get notified of it potentially hours later. In addition, Watir was the only significant piece of Ruby code in our entire company. To try and leverage the lion’s share of our test libraries written in PHP, we had a clumsy RPC system which was almost as much work as just rewriting things in Ruby. These sorts of barriers to entry were formidable when, as an engineer, you just wanted to write a few unit tests for your features on the website. This left Watir relegated to being a second class citizen in our test architecture, overlooked and disliked.
In 2009, when Watir’s driver was JavaScript, the interface was heavily intertwined. There was no “peeling” the JavaScript interface off the Ruby Watir framework. But WebDriver had introduced the potential for language agnosticism, since its protocol was nothing but simple HTTP requests.
From Watir to WebDriver
Once I understood all these factors, my course of action seemed clear: Abandon Watir, create a WebDriver client in PHP, and reuse our mature PHP unit test system for our browser tests. The WebDriver protocol was straightforward enough that I had a fully featured client in PHP by the end of a weekend’s work (https://github.com/facebook/php-webdriver).
This endeavor is just beginning and I’m excited to see it unfold. Soon we’ll hold a company-wide integration test day where we’ll invite interested engineers to come learn about the new framework and how to port their existing Watir tests to begin writing new, better tests for the future.
Facebook’s ability to pivot onto new technologies and abandon current processes when better alternatives are available is one facet that makes me love working here so much. No system is sacred or retained for emotional or legacy reasons. Brian Rosenthal himself, the original implementer of our Watir framework, was one of the first advocates of our transition to WebDriver. Everything must improve, and people are not afraid to roll up their sleeves and start over when the time is right. The quality of our product overshadows any other priority. As a Facebook employee, there’s nothing I love more than our propensity for change and how our rapid development cycle lets us innovate quickly. While not every change has been perfect, our morale is upheld by the confidence that whatever we got wrong, we’ll change it again, and again, until we get it right.
Justin is a software engineer on the Platform team.
