TESTING JAVASCRIPT… WITH RUBY?!

The curious case of non-bundled dependencies

With the above requirements comes a whole set of challenges not usually present in a client-side project, and one class of those issues has to do with testing. Of course, we have an impeccable set of unit tests and we’re running them against a very large matrix of browser / operating system combos in our CI/CD environment, but that alone does not explore everything that can go wrong.

Due to the overarching architecture of the ecosystem we’re running in, we depend on some external libraries being loaded alongside ours – that is to say, we do not bundle them with our code; we can’t and there’s nothing to be done about it. That presents an interesting challenge, because those libraries:

• might not even be there – if someone messes up implementing our library,
• might be there, but in wrong/incompatible versions,
• might have been modified by some other code that’s along for the ride in a particular implementation.

This shows clearly why unit tests are not enough: they test in isolation from the real world. Say we mock up some part of some external library’s public API, based on what we’ve discovered in it’s docs, and run an unit test against that. What does that prove?

You might be tempted to say “that means it works with the external library’s API”, but you’d be – sadly – wrong. It only means that it interacts correctly with a subset of the external library’s public API, and even then only with the version that we’ve mocked up.

What if the library literally changes out from under us? What if – out there in the wild – it gets some weird responses that make it hit a different, undocumented code path? Can we even protect against that?

Reasonable protection

Not 100%, no – the environment is too complex for that. But we can be reasonably sure that everything works how it’s supposed to with some generalized examples of what might happen to our code in the wild: we can do integration testing. The unit tests ensure that our code runs properly internally, and integration tests need to ensure that we “talk” properly with the libraries we cannot control. And not with stubs of them, either – actual, live libraries.

We could just use one of the available integration test frameworks for JavaScript, build a simple HTML page, throw some calls to our library and the remote libraries on it, and give it a good workout. However we don’t want to inundate any of the remote services’ endpoints with calls generated by our CI/CD environments. It would mess with some stats, possibly break some things, and – last but not least – we wouldn’t be very nice making someone’s production a part of our tests.

But was integration testing something so complex even possible? Since Ruby is our first and foremost love, we fell back on our expertise and started thinking about how we usually do integration testing with remote services in Ruby projects. We might use something like the vcr gem to record what is happening once, then keep replaying it to our tests whenever needed.

Enter proxy

Internally vcr achieves this by proxying requests. That was our a-ha! moment. We needed to proxy every request that shouldn’t hit anything on the “real” internet to some stubbed responses. Then that incoming data will get handed to the external library and our code runs as usual.

When prototyping something that seems complicated, we often fall back on Ruby as a time-saving method. We decided to make a prototype test harness for our JavaScript in Ruby to see how well the proxy idea will work before committing to building something more complicated in (possibly) JavaScript. It turned out to be surprisingly simple. In fact it’s so simple that we’re going to build one in this article together. 🙂

Lights, camera… wait, we forgot the props!

Of course we won’t be dealing with the “real thing” – explaining even a bit of what we’re building is far beyond the scope of a blog post. We can build something quick and easy to stand in for the libraries in question and then focus more on the Ruby part.

First, we need something to stand in for the external library we’re dealing with. We need it to exhibit a couple of behaviors: it should contact an external service, emit an event here and there, and most of all – not be built with easy integration in mind 🙂

Here’s what we’ll use:

/* global XMLHttpRequest, Event */

const URL = 'https://poloniex.com/public/?command=returnTicker'
const METHOD = 'GET'

module.exports = {
fetch: function () {
var req = new XMLHttpRequest()
req.responseType = 'json'
req.open(METHOD, URL, true)
var doneEvent = new Event('example:fetched')

req.onreadystatechange = function (aEvt) {
  if (req.readyState === 4) {
    if (req.status === 200) {
      this.data = req.response
    } else {
      this.error = true
    }
    window.dispatchEvent(doneEvent)
  }
}.bind(this)

req.send(null)

},
error: false,
data: {}
}

You’ll note that it calls an open API for some data – in this case some cryptocurrency exchange rates, since that’s what’s all the rage nowadays This API does not expose a sandbox and it is rate-limited, which makes it a prime example of something that should not be actually hit in tests.

You might notice that this is in fact an NPM-compatible module, while I’ve hinted at the script we normally deal with not being available on NPM for easy bundling. For this demonstration it’s enough that it exhibits certain behavior, and I’d rather have ease of explanation here at the cost of oversimplification.

Inviting the actors

Now we also need something to stand in for our library. Again, we’ll keep the requirements simple: it needs to call our “external” library and do something with the output. For the sake of keeping the “testable” part simple, we’ll also have it do dual logging: both to console, which is a bit harder to read in specs, and to a globally available array.

window.remote = require('remote-calling-example')

window.failedMiserably = true
window.logs = []

function log (message) {
window.logs.push(message)
console.log(message)
}

window.addEventListener('example:fetched', function () {
if (window.remote.error) {
log('[EXAMPLE] Remote fetch failed')
window.failedMiserably = true
} else {
log('[EXAMPLE] Remote fetch successful')
log([EXAMPLE] BTC to ETH: ${window.remote.data.BTC_ETH.last})
}
})

window.remote.fetch()

I’m also keeping the behavior staggeringly simple on purpose. As it is, there’s only two actually interesting code paths to spec for, so we won’t get swept under an avalanche of specs as we progress through the build.

It all just snaps together

We’ll throw a simple HTML page up:

 <!DOCTYPE html>
 <html>
 <head>
   <title>Example page</title>
   <script type="text/javascript" src="./index.js"></script>
 </head>
 <body></body>
 </html>

For the purposes of this demo we’ll bundle our HTML and JavaScript together with Parcel, a very simple web app bundler. I like Parcel a lot for times like these, when I’m throwing together a quick example or hacking on a back-of-napkin class idea. When you’re doing something so simple that configuring Webpack would take longer than writing the code you want, it’s the best.

It’s also unobtrusive enough that when I want to switch to something that’s a bit more battle-tested I don’t have to do almost any backpedaling from Parcel, which is not something you could say about Webpack. Note of caution, however – Parcel is in heavy development and issues can and will present themselves; I’ve had an issue where the transpiled JavaScript output was invalid on an older Node.js. Bottom line: don’t make it a part of your production pipeline just yet, but give it a spin nonetheless.

Harnessing the power of integration

Now we can construct our test harness.

For the spec framework itself, we’ve used rspec. In development environments we test using actual, non-headless Chrome – the task of running and controlling that has fallen to watir (and it’s trusty sidekick watir-rspec). For our proxy, we’ve invited Puffing Billy and rack to the party. Finally, we want to rerun our JavaScript build every time we run the specs, and that is achieved with cocaine.

That’s a whole bunch of moving parts, and so our spec helper is somewhat… involved even in this simple example. Let’s take a look at it and pick it apart.

Dir['./spec/support/*/.rb'].each { |f| require f }

TEST_LOGGER = Logger.new(STDOUT)

RSpec.configure do |config|
config.before(:suite) { Cocaine::CommandLine.new('npm', 'run build', logger: TEST_LOGGER).run }

config.include Watir::RSpec::Helper
config.include Watir::RSpec::Matchers

config.include ProxySupport

config.order = :random
BrowserSupport.configure(config)
end

Billy.configure do |c|
c.cache = false
c.cacherequestheaders = false
c.persistcache = false
c.recordstubrequests = true
c.logger = Logger.new(File.expandpath('../log/billy.log', FILE))
end

Before the whole suite we’re running our custom build command through cocaine. That TEST_LOGGER constant might be a little bit of overkill, but we’re not very concerned about number of objects here. We’re of course running specs in random order, and we need to include all the goodies from watir-rspec. We also need to set up Billy so that it does no caching, but extensive logging to spec/log/billy.log. If you don’t know whether a request is actually getting stubbed or is hitting a live server (whoops!), this log is pure gold.

I’m sure that your keen eyes have already spotted ProxySupport and BrowserSupport. You might think that our custom goodies sit in there… and you’d be exactly right! Let’s see what BrowserSupport does first.

A browser, controlled

First, let’s introduce TempBrowser:

class TempBrowser
def get
@browser ||= Watir::Browser.new(web_driver)
end

def kill
@browser.close if @browser
@browser = nil
end

private

def web_driver
Selenium::WebDriver.for(:chrome, options: options)
end

def options
Selenium::WebDriver::Chrome::Options.new.tap do |options|
options.addargument '--auto-open-devtools-for-tabs'
options.addargument "--proxy-server=#{Billy.proxy.host}:#{Billy.proxy.port}"
end
end
end

Working backwards through the call tree, we can see that we’re setting up a Selenium browser options set for Chrome. One of the options we’re passing into it is instrumental in our setup: it instructs the Chrome instance to proxy everything through our Puffing Billy instance. The other option is just nice to have – every instance we run that’s not headless will have the inspection tools automatically open. That saves us uncountable amounts of Cmd+Alt+I’s per day 😉

After we set up the browser with these options, we pass it on to Watir and that’s pretty much it. The kill method is a bit of sugar which lets us repeatedly stop and restart the driver if we need to without throwing away the TempBrowser instance.

Now we can give our rspec examples a couple of superpowers. First of all, we get a nifty browser helper method which our specs will mostly revolve around. We can also avail ourselves of a handy method to restart the browser for a particular example if we’re doing something super-sensitive. Of course, we also want to kill the browser after the test suite is done, because under no circumstances do we want lingering Chrome instances – for the sake of our RAM.

module BrowserSupport
def self.browser
@browser ||= TempBrowser.new
end

def self.configure(config)
config.around(:each) do |example|
BrowserSupport.browser.kill if example.metadata[:clean]
@browser = BrowserSupport.browser.get
@browser.cookies.clear
@browser.driver.manage.timeouts.implicit_wait = 30
example.run
end

config.after(:suite) do
  BrowserSupport.browser.kill
end

end
end

Wiring up the proxy

We’ve got a browser and spec helpers set up, and we’re ready to start proxying requests to our proxy. But wait, we didn’t set it up yet! We could bang out repeated calls to Billy for each and every example, but it’s better to get ourselves a couple of helper methods and save a couple thousand keystrokes. That’s what ProxySupport does.

The one we use in our test setup is slightly more complex, but here’s a general idea:

frozenstringliteral: true

require 'json'

module ProxySupport
HEADERS = {
'Access-Control-Allow-Methods' => 'GET',
'Access-Control-Allow-Headers' => 'X-Requested-With, X-Prototype-Version, Content-Type',
'Access-Control-Allow-Origin' => '*'
}.freeze

def stubjson(url, file)
Billy.proxy.stub(url).andreturn({
body: open(file).read,
code: 200,
headers: HEADERS.dup
})
end

def stubstatus(url, status)
Billy.proxy.stub(url).andreturn({
body: '',
code: status,
headers: HEADERS.dup
})
end

def stubpage(url, file)
Billy.proxy.stub(url).andreturn(
body: open(file).read,
content_type: 'text/html',
code: 200
)
end

def stubjs(url, file)
Billy.proxy.stub(url).andreturn(
body: open(file).read,
content_type: 'application/javascript',
code: 200
)
end
end

We can stub:

• HTML page requests – for our main “playground” page,
• JS requests – to serve our bundled library,
• JSON requests – to stub the request to the remote API,
• and just a “whatever” request where we only care about returning a particular, non-200 HTTP response.

This will do nicely for our simple example. Speaking of examples – we should set up a couple!

Testing the good side

We need to wire together a couple of “routes” for our proxy first:

let(:pageurl) { 'http://myfancypage.local/index.html' }
let(:jsurl) { 'http://myfancypage.local/dist/remote-caller-example.js' }

let(:pagepath) { './dist/index.html' }
let(:jspath) { './dist/remote-caller-example.js' }

before do
stubpage pageurl, pagepath
stubjs jsurl, jspath
end

It’s worth noting that from rspec’s perspective the relative paths here refer to the main project directory, so we’re loading our HTML and JS straight from the dist directory – as built by Parcel. You can already see how those stub_* helpers come in handy.

It’s also worth noting that we’re placing our “fake” website on a .local TLD. That way any runaway requests should not escape our local environment should something go awry. As a general practice I’d recommend at least not using “real” domain names in stubs unless absolutely necessary.

Another note we should make here is about not repeating ourselves. As the proxy routing gets more complex, with a lot more paths and URLs, there’ll be some real value in extracting this setup to a shared context and simply including it as needed.

Now we can spec out how our “good” path should look like:

context 'with correct response' do
before do
stubjson %r{http://poloniex.com/public(.*)}, './spec/fixtures/remote.json'
goto pageurl
Watir::Wait.until { browser.execute_script('return window.logs.length === 2') }
end

it 'logs proper data' do
expect(browser.execute_script('return window.logs')).to(
eq(['[EXAMPLE] Remote fetch successful', '[EXAMPLE] BTC to ETH: 0.03619999'])
)
end
end

That’s pretty simple, isn’t it? Some more setup here – we stub the JSON response from the remote API with a fixture, go to our main URL and then… we wait.

The longest wait

The waits are a way of working around a limitation we’ve encountered with Watir – we can’t reliably wait for e.g. JavaScript events, so we need to cheat a bit and “wait” until the scripts have moved some object that we can access into a state that is of interest to us. The downside is that if that state never comes (due to a bug, for example) we need to wait for the watir waiter to time out. This drives the spec time up a bit. The spec still reliably fails though.

After the page has “stabilized” on the state that we’re interesting in, we can execute some more JavaScript in the context of the page. Here we call up the logs written to the public array and check whether they are what we expected.

As a side note – here’s where stubbing the remote request really shines. The response that gets logged to the console is dependent on the exchange rate returned by the remote API, so we couldn’t reliably test the log contents if they kept changing. There are ways of working around it of course but they aren’t very elegant.

Testing the bad branch

One more thing to test: the “failure” branch.

context 'with failed response' do
before do
stubstatus %r{http://poloniex.com/public(.*)}, 404
goto pageurl
Watir::Wait.until { browser.execute_script('return window.logs.length === 1') }
end

it 'logs failure' do
expect(browser.execute_script('return window.logs')).to(
eq(['[EXAMPLE] Remote fetch failed'])
)
end
end

It’s very similar to the above, with the difference being that we stub the response to return a 404 HTTP status code and expect a different log.

Let’s run our specs now.

% bundle exec rspec
Randomized with seed 63792
I, [2017-12-21T14:26:08.680953 #7303] INFO -- : Command :: npm run build

Remote calling
with correct response
logs proper data
with failed response
logs failure

Finished in 23.56 seconds (files took 0.86547 seconds to load)
2 examples, 0 failures

Woohoo!

Conclusion

We’ve briefly discussed how JavaScript can be integration tested with Ruby. While originally it was considered more of a stopgap, we’re pretty happy with our little prototype now. We’re still considering a pure JavaScript solution, of course, but in the meantime we have a simple and practical way of reproducing and testing some very complex situations that we’ve encountered in the wild.

If you’re considering building something similar yourself, it should be noted it’s not without its limitations. For example if what you’re testing gets really AJAX-heavy, Puffing Billy will take a long time to respond. Also if you have to stub some SSL sources some more fiddling will be required – look into the watir documentation if it’s a requirement you have. We’ll surely keep exploring and looking for the best ways to deal with our unique use case – and we’ll make sure to let you know what we found out, too.