I recently finished cleaning up our build system and adding support for both client-side and server-side unit tests.  The build is cross-platform, meaning our developers can build and run the application whether they are on Windows, OSX, or Linux.

The gulp build script uses:

• Webpack to build both the client and the server pieces, gathering and transforming all of the source code (ES6, SASS, etc) into deployment bundles.
• Mocha to run the server-side unit tests
• Karma to run the client-side unit tests

Once launched, the build script watches the source code and automatically recompiles, retests, and relaunches the app whenever anything changes.

Once I finished, it was a glorious thing to behold.  If I can pry myself away from watching it run, I might blog about some of the more interesting bits of it.

Alas even as I finished getting it all working, the decision was made to start building and running our app within Linux virtual machines.  We chose to use Vagrant to manage the VM configuration.

This was meant to simplify the development environment configuration.  Unfortunately it also completely broke the build.

Here are the problems we ran into, along with solutions.  It took me two days to track all of this down.  Hopefully I’ll save someone else the trouble.

Line Endings

This only affected our Windows developers.  We had already configured git with a .gitattributes file to normalize line endings within the repo.  For our Windows developers, files would get checked out with Windows-style line endings but get normalized with Unix-style line endings when commited to the repo.

Once we mounted the source directory as a shared folder within Vagrant and tried to get Vagrant to run some of the scripts, we discovered Vagrant would error on the Windows-style line endings.

Solution

Since we are using smart text editors (WebStorm) that can handle either line ending style, we reconfigured git to always checkout with Linux line endings.

This involved changing our .gitattributes file from:

[code] * text=auto [/code]

to:

[code] * text=auto eol=lf [/code]

Just for added measure, we also added an .editorconfig file that would force supporting text editors to conform to our convention:

[code] root = true# By default, use Linux line endings (matches our .gitattributes settings)# Ensure file ends in a newline[*]end_of_line = lfinsert_final_newline = truetrim_trailing_whitespace = truecharset = utf-8 [/code]

Symbolic Links

Another problem that only affected Windows developers. Windows ships with the permission to create symbolic links turned off for users. However, npm loves to use symbolic links. So when you try to run npm install on the virtual machine within the shared folder, you end up getting errors because the Windows host will not allow the symbolic links to be created.

Solution

There are two steps to the solution:

Give yourself permission to create symlinks

• Start -> Run: secpol.msc
• Browse to: Security Settings -> Local Policies -> User Rights Assignments
• Double-click Create symbolic links
• Add yourself to the list of allowed users

Disable UAC

If you are not an administrator then you can skip this step. If you are an administrator, then this step is required because otherwise you need to elevate permissions via the UAC prompt before making symlinks. And vagrant won’t trigger the UAC prompt and so it just errors out.

Disable UAC:

• Start -> Control Panel -> Uer Accounts
• Change User Account Control settings
• Slide the slider down to Never notify

Finally, reboot for these changes to take effect.

Long Paths

Yet another Windows-only issue.  npm loves making files in deeply nested folders.  Again, if you are running npm install on the virtual machine within a folder shared with the host, then npm might end up making a path that is too long for Windows, causing an error.

Solution

Windows actually supports really long paths (32k characters IIRC). You just have to either use their special API, or you need to use special UNC paths. Neither Vagrant, nor our VM provider (VirtualBox) were doing this though.

Luckily a commit was recently made to Vagrant. When the next version (1.7.3) is released, this will no longer be an issue. Until then, if this gets in your way, you can apply the commit manually to your Vagrant install.

This commit changes 3 files. Just make the same change to your local Vagrant installation. No need to recompile or anything like that. With a default installation, you’ll find the files somewhere beneath C:\HashiCorp\Vagrant\embedded\gems\gems\vagrant-1.7.2.

Watch Does Not Work on Shared Folder

This issue affected all of our developers, not just Windows.  It turns out that the shared folder does not support fsevents flowing from the Host to the Client.  This means that if your VM client is running watch (for example, webpack –watch) and you edit a file from the host machine, the watcher will never see the activity and your build will just sit there not recompiling.

If you edit the file from the VM itself, then everything works OK.

Solution

Webpack recently added a polling option to their watch logic.  All we had to do was upgrade to the most recent version of webpack (we were using the positively ancient version from a month ago), and modify our configuration to supply a poll interval.  This causes the webpack watcher to poll for changes instead of relying upon fsevents.

[code language="javascript"]webpack(webpackConfig)    .watch({        aggregateTimeout: 300,        poll: true    }, callback);[/code]

Once we did this, our watch was working again.

Headless Chrome

Finally, our Karma configuration was set to launch instances of Chrome for the client-side unit testing. But our VM neither had Chrome installed nor a way to show it if it was installed.

Solution

I found this blog article which talked about running headless Chrome with Selenium on Vagrant.

While we aren’t using Selenium, the core principles applied.

So I added this vagrant provisioner to install Chrome, xvfb, and Java:

[code language="bash"]# Add Google public key to aptwget -q -O - "https://dl-ssl.google.com/linux/linux_signing_key.pub" | sudo apt-key add -# Add Google to the apt-get source listecho 'deb http://dl.google.com/linux/chrome/deb/ stable main' >> /etc/apt/sources.list# Update app-getapt-get update# Install Java, Chrome, Xvfb, and unzipapt-get -y install openjdk-7-jre google-chrome-stable xvfb# Override the command Karma uses to launch chrome so that we can run it headlessecho "export CHROME_BIN=/vagrant/vagrant-files/headless-chrome" >> /home/vagrant/.bashrc[/code]

Notice the last line specifies a command headless-chrome. This is a custom script which will launch xvfb and then chrome. It will listen for the kill signal from Karma and then shut everything down correctly.

The inspiration for this script came from docker-chromium-xvfb.

[code language="bash"]#!/bin/bash_kill_procs() {  kill -TERM $chromium  wait $chromium  kill -TERM $xvfb}# Setup a trap to catch SIGTERM and relay it to child processestrap _kill_procs SIGTERM# Start XvfbXvfb :99 -screen 0 640x480x8 -nolisten tcp &xvfb=$!export DISPLAY=:99google-chrome "$@" &chromium=$!wait $chromiumwait $xvfb[/code]

Conclusion

Those are all the problems we had to work through. It was rough going but now we have a system that is easy to reconfigure and deploy to our developers.