Will Richardson

Blog Me Twitter GitHub

Why I Dislike ATDD

This was written as the final section to a university lab report on testing, ATDD, and mocking.

Both cucumber and concordion aim to make it easier to write more understandable tests at a higher level - instead of writing unit tests that test very specific and granular aspects of a class, the acceptance tests ensure that the feature behaves as expected for the end user.

At my internship over the summer, I worked on an open source project management system called Redmine, and some of its plugins. The Redmine Backlogs plugin adds agile functionality to Redmine, and has a massive suite of Cucumber tests that I had to maintain. After seeing the ‘bad side’ of computer evaluated acceptance tests and ATDD, I am very sceptical to the benefits of cucumber - and have major doubts in concordion.

The Backlogs tests consisted of about 20 feature files, each ranging from 1-2 scenarios, up to about 6. This could be about 200 lines of steps. The actual definitions were split into 3 files (given, when, and then steps - it was a Ruby project so it isn’t as strict as the Java implementation). These were about 1500 lines each.

Imagine the following scenario: you’re tasked with making the tests pass after some feature was added, or a change in the environment caused them to fail. Running the tests reveals which of the scenarios is failing, and you have a line in a feature file that is causing it to fail. Due to the fact that the actual definition of the step is defined by a regular expression, you can’t find it by simply searching for the line in the feature. Eventually you find it somehow - probably by doing a regex search for something similar to the step text.

Now that you’ve found the step definition, you can debug that step - or any of the steps above or below it in the scenario (which you have to find by repeating the same ordeal outlined before). You fix the scenario and any others that were affected by the change. You decide that it’s good practice to write a new test that tests the feature that was just added.

Here you have the reverse problem from debugging - you don’t know what steps have been defined to create the new test. Your IDE or editor likely doesn’t have any kind of autocomplete to help you fill out the steps in the scenario. Instead you add a expression to the step definition files that will be used in just your test - adding to the mass of bespoke step definitions already written.

This is obviously the worst case of cucumber or any ATDD framework. On the flip side, I created my own plugin for Redmine while I was working. When it came time to test it, we decided that cucumber would be easiest - the whole team understood it and it was already setup for one plugin, so the amount of work needed to get it working on another was minimal.

Working on another project from scratch, cucumber was very easy to use - I knew off the top of my head every valid step definition and the options that I could give it. When creating my own definitions, I could write them in such a way so that they could be reused and extended later to test different situations. Obviously this is the difference between knowing a codebase and being completely new to it, as well as the worst type of codebase - an unmaintained open source project - versus the easiest to understand - a small project by one developer, which is you.

Even knowing that I was working in the worst case, I am sceptical to the benefits of computer evaluated acceptance tests. Talking to Sam - a coworker over the summer, and all-round testing guru - he says that the idea of cucumber is flawed to begin with. It assumes that the client or PO will provide acceptance criteria detailed enough to test the feature sufficiently and specific enough to be turned into valid cucumber instructions. If I was working with a PO that did give this quality acceptance critera, I would jump to cucumber almost immediately

Concordion on the other hand, completely stumps me. I understand that having nicely formatted results that can be shown off to stakeholders could prove useful, however the overhead required to test using concordion seems to be through the roof for little or no gain. In a nutshell what concordion appears to do is take all the assertions out of a normal jUnit test and put them inline in HTML elements. Once again this disconnect between the actual code and the expected results would make it harder to maintain and debug tests. In my mind cucumber is better because the content of the feature files is just the description and expected result, whereas the concodion files mix the description and tests with the layout of the result.

It seems like the end result of concordion could be acheived by parsing jUnit tests with a known format of JavaDoc and assertion messages. These could be parsed as the tests were run and then generate an HTML file - much like a JavaDoc - with the test results, which then can be styled appropriately. In fact, this could probably be done with annotations and reflection, without the need to parse the test code manually.

So far my thoughts on ATDD is that developers should spend time doing what they are best at, with the tools that they work best with - nine times out of ten this is writing code in their preferred IDE, not writing english or HTML-jUnit hybrids that will be run as tests. Perhaps my view of ATDD is skewed because I first used cucumber in the worst possible way. If I do end up using ATDD as part of my group project, I hope it is well managed and used appropriately - maybe I will come around to this way of testing.

Why I use Nginx

There are two very important reasons why I use Nginx to run my website:

  1. It was the first thing I used
  2. It has smaller config files than Apache

Even though I have been using it for quite some time, I didn’t really understand it - until I setup a second static hosting domain to host a Jenkins theme, which made me realise it’s not too bad.

The css would only be applied if the http headers were correct (ie it had text/css rather than just text/plain). Files servered though GitLab’s ‘raw’ mode have a text/plain header.

So this is my nginx config file, in sections.

http {
  include /etc/nginx/mime.types;
  passenger_root # Path to the passenger gem;
  passenger_ruby # Path to the ruby shim, from rbenv;

All of my config is in the http section. I’d guess that I can have other sections for different protocols, but this is just a basic web server so all I need is HTTP.

The include mime types line will make nginx serve static files with the correct Content-Type header for the file extension, which is why serving from this works for my Jenkins server but GitLab doesn’t.

server {
  location / {
    root /var/www/blog;

This section defines a default server - anything that doesn’t match will just be sent to this, for example foobar.javanut.net will just go to the main blog. I could add more things in here if I wanted a subsection to go to somewhere else - maybe I wanted to serve some other content at javanut.net/my_stuff. I could just make a new location block and set the root to be a different location on my server.

server {
  listen 80;
  server_name static.javanut.net;
  root /var/www/static;

This is basically the same as the previous section, it’s just another static file server that points to a different folder. The main point here is that the server_name has been set, so that it is only accessible on static.javanut.net. In the previous example, the location {} block is probably unnecessary as it isn’t needed here.

  server {
    listen 80;
    server_name my_rails_app.javanut.net;
    root /var/www/my_rails_app/current/public;
    passenger_enabled on;

Again this is very similar, but this is for a Rails app using passenger. Passenger needs to be installed when nginx is compiled - there is no plugin system for nginx.

Enumerating The Ways I Love Swift Enumerations

As Casey illustrates, enums in Swift really are quite awesome and really powerful once you get used to them.

When I was working on WORM I kept wanting to make things enums. Every time I did try I wanted to attach values to each of the options. For example I created an @Stored annotation, which you could either tell it to work out the type for you, or to have a custom type string when it created the table. In Swift I could have done:

enum ColumnType {
  case Infer
  case Custom(type: String)

What is interesting is that I thought of the Swift solution before coming up with a Java version - my ‘native’ language.

Testing GitLab CI

During my internship this summer I found myself pining for a continuous integration server. The project I was working on had a massive set of Cucumber tests. The only problem was that they took 40 minutes to run completely, which is a bit too much of a pain to actually run them regularly on a local machine. Last semester for my software engineering group project, we were given Jenkins servers to run our tests on - this enforced the habit of keeping the tests up to date and fixing anything that breaks them.

I started looking around out of curiosity to see what else there was apart from Jenkins, which was not the most friendly thing to set up at the start of the project. After a little search I came across GitLab CI which integrates right into GitLab (obviously) and is written in Go which makes it quite cool right off the bat.

GitLab CI can be the simplest build server that you could imagine - it can be easily set to just run a shell script when a commit is pushed, and if the exit status is zero it succeeded, if it’s non-zero then it failed. This basically means that you don’t have to learn a new configuration syntax to do anything (you can, but it’s definitely not needed). If you can run your tests from the command line, you’re good to go.

Once it has been set up, every commit to the repo will trigger a build on your server and the result will be displayed in the ‘builds’ tab of GitLab and when you view the commit. This can be done with either GitLab.com or a different hosted instance of GitLab.

Full installation instructions for CI Runner are on GitLab’s website. However it’s as simple installing a package and running the setup (Instructions for Ubuntu, other distros on GitLab’s website)

# Add the source to apt-get:
curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.deb.sh | sudo bash
# Install the runner package
sudo apt-get install gitlab-ci-multi-runner
# Run the setup
gitlab-ci-multi-runner register

The instructions say to run the last command with sudo, but when I did this my config file was set to be in /etc/gitlab-runner/config.toml rather than the expected ~/.gitlab-runner/config.toml.

The register command sets up a runner to point to a certain GitLab url (either GitLab.com or your custom instance) and the token needed to pull your code. I setup mine with:

URL: https://gitlab.com/ci
Token: ~~ secret token ~~ # Accessed in the main project settings
Description: Test runner
Executor: shell

I made a quick branch on one of my projects that has fair number of unit tests that are easily run. All I had to do was add a .gitlab-ci.yml file:

  script: "mvn package -B"

maven-package is the name of the build process, and the script key denotes either a single bash command or a list of commands. Once this was pushed to GitLab a build immediately started.

And failed instantly. Thankfully a full log gets output to the web interface and I could see that the runner was getting confused trying to load up a Docker instance, even though I didn’t configure that. So once I’d found the config file location (which wasn’t where I expected, as I mentioned before) and deleted all the entries apart from the main [[runners]] section (getting rid of the [[runners.docker]] section probably would have been enough). Once I’d made this change the build completed successfully.

Right now I’m very impressed with the ease of setting up a GitLab CI Runner and will definitely use one in the future (especially if I get a scooter computer) for the odd occasion that I write unit tests. However if I did set up a CI server I would want to make sure the gitlab-runner user had as few permissions as possible - probably only able to read or write within their own home directory - so that the chance of breaking my setup is reduced.

4K Video Editing on a 12" MacBook?

Of course the difference between Final Cut on one platform vs cross-platform Adobe Premiere is making the main difference, I think this really illustrates the advantage of software running on hardware that it’s expecting.

Welcome to Swift.org

Swift is now open source!

Finally I can start having a more serious look at making something with Taylor and deploying it onto something other than my laptop. At work this morning I downloaded the Swift binary and fired up the REPL. Fully functioning Swift on Ubuntu. The future is now.

Perhaps more interesting than the actual Swift repository is the Swift Evolution page that publicly shows the features and direction that both Apple and Swift community want the language to head in. It makes me very excited to see speed, portibility and API design among the goals for version 3 and beyond. This could mean more consistent APIs and a global Foundation library that wraps the native functions for each system (at the moment pre-processor commands are needed to use platform-specific libraries) which is not very Swift-y.

The first commit to the Swift project is dated July 18, 2010. It’s crazy to think that this was kept completely secret for four years before it was unveiled. Also pointed out in the comments is that Swift was named Swift since its inception.

Along with the dump of projects released this morning is the Swift Package Manager. I am probably far too excited about this that it is normal to be for a tool that I haven’t really looked at yet. However because of the pain that CocoaPods has caused me while trying to write unit tests that access a database, I’m happy to see a first party solution - and will be updating my version of SQLite.swift as soon as I can.

Life with Swift

Since Apple introduced Swift at WWDC last year, I’ve been interested in it as a compiled language that seems as easy and quick to develop as a dynamic scripting language like Python. Especially that Swift will (hopefully) be open sourced late this year, meaning that it could be used to develop applications that could be deployed easily onto a webserver as a simple binary (no Capistrano necessary).

Swift’s basic syntax is incredibly clean and easy to get your head around. Keywords take second place to syntactic symbols - extending a class is done with a colon: class Subclass: Superclass, Protocols {} rather than the more verbose Java syntax class Subclass extends Superclass implements Interfaces {}. I like this both because of the reduced typing but also how the colon is reused to set the type in all instances where a type is needed.

var str: String // variable initialisation
func things(number: Int) // argument definition

This is not the case for return values though. It would make sense that like a variable, a function should have a type attached to it. This is not the case, instead a one-off symbol is used: func getNumber() -> Int {}. This would be nicer and more consistent if it used the same style: func getNumber(): Int {}.

Swift’s optional types are very convenient and make code more explicit - being forced to unwrap values that could be nil makes writing code that deals with user input or stored values a whole lot cleaner. For example if you read a number from a text field and need to turn it into an int, Int(myString) returns an optional int, it may or may not be nil. You can then unwrap it:

if let number = Int(myString) {
    // Do something with the number

This is really handy, and extends to almost all parts of the language and the Cocoa API. This can be further enhanced by using optional chaining - adding the ? operator on to the optional value allows you to call methods on optional values as though they were definite values. The value returned by the last method is always an optional if you do this. For example if you have a dictionary of strings and you want to get one lowercased.

let lowercased = myDict[key]?.lowercaseString

Where this falls down is if the key is an optional value as well - you can’t index a dictionary with an optional value if the key isn’t optional. What I would like to do would be to use the question mark to maybe unwrap the key, and if it isn’t nil, then use the key to look up an item in the dictionary. Like this:

let value = myDict[key?]

But you can’t do that. The closest you can get is something like this:

if let k = key, let value = myDict[k] {
  // value is a definite value that is in the dictionary
} else {
  // either key is nil, or there is no value in the dictionary to match it

What makes Swift that bit cooler than other languages that I’ve dabbled in is that it has the standard functional programming functions - map, filter, and reduce - which makes working with arrays a whole lot less cumbersome for anyone with a bit of functional programming prowess. Paired with the powerful closure support, it’s easy to express an operation in terms of a few closures. To turn a list of strings into a list of all the ones that can be turned into ints you can just map and filter them:

let nums = myStringList.map({ str in
}).filter({ possibleInt in
    possibleInt != nil

To sum these you can use the name-less closure syntax:

nums.reduce(0, { $0 + $1 })

None of this would be possible without Swift’s type system. When first looking at Swift I thought that it was simply statically typed like Java, except you didn’t have to explicitly declare the type of variables - they would be set for you if the compiler could work it out. However Swift can behave somewhat like Haskell’s types to create functions that don’t just work on on a string or a number, but any type that implements a certain protocol.

In Haskell you might come across something like:

isSmaller :: (Ord a) => a -> a -> Bool
isSmaller a b = a < b

Which uses the orderable (Ord) type class to declare a function that can be used on any type that supports ordering - strings, characters, numbers, etc. Swift has an expanse of built-in protocols that let you do similar things. For example, I wanted to be able to do set operations on lists while keeping the order of the elements, so I made an extension that would extend an array of elements that implemented the Hashable protocol - meaning that the contents of the array could be put into a set.

extension Array where Element: Hashable {
  func unique() -> [Element] {
    var seen: [Element:Bool] = [:]
    return self.filter({ seen.updateValue(true, forKey: $0) == nil })

  func subtract(takeAway: [Element]) -> [Element] {
    let set = Set(takeAway)
    return self.filter({ !set.contains($0) })

  func intersect(with: [Element]) -> [Element] {
    let set = Set(with)
    return self.filter({ set.contains($0) })

These functions will be added to any array that contains elements that are hashable - if they aren’t, then I simply can’t use the functions. The more I get used to things like this, the more I like programming in Swift. It successfully combines the things I like in many different languages into one - it’s compiled, quick to write, allows for functional programming as well as rigid object-oriented structures and the ability to extend the language itself seamlessly.

Not Enough Magic

There were some minor murmurings this week after Apple released their new peripherals - The new (Magic) Trackpad, Mouse and Keyboard. (This was mostly drowned out by the far more significant news that the new iMac has a platter HDD). A very vocal portion of the internet were outraged that the Magic Mouse has a charging port on the bottom:

Magic Mouse with port on bottom

It’s easy to quickly dismiss this as a stupid decision and go about your day. But the mouse is supposed to charge enough for 8 hours of use in one minute or overnight for three months of use. This means that you will only ever see it awkwardly upside down charging less than 2% of the time - and when you’re using it the other 98% of the time you will be magical mouse that has no visible charging port. Had Apple opted to place the port on the front, the upper glass surface wouldn’t be able to dip down nearly as far as it does on the current and previous models. It also saves the mouse from having a little pointy nose/ mouth at the front - which would have made the previous model look far better in comparison.

Yesterday the batteries in my keyboard ran out, and so I had to stop using my computer and wait for them to charge overnight (Or bring out my classic wired keyboard, but that’s a bit too far). It would have been great to be able to simply plug in whatever device is running out of power for literally one minute and be able to continue working as normal. Having in-built batteries also means that remaining charge estimates can be far more accurate - my keyboard and trackpad have no idea whether they have 1.5V AAs or 1.2V rechargables, meaning that the drainage percentage is almost always off.

What the real concern should be with these new peripherals is what the lifetime of the batteries will be. Coupling the battery to the device means that as the battery degrades the device becomes more and more painful to use. You shouldn’t have to buy a new keyboard because your old one can’t keep charged any more. I own three wireless input devices, all of them over 6 years old. This isn’t a problem for this generation of devices because their batteries are replaceable and it’s just a matter of clicking a new set in.

OS X El Capitan

El Capitan default wallpaper

On Thursday I bit the bullet and trashed my home data cap by upgrading to El Capitan. The 6 GB download crawled down at a snails pace overnight, but when I got up in the morning my MacBook was waiting for me with a new version of OS X.

At first look, El Capitan appears no different to Yosemite - it has the same window styles, menu bar, and login screen. For me, the most notable change is the new Mission Control layout. Instead of showing the bar of desktops along the top with a thumbnail, there is just a thin list of labels. When you move your mouse close to the top of the screen this will expand into a set of thumbnails. Once it has been expanded it will remain so until you exit Mission Control. Much like the dock, I find it to always appear when I want it to, and as a spaces ‘Power User’ I have not been frustrated by this at all.

Along with the new Mission Control there is of course split view. This only works on applications that can be freely resized - MailBox will not split, but editors, terminals and browsers all split just fine. Windows can either by split by dragging the window into an already full screen application in Mission Control, or by clicking and holding on the green fullscreen button before dragging the window to either the right or the left - once that window is split you can select the window that will occupy the other half of the screen.

The whole splitting process seems a bit janky - press and hold doesn’t feel quite right on a trackpad (My guess is that it’s designed to be Force Touched) and once two windows are split together, you can’t substitute one with another application without exiting the split and then re-adding the new app. Neither Apple’s nor Microsoft’s can be said to be better, it entirely depends on how the user ‘maps’ its functions in their mind as to which will be more useful for them.

When I set out to do some work, I was glad to see that Ruby and related gems, Python, Brew, Java, MySQL and Postgres all made the transition without any immediately visible issues. This is usually the factor that makes me wait before upgrading. My development setup wasn’t completely untouched though - it appears that the way fonts (or at least monospaced fonts) are rendered has changed, so that the text appears to be thinner in most cases. In IntelliJ it looks like someone turned of LCD font smoothing off (it is still turned on in Preferences). In both TextMate and the Terminal it is better looking. I’m yet to find out if this is a system-wide change that effects all fonts or just my editor font of choice Anonymous Pro behaving badly.

Rummaging through the preferences, I only found one notable change - it is now possible to auto-hide the menu bar (General -> Automatically show and hide the Menu Bar). After turning it on for two seconds, I am of the opinion that this is an awful feature that no one should use.

El Capitan is definitely worth the upgrade for the sake of being up-to-date and brings some nice features to make the download worth the wait.

If you want a more Siracusian review, Ars Technica has picked up John’s baton to prove a more in-depth look at the latest iteration of OS X.

Absurd infinitum: Deliberately misunderstanding Steve Jobs

It would seem that some people do not understand the difference between wanting to wear high heels to a weekend wedding and having to wear them all day every day at your job at the cranberry farm.

Excellent point that Apple hasn’t blown the iPad Pro by selling a stylus Pencil.