When I think about automation, the first thing that pops into my head is a giant warehouse teeming with robots that scurry around, filling orders and shipping them off to fulfill the whims of internet shoppers. Something like this:
It’s not exactly the Jetson’s yet, but they’re getting there. And robots are cool.
But automation really applies to anything that can save us time, reduce errors and make us more efficient (which is code for saving money). The best targets for automation are tasks that are well defined, repetitive and common. In other words, chores that are boring, tedious and error prone. Fortunately for developers, we have plenty of these targets.
Our primary goal here is to automate the journey from code repository to finished product. A developer should be able to check code into the git repo and the system will build an installable product with no human intervention.
Benefits of Automation — Fast, Consistent, Correct
Let’s look more specifically at what automation can do for developers. At Oak City Labs, we build mobile apps and their backend servers that live in The Cloud. What sort of concrete things do we get from automation? We’re able to save time, ensure dependability and have confidence in every build. Automation creates builds that are fast, consistent and correct.
Saving time is the most obvious benefit of the whole process. When I have a bug to fix, I track it down and fix the errors on my laptop. Once I check that fix into git, I’m done. The automation kicks in, notices the change, runs through the build process and uploads the fixed app to a beta distribution service. Our QA folks get an email that the new app is ready to install and test. That whole process takes 30 minutes, but I’m free and clear as soon as I check into git. That’s 30 minutes of waiting for builds, running tests, and waiting for uploads that I don’t have to worry about or monitor. I’m on to tracking down the next bug. Saving half an hour a couple of times a week adds up, and sometimes it’s more like a couple of times a day. With those extra hours, I can fix more bugs and write more tests!
Less obvious than the time savings is consistency. Automation is codified process, so these builds happen exactly the same way each time. The automation always takes the same steps, in the same order, in the same context every time. Doing builds manually, I might use my laptop or my desktop, which are mostly the same, but not quite. Because I’m in a hurry, I might forget one of those simple steps, like tagging the repo with the build number, which won’t matter until I try to backtrack a buggy build later. With automation, we just don’t have those worries. Even better, I can go on vacation. Any of the developers on our team can build the app correctly. If the client needs a trivial fix, like changing a copyright date, it’s simple. Anyone on the team can update the text in the code repository and a few minutes later, a build is ready to test. Not only does the automation reduce the chances of human error, but it makes sure we no longer have to rely on a particular human to operate the controls.
Along with consistency, we also have confidence in every build. Consistency builds confidence, but so does testing and regimen. We build our software with a healthy dose of testing. As part of our automated builds, the testing ensures that the code behaves as we expect and that a change in one part of the code hasn’t inadvertently caused an error elsewhere. Of course we don’t catch every bug, but when there is a bug, we add a test to make sure we don’t make that mistake again.
Our automation is a tool we use every day as part of our development habits. It’s not a special task that’s only run at the full moon to bless our new release. It’s our daily driver, that reliable Toyota Camry we drive every day that always runs. It might need a little maintenance now and then, but it’s not your crazy uncle’s antique Mustang that only works a third of the time when he tries to take it out on a Saturday afternoon. This is really important when crisis mode comes around. Imagine if the app has some critical bug that needs to be patched ASAP. Because we have confidence in our consistent and correct automation, we can focus on fixing the bug and know that once it’s fixed, releasing the new version to users will be a smooth standard procedure.
Automation has been something we’ve grown to rely on in our development process because it make us more efficient and saves us time. For our clients, saving time means saving them money. We can get more work done because we can focus on the real challenge of writing apps and leave the boring tedium to our trusted automation. With the consistency and confidence that automation adds to our workflow, we can always be proud to deliver a top notch product to our clients. Fast, consistent and correct — automation delivers all three!
How the Magic Happens
So… automation is great and wonderful and makes the grass greener and the sun shine brighter, but how does it work? We’ve been talking in very vague terms so far, so let’s get down to the bits and bytes of how to put it all together.
For the last several years, we’ve been using a git branching strategy based on the excellent git-flow model. You can read all the brilliant details in the link, but the short story is that you have two long lived branches, master and dev. The master branch contains production level code that is always ready to release. The dev branch is the main development version that gets new features and maintenance fixes. Once a new feature set is ready in dev, it gets merged into master. This maps very nicely onto our concrete goals for automation.
Our projects have two deployment modes: beta and production. Beta is code from the dev branch. This is code ready for internal testing. For mobile apps, beta builds are distributed to QA devices for testing against the staging server. For server apps, beta builds are deployed to the staging server for testing before rolling out to production. Production mode is the real deal. Production mobile apps go to the app stores and to real users. Production server apps are rolled out to public servers ready to support users.
The automation workflow maps from git-flow into our deployment environments with the dev branch always feeding the beta environment and the master branch feeding the production environment.
The engine we use for our automation is a continuous integration server called TeamCity from JetBrains. It’s a commercial product, but it’s free for small installations. TeamCity coordinates the automation workflow by
- monitoring both
masteranddevbranches in git - downloading new code
- building the new version of the app
- running all the tests
- deploying to beta or production
If any of those steps fail, the process stops and the alarms go off. Our dev team is alerted via email and Slack and we’re on the problem like minions on a banana.
The last three steps are specific to the product type. For iOS, we use the wonderful Fastlane tools to orchestrate building the app, running tests, and uploading to either Fabric’s Crashlytics beta distribution service or to Apple’s App Store for production release.
We use Fastlane for Android, as well, so the flow is very similar. Beta releases go to Crashlytics, but production releases are shipped to the Google Play Store.
Our servers are written in Python and our web applications are in Angular. Testing for both uses the respective native testing tools, driven from TeamCity. For deployment, we use Ansible to push new versions to our beta and production clusters. We love Ansible because it’s simple and only requires an ssh connection to a node for complete management. Also, since Ansible is Python, it’s easy to extend when we need to do something special.
Since all of our build paths go through TeamCity, the TC dashboard is a great place to get a quick rundown of the build status across all projects.
With TeamCity coordinating our workflow automation and using tools like Ansible and Fastlane to enable builds and deployments, we’ve been able to build a system that is fast, consistent and correct, relieving us of the tedium of builds and letting us focus on the hard problems of building great apps for ourselves and our awesome clients.
