A whole day devoted to Gradle – what fun!
The fact that the event took place shows how much traction, and how prominent Gradle is becoming in the build community. It’s becoming the de facto build tool for Android, and its support for other platforms and languages is becoming richer and richer.
Here at Caplin, we’re big fans of Gradle. We’ve been using Ant and Maven for years, but a lot of build engineers have prematurely lost their hair trying to unravel messy pom and build.xml files.
A lot of Caplin’s newer builds have been written in scratch in Gradle and it works well, particularly on our simple Java projects.
Gradle comes into its own for more complicated tasks though. There are a multitude of plug-ins available, plus Ant tasks are available for free. And if you can’t find a way to do something, well, you can of course just write a plug-in to do it.
Our conundrum is finding the balance between Gradle’s flexibility and trying to introduce some general company standards. We like to give development teams the responsibility of writing build scripts, but each team have their own way of solving the same problem. This is incredibly inefficient.
As part of the Release Engineering Team, I’d like to promote best practise for using Gradle, and encourage sharing of our good work via standard project templates we can use, plus company-wide plug-ins that can be shared across teams.
That’s where I hoped GradleX would help!
The Exchange
Unfortunately it was the day of the ‘big storm’ in London. Kick-off was delayed until 10.30am to give people more time to get to the conference, but unfortunately I was the only member of the Caplin team that managed to make it down to Farringdon. Oh well, more beer and pizza for me.
The programme detailed twelve interesting and informative talks by prominent members of the Gradle community plus Gradleware employees. The tough part was choosing what talks to attend.
Keynote Speech
First up was the Keynote speech by the creator of Gradle, Hans Dockter.
Hans spoke about the recent history of Gradle, including the massive upturn in the use of Gradle. He also talked about the roadmap for the next year, and the interaction between Gradleware (Hans’ company) and community contributions towards Gradle.
A couple of things caught my eye. Reducing the size of the Gradle distribution by removing the bundled plug-ins (this is a good move). This will also be a lot more intellij plug-in integration in future releases.
Also, increased support for dependency resolution via metadata, dependency schemes. This is a familiar concept in Ivy, whereas Maven only uses snapshot or release schemes. This will be incredibly useful for our Platform builds, where we want to download dependencies that are tagged for particular Operating Systems, or we can potentially do away with having multiple Artifactory release repositories, and instead tag our artifacts with statuses such as “QA-approved”, “released”. One to watch.
Coming in future releases will be direct support for variants. This is key for Caplin if we want to fully use it for our C builds (more on this later). It will allow us to build separate variations of the same binary, but pull in different compilers, configuration and so forth, all in the same build script.
Automation Dogfood
This was a talk on how Gradleware, the company behind Gradle, do automation. How they build Gradle, what kind of CI set up they have, what sort of automated testing they do, etc.
It was an interesting talk, but I didn’t learn anything knew – we’re already applying all these good practises here at Caplin. http://builds.gradle.org/ is their CI server – feel free to log in as a guest.
Creating a World Class C/C++ Build System
I was very excited about this talk. If you’ve ever had to deal with Caplin’s CDev shell script hell, you know what pain is. Replacing these shell scripts with Gradle is something I REALLY want to do.
Adam Murdoch took us through the current plugin for C/C++. Although it’s only just been released, it seems pretty stable and it nicely encompasses all the features that we are doing via shell scripts and make files. It’s trivial to build executables across all the OS that we generally support (Windows, Linux, OSX).
Incremental builds are supported by default, they are intelligent and are very accurate (this would speed up our build times). There’s support for both shared and static libraries, and we can fairly easily build across multiple platforms with ease – e.g. use different compilers for different operating systems and so forth. There’s also support for Cygwin, so we can carry on using that if we wish.
The plugin is still “incubating”, but it seems good enough for us to try it out on one of our CDev products. The support for variants (mentioned above) will make this even easier.
Gradle in Very Large Projects
Szczepan Faber kicked off by describing how Gradle is getting faster with each new version that is released. We should really be updating all the Caplin builds that are still using 1.3 or 1.4. The builds will probably run almost 50% quicker. It was nice to hear about all the dependency metadata caching, and how this works.
The emphasis on local development was “Use the Daemon!”. I agree. Using it locally to diagnose build problems saves me 10 seconds per build generally.
Configuration On Demand. For large multi-project builds, Gradle will attempt to load the configuration for all projects before it begins the execution phase. If you have lots of projects, this means your build could be very slow. org.gradle.configureondemand is the answer. If you run a build from a subproject with this set, Gradle will only load the configuration for the base project and your subproject. Faster builds! Note – you may run into problems if you’re ignoring best practice of keeping your subprojects separate. If the subprojects talk to each other, configuration may not be loaded, which will of course result in build failures.
Another technique for speeding up large-multi project builds is to push the configuration and all of the dependency resolution into the gradle execution phase. A way to test out if you’re currently doing this is to simply run “gradlew help”. If the build does anything other than show the help page, then you are doing too much in your configuration phase. This is something we definitely need to look at. When I’ve just run a simple “gradlew tasks” some of our scripts download half the internet.
Parallel builds. As long as your projects are decoupled, they can run in parallel. This could potentially be good, but the examples given didn’t show too much of a time saving, so I doubt we’ll invest much time looking at this.
Building a Continuous Delivery Pipeline
How to use Gradle and Jenkins to build a CD pipeline. In retrospect, I should’ve skipped this and attended the “Implementing a Standardized Build Platform” session that was taking place at the same time.
It didn’t introduce me to anything too new, although it did emphasise that our set up at Caplin isn’t too bad.
The main things I got from the session were that we should be splitting out our build files a bit more, making them more readable. For instance we should be creating an integrationTests.gradle file to hold all that logic, maybe a deployment.gradle file for our app deployments and so for.
There were a few tips on task ordering (described in more detail later), and the speaker Peter Niederwieser talked about the new Java code coverage plugin jacoco, plus the standard pmd, jdepend and Sonar add ons.
There was some mention of setting up environmental configuration, and parsing it in via the grooy configslurper class – which seemed cool. The cargo plugin for deployments to Tomcat might be useful for our demo/deploy environments.
Leveraging Task Ordering
The order of tasks in Gradle can get very complicated, very quickly, much like old-style Ant targets. It’s a problem I’ve seen with our browser Acceptance tests – the build scripts are a bit mental as we’ve stuck loads of “dependsOn” inside loads of our tasks.
This talk, by community member Marcin Erdmann, introduced me to a couple of new gradle concepts. “mustRunAfter” and “finalizedBy”. Marcin wrote these himself, as he came across difficulties using the dependsOn task – just like we have.
“must run after” says that taskB must always run after taskA. This is subtly different from the dependsOn task, as it means that B will run directly after A, rather than “some time after A”.
“finalized by” says that taskX is finalised by taskY. i.e. regardless of whether X passes or fails, task Y will run. I want to use this in our deployments and Acceptance tests. We can use taskY as a clean-up task, to delete rogue processes for cancelled and failed builds. I’m sure there are other uses that people can think of.
But basically, it should allow us to simplify our build scripts, so that the ordering of tasks is much clearer.
Leveraging Existing Tools In Your Builds
This was a general talk about converting existing build scripts across to gradle, or how to get the benefits of gradle without doing much work. As we’ve known for some time, we can use Ant tasks direct from Gradle. They are pretty useful for doing simple things like copying files. The task discussed using the command line tool in order to build using xcode – how to capture build output in a new output stream, and manipulate exit codes.
There were also demos on using “pushover” to send build notifications to your iphone, and your laptop playing a certain song when your build fails. Although not all of this would be practical for us here at work, it showed me the flexibility we have using Gradle. Anything is possible.
Summary
There were a few talks that I missed but luckily the videos have all been uploaded onto the skillsmatter site: http://skillsmatter.com/event/home/gradle-exchange-2013
It was a good day listening to the talk and chatting to some of the Gradle experts. I definitely picked up some useful tricks that I plan to put into good use by the time the next GradleX comes around (November 3rd 2014).