Setting Up Android Builds in Gitlab CI Using Shared Runners

August 31, 2016

Almost a year ago, I wrote a post on how to setup your Android builds using Gitlab CI. That guide is still valid and shows one way how you can do it. The downside is that it requires your own server. This was necessary at the time due to restrictions in Gitlab CI config files. However, a while back (only a couple months after I wrote the last guide), Gitlab added the ability to specify a Docker image in your config. On top of that, they struck a deal with DigitalOcean that drastically increased the number of shared runners they provided. Put those two together, and you have one of the most flexible and available free CI solutions out there!

Quick Intro: What is Docker?

This topic really deserves a whole write-up on it’s own, but the TL;DR is that Docker provides a way to create a completely isolated version of an operating system running in it’s own “container”. Anything running inside the container thinks it has the whole machine to itself, but in reality there can be many containers running on a single machine. Unlike full virtual machines, Docker containers are super fast to create and destroy, making them great choices for setting up temporary environments for building and testing.

Setting Up Your Android Build Using Docker

So, now that you’ve got a general idea of what Docker is, let’s look at how we can use it to setup our Android builds using shared runners. Personally, I found two ways to do this that each have their own pros and cons. The first way we’ll look at is super quick to implement, but results in slightly slower builds and a messier build config. The second way takes a little bit more time, but will result in slightly faster builds and a much cleaner build config.

I’ll describe how to use both methods to create a simple CI config that just ensures that the app compiles and makes the generated APK’s available as build artifacts. I’ve also created a sample project that has all of this code in it.

Method 1: Quick and Easy

All you need to do to get a basic build is copy the following into your .gitlab-ci.yml.

image: openjdk:8-jdk

variables:
  ANDROID_TARGET_SDK: "24"
  ANDROID_BUILD_TOOLS: "24.0.0"
  ANDROID_SDK_TOOLS: "24.4.1"

before_script:
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1
  - wget --quiet --output-document=android-sdk.tgz https://dl.google.com/android/android-sdk_r${ANDROID_SDK_TOOLS}-linux.tgz
  - tar --extract --gzip --file=android-sdk.tgz
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter android-${ANDROID_TARGET_SDK}
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter platform-tools
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter build-tools-${ANDROID_BUILD_TOOLS}
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-android-m2repository
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-google_play_services
  - echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-m2repository
  - export ANDROID_HOME=$PWD/android-sdk-linux
  - chmod +x ./gradlew

build:
  script:
    - ./gradlew assembleDebug
  artifacts:
    paths:
    - app/build/outputs/

You can see the example project’s version here.

Notice that line at the top that specifies an image? That’s the Docker part! We’ve specified that we want to use the openjdk image at tag 8-jdk, which is basically a bare-bones linux image with Java 8 installed. Then we just run some commands to download and install the necessary bits of the Android SDK before you run your build script. Pretty cool, huh?

Just make sure that the versions defined at the top of the file represent those found in your build.gradle, and you’re good to go! Just push to master and you’ll see the build trigger. Easy!

Method 2: Work That’s (Probably) Worth It

The first method is nice because all of the work is just in that one .gitlab-ci.yml file. However, downloading the Android SDK every time you build isn’t necessarily the definition of speed. What if I told you that you could compile the entirety of the before_script: section into a Docker image and just use that instead, eliminating the need to download Android every time you build? Well, you can!

To build your own Docker image, you simply create a Dockerfile. A Dockerfile is literally a file in the root of your project named Dockerfile that lists a series of steps needed to produce an image. If you build a Dockerfile, it’ll generate a Docker image that contains all of those steps. A Dockerfile that performs the setup we require looks like this:

FROM openjdk:8-jdk
MAINTAINER Greyson Parrelli <greyson.parrelli@gmail.com>

ENV ANDROID_TARGET_SDK="24" \
    ANDROID_BUILD_TOOLS="24.0.0" \
    ANDROID_SDK_TOOLS="24.4.1"

RUN apt-get --quiet update --yes
RUN apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1

RUN wget --quiet --output-document=android-sdk.tgz https://dl.google.com/android/android-sdk_r${ANDROID_SDK_TOOLS}-linux.tgz && \
    tar --extract --gzip --file=android-sdk.tgz

RUN echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter android-${ANDROID_TARGET_SDK} && \
    echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter platform-tools && \
    echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter build-tools-${ANDROID_BUILD_TOOLS}

RUN echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-android-m2repository && \
    echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-google_play_services && \
    echo y | android-sdk-linux/tools/android --silent update sdk --no-ui --all --filter extra-google-m2repository

ENV ANDROID_HOME $PWD/android-sdk-linux

You can see the example project’s version here.

That’s the Dockerfile equivalent of our previous .gitlab-ci.yml. If you’re interested in what the commands mean, the Dockerfile format is very well-documented.

Now that we have a Dockerfile, we can use Docker commands to turn this into an actual image we can re-use. However, the act of building an image and then uploading that >1GB file to Gitlab isn’t necessarily practical on most network connections. Thankfully, there’s a way to get around this - have Gitlab do it!

Building and Uploading the Image

By default, each Gitlab repository contains it’s own little Docker registry, allowing us to store Docker images that our repository is using. Super convenient.

There’s multiple ways you can go about structuring your project to build these images, but here’s my favorite way: similar to how you might setup Gitlab Pages, we’ll create a special branch on our project called ci-image. In this branch, we’ll delete everything related to the project and just have two files: Dockerfile and .gitlab-ci.yml (see the branch on the example project here).

Your Dockerfile will be the same file we made above, and the .gitlab-ci.yml will be the following (be sure to substitute your project details where necessary):

build_image:
  image: docker:git
  services:
  - docker:dind
  script:
    - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com
    - docker build -t registry.yourdomain.com/group-name/project-name .
    - docker push registry.yourdomain.com/group-name/project-name:latest
  only:
    - ci-image

You can see the example project’s version here.

This config just tells Gitlab CI to login with the proper credentials, build the image, and then push it to your project’s container registry.

If you commit and push these files, a build will trigger that will do all of the heavy lifting for you, including the huge upload. The best part is that (at least in my experience), everything happens in around 15 minutes! Certainly much faster than most consumer internet connections are capable of.

Using Your New Image

Ok, now that your image is built and uploaded, you should be able to see it at yourgitlabdomain.com/group-name/project-name/container_registry. Now that it’s up there, we can simplify our .gitlab-ci.yml for our Android project to the following:

image: registry.yourdomain.com/your-group/your-project:latest

before_script:
  - chmod +x ./gradlew

build:
  script:
    - ./gradlew assembleDebug
  artifacts:
    paths:
    - app/build/outputs/

You can see the example project’s version here.

So much nicer! Push up your changes and you’ll see your build kick-off, working just as well as before.

Epilogue: Method #1 vs Method #2

So, why use one method over the other? Here’s some advantages that Method #1 has.

  • Easy to setup. Just copy-paste one file and make adjustments. Method #2 involves creating a new branch, writing separate build configs, etc.
  • Easy to make changes in the future. Upgrade your build tools? Just change the version number in your .gitlab-ci.yml. Method #2 would require that you change your Dockerfile and then push and wait for a new build to finish.

Here are the advantages of Method #2.

  • Speed. It’s hard to give definitive numbers as I was seeing a lot of variance, but in general Method #1 took around 4 minutes in my sample project, while Method #2 took from 3-3.5 minutes.
  • Cleanliness. Your .gitlab-ci.yml file for your Android project is going to be much simpler than in Method #1 because you’ve effectively isolated the complexity elsewhere. It might not seem like a big deal now, but as you start writing tests and automatic APK uploads, your build config is going to start getting more complex. It’s nice to isolate ugly parts when you can.
  • Less stress on the server. While you still have to download your image at the start of every build, this is likely going to be a fast transfer within your instance’s local network. Method #1 involves making round trips to Google’s servers on every build to download the Android SDK.

So, there you have it! No matter which way you choose, you now have a simple, free way to setup your Android builds.

comments powered by Disqus