Automated Unit testing an iOS app with Jenkins

  • Setting up and executing automated unit tests is slightly more involved than generating built apps for distribution.
  • The following is the set up that I use for some of my projects, I have just renamed the project to TestProject for convenience.
  • For this article I am assuming that you have Jenkins, Xcode, Xcode Command Line Tools, and the Xcode Jenkins plugin already installed.
  • Because I use cocoaPods for dependency management, I build a workspace using custom build schemes. To build a single project with a unit-test target would make these steps easier.

Dependencies

  • HomeBrew
  • Ruby 1.9.3
    • Needed for other dependencies.
    • Can be installed via brew, or with RVM.
    • I used RVM to set up my ruby install.
  • Sinatra
    • Sinatra is a ruby based server that we use for serving JSON fixtures to the unit tests.
    • Can be installed using the ruby package manager sudo gem install sinatra
  • ios-sim
    • Required because Xcode doesn’t allow unit tests to be run natively in the iOS simulator from the command line.
    • Can be installed using brew: brew install ios-sim

Step 1 – Poll SCM

  • The TestProject Jenkins job polls the SCM looking for changes to the master branch at midnight every night.
  • If no changes have occurred, then the project is not built.
  • If modifications have been made, the next step is executed.
  • If you want, Jenkins can be set up to build on a push to a branch. E.g. Pushing to the remote master branch.

Step 2 – Simulator and Sinatra setup

We run the following script:

#!/bin/bash

#reset the content and settings of the iphone sim
rm -r ~/Library/Application\ Support/iPhone\ Simulator/

#open the iphonesimulator and kill it
#this is required after a system restart
#so the simulator knows to run iPad rather than iPhone apps
echo "Opening iphone simulator"
open "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/Applications/iPhone Simulator.app"
sleep 10
killall 'iPhone Simulator'
echo "iphone simulator killed"

#delete previous build folders
echo "Removing previous build folder"
rm -r ${WORKSPACE}/build
mkdir ${WORKSPACE}/build

#Start sinatra server in the background
ruby TestProject/server.rb &

#get the PID of the process
PID=$!

#save PID to file
echo $PID > ${WORKSPACE}/sinatra.pid
  • We first remove the iPhone Simulator folder
    • This makes sure that no previous TestProject apps are installed on the simulator. Otherwise we may get core data upgrade problems.
  • We then have to open the iphone simulator and then kill it
    • This is a stupid workaround that has to be done so that the iphonesimulator recognizes that we have to run an iPad application rather than an iPhone app.
  • We then remove any previous build folders.
    • Because we have our project set up as a workspace, there are multiple .xcodeproj files and libraries that we have to build, including our pods dependencies. Because of this, our default build location is relative to the project, not located in the iPhone Simulator folder or in Xcode’s DerivedData folder.
  • We then start the sinatra server in the background
    • The & operator detaches the ruby process from the current shell so that once this script has finished, the sinatra server is still running.
    • We store the PID of the process to the PID variable.
    • The $! expands to the process ID of the most recently executed background (asynchronous) command. More details here
    • The PID is then written to file so it persists.

Step 3 – Xcode Build

Below is a screenshot from jenkins showing the fields used for the xcode plugin

Screen Shot 2013-04-23 at 3.33.32 PM

  • Clean before build – we don’t want any cached compiled objects hanging around.
  • Xcode Schema FileTestProjectTests
    • Because of a limitation where workspaces can’t build targets directly, we have to use a Build Scheme to run unit tests. This scheme is set up the run the attached unit test target included in the production scheme TestProject
  • SDKiphonesimulator
    • We are targeting the simulator to run unit tests so we specify it here.
  • ConfigurationDebug
    • Unit tests only execute in Debug mode, so this option has to be this.
  • Custom xcodebuild arguments
    • TEST_AFTER_BUILD – We manually specify that we want to run unit tests after building the project.
    • ARCHS=i386 – We have to force the architecture to i386 because xcode wants to default to armv6, armv7 or armv7s.
    • ONLY_ACTIVE_ARCH=NO – Tell Xcode to not build just the architectures that it wants to by default.
    • VALID_ARCHS=i386 – We have to specify the architecture here again. Xcode does not make this easy for us.
    • SL_RUN_UNIT_TESTS=YES – This is where the magic happens, this will be explained in more detail in the next section.
  • Clean test reports?
    • This outputs clean test reports so we can export them to JUnit reports later.
  • Unlock keychain?
    • Required so we don’t have to enter the password to use debugging.

Step 4 – Unit testing

As explained in the previous step, the SL_RUN_UNIT_TESTS=YES xcodebuild argument is extremely important.

The TestProjectTests target in Xcode has a custom script that it executes after building. The script can be found in Project Settings -> TestProjectTest -> Build Phases -> Run Script

Screen Shot 2013-04-23 at 3.38.26 PM

The script is shown below:

ruby -v
ruby "${SRCROOT}/commandlineunittests.rb"
  • The first line is unnecessary, and just used for outputting the ruby version.
  • The second line calls a ruby script that is present in the repository that kicks off the unit tests.

The second ruby script is shown below:

if ENV['SL_RUN_UNIT_TESTS'] then
    launcher_path = "/usr/local/bin/ios-sim"    
    #File.join(ENV['SRCROOT'], "Scripts", "ios-sim")
    test_bundle_path= File.join(ENV['BUILT_PRODUCTS_DIR'], "#{ENV['PRODUCT_NAME']}.#{ENV['WRAPPER_EXTENSION']}")

    environment = {
        'DYLD_INSERT_LIBRARIES' => "/../../Library/PrivateFrameworks/IDEBundleInjection.framework/IDEBundleInjection",
        'XCInjectBundle' => test_bundle_path,
        'XCInjectBundleInto' => ENV["TEST_HOST"]
    }

    environment_args = environment.collect { |key, value| "--setenv #{key}=\"#{value}\""}.join(" ")

    app_test_host = File.dirname(ENV["TEST_HOST"])
    system("#{launcher_path} launch \"#{app_test_host}\" #{environment_args} --args -SenTest All #{test_bundle_path}")
else
    puts "SL_RUN_UNIT_TESTS not set - Did not run unit tests!"
end
  • The script checks for that magic variable SL_RUN_UNIT_TESTS and if it’s present runs the unit tests.
  • Using the ios-sim dependency, the script dynamically patches the TEST_HOST of the ios simulator and runs the unit tests. This is really complicated to try and do by hand, which is what we were doing before using ios-sim

Step 5 – Cleanup

The following script is executing after the unit tests have finished, regardless of the output status (PASS or FAIL).

#!/bin/bash

PID=$(<${WORKSPACE}/sinatra.pid)
echo "Sinatra server pid $PID"

kill -9 $PID
  • This script reads the process id (PID) from the file we stored earlier containing the sinatra server’s PID.
  • We then kill the sinatra process.
    • We don’t want the sinatra server hanging around after the unit tests have run, because subsequent tests will fail because they will try to start a sinatra server using the same port as the previous process.
Advertisements

Auto build and deploy iOS apps using Jenkins

We will have a look at a TestProject to get an idea of how jenkins can be used to build, sign and deploy iOS projects. The whole process consists of 4 steps. Note that to deploy an app in this manner, an enterprise distribution certificate is required for each app.
This guide is how I have set up my enterprise builds for my apps that use cocoaPods for dependencies, but most of the same principles apply for any iOS apps.
Because the project uses cocoaPods, it means we have to build a workspace and have a build scheme set up.

Note: Apple Enterprise licenses can ONLY be used for builds internal to your company. I am not responsible for anything that may happen if you try to distribute apps outside your company using an enterprise license.

This guide may be a bit brief, don’t hesitate to ask me any questions. This is mostly for my own reference.

Step 1 – Poll SCM

  • Dev builds poll the SCM looking for changes to the master branch at midnight every night.
  • If no changes have occurred, then the project is not built.
  • If modifications have been made, the next step is executed.

Step 2 – Xcode build

Below is a screenshot from jenkins showing the fields used for the xcode plugin.

Screen Shot 2013-04-23 at 4.01.12 PM

  • Clean before build – we don’t want any cached compiled objects hanging around.
  • Xcode Schema FileTestProjectDev
    • Because of the way we have our project set up, our top level item is a workspace (TestProject.xcworkspace), we have to build based on Schemes as Xcode does not support building a target from a workspace.
    • TestProjectDev specifies the scheme we want to build with.
  • ConfigurationRelease
    • We want the release version. There are several subtle differences between Debug and Release versions. The most notable is that Debug builds are usually built for just one architecture, where Release builds are built for all supported architectures, usually with compile time optimisations as well.
  • Workspace File – This is our top level workspace file (TestProject.xcworkspace) that we build from. If this option wasn’t specified, by default the xcode build would look for the first *.xcodeproj file it can find. This would cause our build to fail.
  • Unlock Keychain
    • This allows us to unlock the keychain which is required for signing or resigning .ipa files.
    • The location of the keychain is the default OSX location

Step 3 – Resigning

  • This step only occurs if previous steps were successful
  • This step only occurs if the text BUILD is present in the log text (basically every build)
  • The result of this build will be escalated to the job status. If this step fails and other steps before it were successful, the job will still fail.

Below is the script used for resigning:

#!/bin/sh

PROJECT_BUILDDIR="${WORKSPACE}/build/TestProject/Build/Products/Release-iphoneos"
APPLICATION_NAME="TestProjectDev"
BUILD_HISTORY_DIR="/Users/administrator/Provisioning/builds"
DEVELOPER_NAME="iPhone Distribution: Test Company LTD"
PROVISIONING_PROFILE="/Users/administrator/Provisioning/TestProjectDevDistributionProfile.mobileprovision"
HOST_LOCATION="/Library/WebServer/Documents/apps"

#Sign The .app file and create .ipa file
/usr/bin/xcrun -sdk iphoneos PackageApplication -v  "${PROJECT_BUILDDIR}/${APPLICATION_NAME}.app" -o  "${BUILD_HISTORY_DIR}/${APPLICATION_NAME}.ipa" --sign ${DEVELOPER_NAME} --embed ${PROVISIONING_PROFILE}

#Get the version from the Info.plist file
APP_PATH="${PROJECT_BUILDDIR}/${APPLICATION_NAME}.app"
VERSION=`defaults read ${APP_PATH}/Info CFBundleShortVersionString`
BUNDLE_ID=`defaults read ${APP_PATH}/Info CFBundleIdentifier`

# Create plist
cat ${HOST_LOCATION}/template.plist | sed -e "s/\${APP_NAME}/$APPLICATION_NAME/" -e "s/\${BUNDLE_ID}/$BUNDLE_ID/" -e "s/\${BUNDLE_VERSION}/$VERSION/" > ${HOST_LOCATION}/${APPLICATION_NAME}.plist
  • The first part of this script sets up the locations and script variables
    • PROJECT_BUILDDIR – The folder that the built .ipa file resides from the previous build step.
    • APPLICATION_NAME – In this case it is TestProjectDev
    • BUILD_HISTORY_DIR – The folder that the re-signed app will be output to
    • DEVELOPER_NAME – Has to match the .mobileprovision file used for signing.
    • PROVISIONING_PROFILE – The location of the .mobileprovision file used for signing. These must be downloaded from the Apple developer website or shown in Finder from the Xcode organizer.
    • HOST_LOCATION – Where the final signed app will exist
  • The next stage of the build process is actually re-signing the app using the Xcode command line tool called xcrun. Note that the version in /usr/bin/ is a symlink to the Xcode version that has been selected by the xcodeselect command line tool. This is important if more than one version of xcode is installed. Each version of Xcode stores command line tools inside the .app file in applications.
  • After re-signing, we grab the version, and bundle-id. These are used in the next part
    • APP_PATH – The full path including the .app extension of the re-signed app
    • VERSION – This is the CFBundleShortVersionString. This value is set when you change the version in Xcode for each of the build schemes.
    • BUNDLE_ID – The CFBundleIdentifier for the built app. For TestProject, this will be au.com.wordpress.danielbeard.TestProject. These can be set by modifying the build Schemes, but shouldn’t be changed for existing applications. Production has a different bundleId, which is what allows for both the dev and prod versions to exist on a single iPad at the same time.
  • We then auto generate the enterprise install PLIST file. This is needed to install apps from a URL location and acts as a “description” of the app that is about to be installed.
    • The actual creation of this file is quite simple, we use cat to copy the template.plist file to the new location and then using sed we replace the BUNDLE_ID and BUNDLE_VERSION values. These values are read from the Apps Info.plist from inside the .app folder.
  • Below is an example template.plist file
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>items</key>
        <array>
                <dict>
                        <key>assets</key>
                        <array>
                                <dict>
                                        <key>kind</key>
                                        <string>software-package</string>
                                        <key>url</key>
                                        <string>http://hostlocation/apps/${APP_NAME}.ipa</string>
                                </dict>
                        </array>
                        <key>metadata</key>
                        <dict>
                                <key>bundle-identifier</key>
                                <string>${BUNDLE_ID}</string>
                                <key>bundle-version</key>
                                <string>${BUNDLE_VERSION}</string>
                                <key>kind</key>
                                <string>software</string>
                                <key>title</key>
                                <string>${APP_NAME}</string>
                        </dict>
                </dict>
        </array>
</dict>
</plist>

Step 4 – Copying to host location

  • This step simple grabs the signed .ipa file and puts it in the host location so that the macmini website can serve it over a URL. The host location is /Library/Webserver/Documents/apps

Script looks like this:

#!/bin/sh

APPLICATION_NAME="TestProjectDev.ipa"
BUILD_HISTORY_DIR="/Users/administrator/Provisioning/builds"
HOST_LOCATION="/Library/WebServer/Documents/apps/"

cp "${BUILD_HISTORY_DIR}/${APPLICATION_NAME}" "${HOST_LOCATION}${APPLICATION_NAME}"

And that’s it! The output is a .ipa file and a .plist file. These are both required to install an enterprise app.