HCSS has used BuddyBuild to help us build our construction software apps for iOS and Android mobile devices.
In my opinion, BuddyBuild had the most seamless experience for Android continuous integration (CI). It could build multiple variants and run instrumented tests on an Android emulator out of the box. Unfortunately, BuddyBuild discontinued Android support on March 1, 2018, so HCSS developers were left to explore other alternatives.
Here’s the quandary: most CI companies have built super-generic workflows that can be customized. Customization is great, but the reality is that none of the super-generic workflows “just work.” Even for a standard build-test-deploy workflow, we have to write custom scripts.
We checked out Appcenter for Android to see if it would fit our needs.
After signing up for an account, Appcenter was able to find our GitHub repositories, and we were able to get a basic build up and running fairly easily. Creating and running unit tests was also straightforward, because there’s a toggle in the build configuration for each branch.
Getting instrumented UI tests working, however, required a lot of trial and error. Appcenter provides three different hooks into the build process: post-clone, pre-build, and post-build. These scripts need to be placed in a certain part of your Android project.
Problem No. 1: The documentation states that you need to place your appcenter-post-build.sh at the same level as your project-level build.gradle file, (i.e., $APPCENTER_SOURCE_DIR/build.gradle). This is NOT correct. In order for Appcenter to correctly pick up your build scripts, they need to be at the module level, (i.e., $APPCENTER_SOURCE_DIR/app/build.gradle).
Problem No. 2: Even though Appcenter is able to run a basic launch test on a device, it does not automatically build your instrumented test APK file, nor does it have the ability to automatically kick off the test service to run your instrumented tests. In other words, the post-build script has to generate the test APK file and send it up to Microsoft’s test service.
For us at HCSS, one of our build variants is “devDebug.” Here’s what the script looks like:
#!/usr/bin/env bash
# generate test apk
$APPCENTER_SOURCE_DIRECTORY/gradlew assembleDevDebugAndroidTest
# launch the test service
appcenter test run espresso \
--app "{username}/{appname}" \
--devices "{username}/{deviceSet}" \
--app-path "$APPCENTER_OUTPUT_DIRECTORY/app-dev-debug.apk" \
--test-series "master" \
--locale "en_US" \
--build-dir "$APPCENTER_SOURCE_DIRECTORY/app/build/outputs/apk/androidTest/dev/debug" \
--token "{apiToken}"
Catch #3: When entering spaces in a user name or app name, appcenter automatically converts them to hyphens. Our app name is “HCSS Field”, but the expected app name in the post-build script is “HCSS-Field”. This is not obvious.
Problem No. 4: The documentation for starting up the test service includes the command Appcenter login, which opens your browser and generates an API token. A post-build script obviously cannot be interactive. In order to authenticate the post-build script, you need to include a token, which is generated by going to account settings at the bottom left of the screen.
Problem No. 5: The locale setting in Appcenter-cli does not work on all devices. (The Pixel 2 device we were assigned is in French). Make sure your tests are locale-independent. We had to fix a few tests that assumed the device was en_US.
Problem No. 6: The appcenter-cli –app-path is looking for the full path to the APK, while the –build-dir is looking for the path to the directory containing the test APK. In hindsight, this is obvious given the name of the flag, but it’s strange because the test service is ultimately looking for an APK in both places.
Problem No. 7: The documentation seems to indicate that using the ReportHelper is necessary only if you want to add custom labels to your tests. This is NOT the case. If you don’t add it, your tests will pass, but Appcenter will mark the test suite as failed.
After a bit of back-and-forth, we got our build-test-deploy workflow automated, but it wasn’t easy. Hope this helps anyone migrating to appcenter.ms.