Skyscanner

company news Developing a mobile cross-platform library – Part 2. C++

All articles
Posted on by Kostis Dadamis

Developing a mobile cross-platform library – Part 2. C++

Developing a mobile cross-platform library in C++.

Here, I am including the experience I had while exploring solutions for developing a mobile cross-platform library, i.e. a single codebase that could be part of mobile apps running under different platforms. It covers my journey from mobile cross-platform developments tools (PhoneGap, Titanium, and the likes), code porting tools, and WebViews that weren’t up to the task, to C++ and JavaScript engines that did work. There aren’t many resources out there explaining how to approach this problem, so we thought it could be helpful if we shared this experience. This is the second of three parts, which documents the steps taken to develop the cross-platform library in C++. In part one, you can find the other candidate solutions that were investigated. In part three, you can find the JavaScript solution.

When the cross-platform development tools failed to provide the functionality we needed (check part one), we decided to try a lower-level solution that is supported by both platforms; C++. In Android, we’re able to interface C++ code through the Android Native Development Kit (NDK) and the Java Native Interface (JNI) framework. As for iOS, this is possible with Objective-C++, a language variant that allows source files to include both Objective-C and C++. Hooray!!

Cross-platform library

One of the first problems that we encountered when developing the cross-platform library was STL’s lack of functionality. The Standard Template Library (STL) in C++ provides some minimum functionality, essential for most C++ projects, but unfortunately, it is not as powerful as the JRE/Android and Cocoa Touch frameworks. This problem became apparent when we tried to perform HTTP requests, something that is not part of STL. The solutions to this kind of (missing functionality) problems are the following:

  1. Delegate missing functionality to the native code, i.e. develop everything you can in your cross-platform library and when that extra functionality is needed, call native methods that will perform what you need and pass the result back to the library. Not so clean…
  2. Find C/C++ libraries with their source, and either include the source to your cross-platform library, or build the binaries (and include them) for every target platform and CPU architecture/ABI (armv7, armv7s and arm64 for iOS, and armeabi, armeabi-v7a, x86 and mips for Android).
  3. Find existing prebuilt library binaries for the platforms. Very rare for mobile platforms, but occasionally, you might get lucky (like I did)!

It was decided to use libcurl for the requests due to its portability and robustness. Luckily, the iOS binaries were found through this link, which also contains information on how to build such libraries in iOS. Unfortunately, it wasn’t that easy for Android, as no binaries were provided, and we needed to configure and build it for every target CPU ABI. This can be a painful process as the following steps must be taken:

  1. Build the Android NDK toolchains for the target Android version and each target architecture. This is possible through the ${NDK_HOME}/build/tools/make-standalone-toolchain.sh script provided with the NDK. The NDK toolchains are essential for building libraries, as they provide the cross-compilers, sysroots, and everything that is needed for building. There are some prebuilt toolchains included in ${NDK_HOME}/toolchains, however, they don’t provide support for STL and exceptions. Here is the script I wrote for automating the toolchain building: 

    01: _# buildToolchain.sh_

    02: 03: #!/bin/sh 04: 05: # the NDK directory 06: NDKHOME=$1 07: # the directory to store the toolchains_ 08: BASE_TOOLCHAINDIR=$2 09: # the platform (target version) for which the toolchain 10: # should be generated. android-9 used here 11: PLATFORM=$3 12: # target architecture: armeabi, armeabi-v7a, x86, or mips 13: ARCH=$4 14: # the host system, e.g. linux-x86_64, darwin-x8664 15: SYSTEM=$5 16: 17: export ANDROID_NDK_ROOT=${NDK} 18: 19: TOOLCHAIN_DIR=${BASE_TOOLCHAIN_DIR}/${ARCH} 20: 21: mkdir -p ${TOOLCHAIN_DIR} 22: 23: if [ "${ARCH}" = "mips" ] || [ "${ARCH}" = "x86" ]; then 24: ARCH_SWITCH=${ARCH} 25: fi 26: 27: ${NDK_HOME}/build/tools/make-standalone-toolchain.sh 28: –platform=${PLATFORM} 29: –install-dir=${TOOLCHAIN_DIR} 30: –arch=${ARCH_SWITCH} 31: –system=${SYSTEM}

  2. Build the 3rd-party libraries for every target architecture (armeabi, armeabi-v7a, x86 and mips) using the NDK toolchains. Alternatively, if we don’t care much about coverage, building the libraries for just the armeabi target would be sufficient, as it covers most devices. Here is the script I used for building libcurl: 

    01: _# buildLibCurl.sh_

    02: 03: #!/bin/sh 04: 05: # directory with all toolchains 06: BASE_TOOLCHAINDIR=$1 07: # directory with libcurl’s source_ 08: BASE_SOURCEDIR=$2 09: # target architecture 10: ARCH=$3 11: 12: # find appropriate toolchain_ 13: TOOLCHAIN_DIR=${BASE_TOOLCHAINDIR}/${ARCH} 14: 15: # get the toolchain’s sysroot_ 16: SYSROOT=${TOOLCHAINDIR}/sysroot 17: 18: # arrange the compiler flags according to the architecture 19: # more info can be found in ${NDKHOME}/docs 20: HOST="" 21: CFLAGS="" 22: LDFLAGS="" 23: if [ "${ARCH}" = "armeabi" ]; then 24: HOST=arm-linux-androideabi 25: CFLAGS="-mthumb" 26: else 27: if [ "${ARCH}" = "armeabi-v7a" ]; then 28: HOST=arm-linux-androideabi 29: CFLAGS="-march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16" 30: LDFLAGS="-march=armv7-a -Wl,–fix-cortex-a8" 31: else 32: if [ "${ARCH}" = "mips" ]; then 33: HOST=mipsel-linux-android 34: else
    35: if [ "${ARCH}" = "x86" ]; then 36: HOST=i686-linux-android 37: fi 38: fi 39: fi 40: fi 41: 42: export PATH=${TOOLCHAINDIR}/bin/:$PATH 43: # set the compilers_ 44: export CC="${HOST}-gcc –sysroot=${SYSROOT}" 45: export CXX="${HOST}-g++ –sysroot=${SYSROOT}" 46: export AR="${HOST}-ar" 47: 48: cd ${BASE_SOURCE_DIR} && rm -rf build && mkdir build 49: ./configure –prefix=$(pwd)/build 50: –host=${HOST} 51: CFLAGS="${CFLAGS}" 52: LDFLAGS="${LDFLAGS}" 53: 54: make && make install

After building all the 3rd-party dependencies (libcurl), coding started! I don’t really need to expand on this, everyone can develop their own library logic. My tiny library exposed one function in its API, getItineraries(const char *from, const char *to, time_t date):std::vector<itinerary *>. Calling this function involved querying a Skyscanner Web API with an HTTP request, fetching the response, parsing it, creating the necessary itinerary and itinerary_leg C++ objects, putting them in a vector, sorting them by price, and returning the vector object. There isn’t much logic in there, but it’s enough to prove that developing a mobile cross-platform library is feasible.

In the next sections, we’re going to expand on implementing the Android and iOS glue code. We decided to write the wrappers manually, but alternatively, we could have used SWIG for wrapper generation. SWIG is a powerful tool for interfacing C/C++ code, but it can be a bit complicated to get started with. It would definitely benefit larger-scale projects.

Android

After developing the cross-platform library, we continued with the Android app. This involved the JNI wrappers and the Android UI code. The JNI wrappers consisted of 2 levels; the Java wrappers and the C++ wrappers. The Java wrappers were responsible for providing a Java API to our (C++) library, and the C++ wrappers took care of bridging the Java wrappers with the library by conforming to the JNI naming specifications and object translations.

JNI

In my library, 3 Java wrapper classes were implemented; one for the library’s API (the getItineraries() method) and one for each of the C++ data classes (itinerary and itinerary_leg). Each of the methods that are delegated to the C++ wrapper needs to be declared with the native keyword. These methods do not have a body, and instead, they are implemented in the C++ wrappers. JNI is able to find the corresponding C++ functions in the wrappers through their unique name which consists of the Java package, class and method name, separated by underscores.

1: _// SkyscannerSdkWrapper.java_
2: 
3: **package** net.skyscanner.mobilesdk.wrappers;
4: 
5: **public** **class** SkyscannerSdkWrapper {
6:         
7:         **public** **native** void **test**();
8: 
9: }

01: _//  skyscanner_sdk_jni.cpp_
02: 
03: **#include** <jni.h>
04: _// includes the Android logging functions_
05: **#include** <android/log.h>
06: 
07: _// preventing C++ name mangling in order to work with JNI_
08: **#ifdef** __cplusplus
09: **extern** "C" {
10: **#endif**
11:     
12:     _// corresponds to the net.skyscanner.mobilesdk.wrappers.SkyscannerSdkWrapper.test()_
13:     _// method_
14:     void
15:     **Java_net_skyscanner_mobilesdk_wrappers_SkyscannerSdkWrapper_test**(JNIEnv *env,
16:                 jobject obj) {
17:         _// prints "test" to the LogCat output with "skyscanner_sdk_jni" as the tag_
18:         **__android_log_print**(ANDROID_LOG_DEBUG, "skyscanner_sdk_jni", "%s", "test");
19:     }
20: 
21:     
22: **#ifdef** __cplusplus
23: }
24: **#endif**

The Java wrappers are placed inside the ${ANDROID_PROJECT_HOME}/src/ directory along with the rest of the source code, while the C++ wrappers should be in ${ANDROID_PROJECT_HOME}/jni/.

So, how do the proxy objects work? How can the Java wrapper objects communicate with the original C++ objects? The architecture followed here was to allocate the C++ objects in the heap and store their pointers in the corresponding Java wrapper objects. This way, whenever a C++ object’s field or function needs to be evaluated or called in Java:

  1. the respective Java wrapper object’s method is called with the proper Java arguments,
  2. the Java wrapper method calls the C++ wrapper’s corresponding function, and

  3. the C++ wrapper

    1. converts the Java method arguments to C++ ones,
    2. finds the C++ object through the pointer stored in the Java wrapper object,
    3. calls the C++ object’s method with the C++ arguments, and

    4. if the method is non-void,

      1. it gets the C++ result,
      2. converts it to a Java primitive or new Java object, and
      3. returns it to Java.

    01: // ItineraryWrapper.java 02: 03: package net.skyscanner.mobilesdk.wrappers; 04: 05: public class ItineraryWrapper { 06:
    07: // the pointer to the original C++ itinerary object in the heap 08: private long nativeHandle; 09: 10: public ItineraryWrapper(long nativeHandle) { 11: this.nativeHandle = nativeHandle; 12: } 13:
    14: public native ItineraryLegWrapper getOutboundLeg(); 15:
    16: }

    01: _// itineraryjni.cpp 02: 03: #include "itinerary.h" 04: #include "itineraryleg.h" 05: 06: // … 07: 08: // returns the ItineraryLegWrapper object itineraryleg object 09: jobject Java_net_skyscanner_mobilesdk_wrappers_ItineraryWrapper_getOutboundLeg( 10: JNIEnv env, jobject obj) { 11: 12: // getting the C++ pointer from the Java object 13: 14: // find the ItineraryWrapperClass 15: jclass itinerary_wrapperjclass = env->GetObjectClass(obj); 16: // find the nativeHandle field ID_ 17: jfieldID native_handle_field_id = env->GetFieldID(itinerary_wrapperjclass, 18: "nativeHandle", "J"); // J is the type signature for long_ 19: itinerary it = env->GetLongField(obj, native_handle_fieldid); 20: 21: // get the pointer to the itineraryleg object 22: const itinerary_leg leg = &it->get_outbound_leg(); 23: 24: _// create an ItineraryLegWrapper Java object with the itineraryleg C++ pointer 25: // to be returned to Java 26: 27: // find the ItineraryLegWrapper class 28: const char itinerary_leg_jclass_name = 29: "net/skyscanner/mobilesdk/wrappers/ItineraryLegWrapper"; 30: 31: jclass itinerary_leg_wrapper_jclass = env->FindClass( 32: itinerary_leg_jclassname); 33: // find the constructor_ 34: jmethodID itinerary_leg_wrapper_jconstructor = env->GetMethodID( 35: itinerary_leg_wrapperjclass, "", 36: "(J)V"); 37: // create the Java object_ 38: jobject leg_jobject = env->NewObject(itinerary_leg_wrapper_jclass, 39: itinerary_leg_wrapper_jconstructor, (jlong) leg); 40: 41: return legjobject; 42: } 43:
    44:
    45:     // …_

Here is a more detailed tutorial on JNI.

There is a major issue with this approach, as you probably have guessed. What about garbage collection? Java is a language with automatic garbage collection, while C++ objects located in the heap should be explicitly deleted (well, C++11 has smart pointers, but they were not investigated here). So what happens when Java objects get garbage-collected? Will the original C++ object be garbage collected or will there be a memory leak?

A solution to this problem would be to use Java’s finalize() method which is called when an object gets garbage-collected. Unfortunately, there is no guarantee that this method will be called, so we couldn’t rely on this. The safe, but not so clean, approach that we followed was implementing a dispose() method on each Java wrapper class, which takes care of deallocating the original C++ object. This method has to be called explicitly by the Android app developer before the Java wrapper object gets garbage-collected. If he fails to do so, memory leaks will occur.

01: _// ItineraryWrapper.java_
02: 
03: **package** net.skyscanner.mobilesdk.wrappers;
04: 
05: **public** **class** ItineraryWrapper {
06:         
07:         _// ..._
08:         
09:         **public** **native** void **dispose**();
10:         
11: }

01: _//  itinerary_jni.cpp_
02: 
03: _// ..._
04: 
05:     void
06:     **Java_net_skyscanner_mobilesdk_wrappers_ItineraryWrapper_dispose**(JNIEnv *env,
07:                 jobject obj) {
08:         _// similarly to getOutboundLeg() above_
09:         jclass itinerary_wrapper_jclass = env->**GetObjectClass**(obj);
10:         jfieldID native_handle_field_id env->**GetFieldID**(itinerary_wrapper_jclass,
11:                         "nativeHandle", "J");
12:         itinerary *it = env->**GetLongField**(obj, native_handle_field_id);
13: 
14:         **delete** it;
15:     }
16:     
17: _// ..._

Undoubtedly, this manual garbage-collecting approach will appear strange to Java developers, and it can be dangerous if developers are not careful. There is a workaround to this, but it comes at a cost. Instead of developing Java wrappers for C++ classes, we could translate the exposed C++ classes into Java ones, and when a C++ object would need to be passed to Java, the C++ object’s fields could be copied into a Java object. As a result, C++ objects would no longer be necessary, and they could be freed right after they are copied.

There are three main drawbacks to this workaround:

  1. Objects lose their flexibility. If the objects will need to be manipulated again in C++, two more translations (Java->C++ and C++->Java) will need to be performed.
  2. Allocations and deallocations that take place during copying have an impact on the library’s performance.
  3. Copying the C++ classes into Java ones will result in increased glue code size and possible inconsistencies.

NDK

Let’s continue with more NDK! Apart from the standalone toolchains (for building 3rd-party libraries), the NDK also provides the ${NDK_HOME}/ndk-build script which builds our app’s native code dependencies, using the Application.mk and Android.mk Makefiles. They are located in the JNI directory (${PROJECT_HOME}/jni/) and contain information about target architectures, native/external modules, compiler flags, source files, etc.

1: _# Application.mk_
2: 
3: _# Build for all 4 target ABIs_
4: APP_ABI := all  
5: _# Targeting 2.3 (Gingerbread)_
6: APP_PLATFORM := android-9
7: _# STL to include _
8: APP_STL := stlport_shared

01: _# Android.mk (for the C++ wrappers)_
02: 
03: _# set the local path_
04: LOCAL_PATH := $(call my-dir)
05: 
06: _# clear previously set variables_
07: include $(CLEAR_VARS)
08: 
09: _# name of this module_
10: LOCAL_MODULE := mobile_sdk_jni
11: _# source files to include_
12: LOCAL_SRC_FILES := itinerary_jni.cpp itinerary_leg_jni.cpp 
13:         skyscanner_sdk_jni.cpp utilities_jni.cpp
14: 
15: _# add logging support_
16: LOCAL_LDLIBS := -llog
17: _# this module depends on the mobile_sdk (the cross-platform _
18: _# library) module (imported at the end of this file)_
19: LOCAL_SHARED_LIBRARIES := mobile_sdk
20: 
21: _# include the GNU Makefile script that is in charge of _
22: _# collecting all the information defined in LOCAL_XXX _
23: _# and determine what to build, and how to do it exactly_
24: include $(BUILD_SHARED_LIBRARY)
25: 
26: _# import the module located in the _mobile_sdk_/ directory in the _
27: _# NDK_MODULE_PATH environment variable. _
28: _# the module in this case is mobile_sdk_
29: $(call import-module,mobile_sdk)

Android.mk is required at ${PROJECT_HOME}/jni/, and there can be several Android.mk files, one at each directory where there is C/C++ source code or binaries to be included. However, Application.mk is optional and we can have only one such file (in ${PROJECT_HOME}/jni/) per project. For example, in our project, there were one Application.mk and three Android.mk files; one for Android’s JNI C++ wrappers, one for the library’s source code, and one for libcurl’s binaries.

After preparing the wrappers and Makefiles, executing ${NDK_HOME}/ndk-build at ${PROJECT_HOME}/ builds the C++ dependencies, packages them into shared libraries (one for each CPU ABI), and places them at the ${PROJECT_HOME}/libs/${TARGET_ARCH_ABI}/ directory.

The last step is to make sure that the C/C++ libraries are loaded at runtime through System.loadLibrary(), and finally, implement our Android UI code.

01: _// SkyscannerSdkWrapper.java_
02: 
03: **package** net.skyscanner.mobilesdk.wrappers;
04: 
05: **public** **class** SkyscannerSdkWrapper {
06: 
07:         **static** {
08:                 _// STL_
09:                 System.**loadLibrary**("stlport_shared");
10:                 _// the mobile_sdk (cross-platform library + libcurl) module_
11:                 System.**loadLibrary**("mobile_sdk");
12:                 _// the C++ wrappers (JNI)_
13:                 System.**loadLibrary**("mobile_sdk_jni");
14:         }
15:         
16:         _// ..._
17: 
18: }

Documentation about the NDK can be found under the ${NDK_HOME}/docs directory.

iOS

In iOS, things were much simpler. As previously mentioned, the prebuilt libcurl binary was found, but it shouldn’t have been too hard building it. The Objective-C-C++ bridge is achieved with Objective-C++, a language variant that allows the two languages to co-exist in a source file. In contrast to JNI’s verbose (and strange) specifications, the only step taken here was to add the .mm extension to the files containing the wrapper code, so that the compiler can understand that they are Objective-C++ files. You don’t have to do anything else! Even though I’m an Android fan, I was quite impressed by this.

Consequently, the wrappers were written in Objective-C++. The same architecture was followed as in Android, i.e. 3 Objective-C++ wrappers were implemented, with each wrapper object holding the pointer to the original C++ object which resides in the heap.

01: _//  ItineraryWrapper.mm_
02: 
03: **#import** "ItineraryWrapper.h"
04: **#import** "ItineraryLegWrapper.h"
05: 
06: **#include** <vector>
07: **#include** "itinerary_leg.h"
08: 
09: @interface ItineraryWrapper ()
10: 
11: _// the pointer to the original C++ itinerary object in the heap_
12: @property itinerary *nativeHandle;
13: 
14: @end 
15: 
16: @implementation ItineraryWrapper
17: 
18: _// the constructor_
19: - (id)**initWithItinerary**:(itinerary *)it {
20:     **if** (self = [super **init**]) {
21:         _nativeHandle = it;
22:     }
23:     **return** self;
24: }
25: 
26: @end

Fortunately, garbage collection was much easier. Objective-C objects have a dealloc method which is called when they are deallocated; it is similar to Java’s finalize() method, but in Objective-C, it is guaranteed that it will be called upon deallocation. Therefore, the C++ destructor was called inside dealloc, and no special instructions need to be given to app developers.

01: _//  ItineraryWrapper.mm_
02: 
03: _// ..._
04: 
05: @implementation ItineraryWrapper
06: 
07: _// ..._
08: 
09: _// called automatically when the Objective-C++ object is destroyed_
10: - (void)**dealloc** {
11:     _// free the original C++ object_
12:     **delete** self.nativeHandle;
13: }
14: 
15: @end

After the wrappers were implemented, a static library (iOS doesn’t support shared libraries) was built, containing our library along with Objective-C++ wrappers. The library and the headers were imported in the iOS project, the UI was implemented, and the iOS app was done! Much easier than Android..

That’s it! I hope this will help those of you trying to create mobile cross-platform libraries, reusing C++ libraries in Android/iOS, or even the ones getting started with Android NDK, JNI, or Objective-C++.

Good luck!

android, c++, cross-platform, ios, library, mobile, Technology