With Android 5.0, Google has introduced the Android Runtime, or ART. ART “brings improvements in performance, garbage collection, applications debugging and profiling.” However, it also replaces Dalvik’s Just-in-Time compilation with Ahead-of-Time compilation performed by translating the Dalvik bytecode to a native ELF executable at application install time.
My question is: what were Google’s technical reasons for making this choice and why is it more performant? I’ve long been an advocate of native code over managed code, but even so, I thought that JIT’d code could outperform AOT-compiled code in many or perhaps even most situations due to its ability to re-optimize for actual application behavior at runtime.
Are the advantages of AOT specific to Android’s implementation or environment (i.e. mobile devices with ARM CPUs and limited RAM), or was JIT just oversold?
8
Compilers need two things to generate performant code: information and resources.
JIT compilers have way more information at their disposal than AOT compilers. Static analysis is impossible in the general case (nearly everything interesting you would want to know about a program can be reduced to either the Halting Problem or Rice’s Theorem), and hard even in the special case. JIT compilers don’t have this problem: they don’t have to statically analyze the program, they can observe it dynamically at runtime.
Plus, a JIT compiler has techniques at its disposal that AOT compilers don’t, the most important one being de-optimization. Now, you might think, we is de-optimization important for performance? Well, if you can de-optimize, then you can be over-aggressive in making optimizations that are actually invalid (like inlining a method call that may or may not be polymorphic), and if it turns out that you are wrong, you can then de-optimize back to the un-inlined case (for example).
However, there’s the problem of resources: an AOT compiler can take as much time as it wants, and use as much memory as it wants. A JIT compiler has to steal its resources away from the very program that the user wants to use right now.
Normally, that is not a problem. Our today’s machines are so ridiculously overpowered, that there are always enough resources at the JIT’s disposal. Especially since the JIT will use the most resources when a lot of new code is introduced into the system at once, which is usually during program startup, or when the program transitions between phases (for example, from parsing the configuration files to setting up the object graph, or from finishing configuration to starting the actual work), at which times the program itself typically does not yet use that many resources (especially during program startup). The Azul JCA is a good example. It has 864 cores and 768 GiByte RAM in its biggest configuration (and note that they haven’t been sold for quite some time, so that is actually several years old technology). According to Azul’s measurements, the JIT uses maybe 50 cores, when it is working very hard. That’s still more than 800 cores leftover for the program, the system and the GC.
But your typical Android device doesn’t have 1000 cores and a TiByte of RAM. And it is extremely interactive and latency-sensitive, when the user starts, say, WhatsApp, he wants to write a message right now. Not in 500msec, when the JIT has warmed up. NOW.
That’s what makes AOT attractive here. Also note that JIT compiling will not only steal resources away from the running program, it will also need battery power, and it will need that every time the program runs, whereas an AOT compiler will only need to spend that power budget once, when the app is installed.
You could go even more extreme and push the compilation off to the app store or even to the developer, like Apple does, but Apple has the advantage of a much more limited set of possible target platforms to consider, so on-device AOT compilation seems a reasonable trade-off for Android.
6