There are countless war stories about how long a compile can take. Even xkcd made a mention of it.
Now, I haven’t been programming for a long time and have mostly just been exposed to Java and Python (and Python is an interpreted language, not a compiled one). I realize it’s possible that I just haven’t come across projects that take very long to compile, but even for decent sized apps, it has either been instantaneous for me (usually handled in the background by an IDE) or taking no more than 30 seconds or so for an extremely large project. Even in a business environment (where the comic takes place), I’ve never had code take that long to compile.
Have I just not been exposed to projects with long compile times? Is this a relic of the past that is no longer something that happens in the modern day? Why would a compile take such a long time?
7
Compilation can take a while, especially for large projects written in languages like C, C++, or Scala. Compiling parts in the background can reduce the compilation time, but occasionally you have to do a fresh compile. Factors that can lead to long compilation times include:
-
Large code size, obviously. Large projects will have hundreds of thousands of lines of code.
-
C’s
#include
preprocessor directive, which effectively causes the same code to be compiled hundreds of times. The macro system has similar issues, as it works on a text-level. The preprocessor really bloats up the code size that’s actually passed to the compiler. Looking at a file after preprocessing (e.g. viagcc -E
) should open your eyes. -
C++’s templates are Turing complete, which means that in theory you can perform arbitrary computations at compile time. Nobody really wants to do that, but even lots of simple cases do add up to quite some time spent specializing the templates.
-
Scala is a fairly young language, and the compiler is horrendously under-optimized. Currently, the compiler uses a very large number of compilation passes (C is designed to require only two compilation passes). Typechecking is one of these passes, and can take some time due to the complicated type system featured by the language.
Compilation isn’t the only thing that takes time. After the project has been compiled, a test suite should be run. The time spent on this can range from a few seconds to a couple of hours (if the tests are written badly).
5
It’s by no means a relic of the past. One of the projects I work on requires 45 minutes for a clean build from scratch. Aside from our own code we also have to pull and build the source from several large C and C++ libraries from external repositories. Compiling and linking C and C++ code is computationally expensive. As you point out, Python is typically implemented as an interpreted language, and Java usually uses a JIT (Just in Time) compiler, so your projects are skipping the upfront compilation and linking costs altogether. The price you pay is in longer start up times and (for Python at least) slower execution speed.
When build times get that long it becomes more important to take advantage of continuous integration systems like Jenkins or TeamCity. This allows individual developers to (mostly) avoid the pain of building from scratch, while still testing that changes aren’t breaking the build.
1
Large projects can take a long time. It can be an hour or more for a big enough project. There are a couple of libraries that I have to compile from source on my computer that take a very long time – e.g. opencascade. The Linux kernel itself also takes quite a long time if you have to build it from scratch.
However, there are other compilation-like processes that can take far longer. Digital circuit design (either for ASICs or FPGAs) requires a place and route step. The place and route step is where the placement of individual logic gates, flip-flops, registers, RAMs, and other components is determined along with the routing for the interconnect wiring. The software uses timing models to determine the gate and routing delays for possible placements, compares these to the limits provide by the timing constraints, and then adjusts the placement locations and wire paths to try to get the timing requirements met. Sometimes the software will even have to resize gates and add buffers in order to meet timing. This step is extremely computationally intensive and can take many hours or even days to complete. It also does not really parallelize very well. There was an FPGA design that I was working on a year or so ago that consumed about half of a Virtex 6 HXT 565 FPGA (~300k out of 565k LUTs) and took about 7 hours to complete place and route. I can’t imagine how long it would take to run place and route on something like a Core i7 CPU design – probably at least several weeks.
Other answers have already mentioned that yes, code on large projects, where large means 500k lines and up, can take significant times, especially when building from scratch.
The additional point is that some projects have to be built for multiple target environments. When the machines that host those environments are not available, the build must be done by cross-compilation, in serial on the machines that you do have. This can lead to significant build times. For one project I worked on, the nightly build would take 10 hours. Woe betide that you were the one who broke it!
I would add that you wouldn’t get away with any such excuse for time wasting. A professional person should be planning their tasks so that they do have something useful to do in such periods.
A little of both. C++ (and C to a lesser extent) were notorious for their slow compilation times, especially on period hardware. Around the turn of the millennium, I worked on a project that took about 4 hours to build due to macro shenanigans.
These days things are better, but 30 seconds is fairly low in my experience – especially in formal builds where things need to be checked out from source control, unit tests run, installers built, and everything sent out to some SAN somewhere.
It depends on the project and the environment in which it is compiled. I’ve worked on C++ projects that took several minutes to compile (set up as multiple projects in MSVS), which is probably enough time for a swordfight.
If you work for a large company with a huge code and data base (Proctor and Gamble, Google, etc.) or for a small company or startup focused on one or two primary products which are very complex (e.g. scientific simulation and rendering), then waiting for a big project to compile is a realistic thing to expect even on powerful machines. This can affect how you develop and debug code (as well as how often you choose to update and merge changes through versioning).