An Incremental Java Builder

Posted on 2015-3-12

It's probably rare for a Java developer to run javac directly nowadays. Though it's gotten better, there was a time when it didn't even support wildcards, it's still hard to use for even moderately complex projects. Fortunately there's very little need for it thanks to Maven and Gradle. Feature-rich project builders that call the compiler for you, which makes it irrelevant how unfriendly the interface is. However, what neither of these tools do, is minimal complete rebuilds.

Maven's useIncrementalCompilation option (which is currently bugged in 3.2.5 and the meaning of the value is reversed) uses the last modified date to determine if something needs to be rebuilt. However, if you change a file that other files depend on, those files will not be rebuilt and the resulting class files will not run correctly. A proper incremental compiler would understand the relationships between classes. Eclipse and other IDE's have intimate knowledge of class dependencies, which they need for things like code completion, so they can already do this.

I recently discovered a new tool in Java 8 called jdeps. With it you can list dependencies, which now lets anyone construct a dependency graph for any compiled jvm bytecode. Using this, how hard would it be to make a proper incremental compiler for java? Turns out, not so hard...

On my Github repo you'll find a small example of what can be done with the new jdeps tool. I call both javac as well as jdeps programmatically and I buffer and parse their output using a StringWriter. After an initial clean build all dependency info is stored in a HashMap. When a file changes, all java files that have a direct or transitive dependency on the changed file are also added to the command-line for javac. Using Maven I tested the exact same scenario as in the unit test, which illustrates the core concept and benefit. Maven did not rebuild dependant classes, resulting in code that would fail during runtime. So, does that mean my tiny java builder better than Maven??!?!

As mentioned before and in the project's README: this is not something that I expect will be incredibly useful, nor is it a new concept. Java compilation is quite fast. The slowest parts of a builds are moving files around, compressing files, running tests. The cost of compilation is negligible, so doing full rebuilds is hardly ever a problem. However, the ease with which it can be implemented now hints at an opportunity for something more. Combined with hot code reload like spring-loaded, an incremental compiler can give a faster and complete reload cycle for development. Combined with a next generation build tool that is incremental in everything, including testing, resource processing as well as code. The feedback loop can be shortened for the most complex OO applications in such a way that it resembles a REPL. I hope this is where software engineering is headed, among other places.

Written with StackEdit.