The State of Java Build Systems

Most people eventually get it that building release artifacts using their IDEs is not the way to nirvana. Builds have to stay stable and reproducible between IDE revisions, and being able to execute them in a command line environment or especially continuous integration servers is key to agile development.

Fortunately, there are open source build tools for Java. Less fortunately, they’re not quite perfect. Let’s see what we’ve got.

The Sorry State of Affairs

There’s Ant, the undying classic, but pretty much in maintenance mode, I understand. Ant-based build systems usually start small and innocent but quickly turn into a tangled mess. Every Ant script is a unique piece of modern art that has to be deciphered on its own. They are the cockroaches of Java development, and I guess we’ll have to deal with them again in 30 years when we’ll be porting ancient Java applications to the next big platform.

Maven 2 promises relief and introduces convention over configuration, dependency management, lifecycle support and a few other things that aren’t strictly relevant for a build system. Great concepts, but the actual implementation makes you want to cry. The state of documentation is poor, the XML syntax is unnecessarily bulky, and trivial one-off tasks aren’t possible without a proper plugin (go and write one if you’re feeling brave). Not to mention that stunts like multi-module projects are necessary to build a simple EAR file. And no, Eclipse doesn’t like them either.

Despite all its shortcomings, a lot of people including me tried hard to love Maven, but it’s not easy.

The Winds of Change

Where does that leave us?

I don’t have much hope for Ant. It is what it is and lacks too many things a build system needs, most importantly built-in conventions and higher level abstractions. Granted, there’s Ivy, a pretty nice dependency manager that adds a couple of interesting features, but that’s about it (Ivy looks heavily inspired by Maven, but it’s funny how they changed all the terminology).

Maven 2’s future looks a little bit brighter since it seems the project is opening up to common sense, which maybe be a direct effect of recent discussions and probably Don Brown’s patches. For example, the XML syntax in 2.1.0 will be much more concise and there are fixed default plugin versions in 2.0.9 so that it’s easier to have reproducible builds. Unfortunately, the public repository is a swamp of misplaced software and crappy metadata, so there may be no alternative to setting up a local repository.

At the moment, I’m taking a closer look at Buildr, a ruby-based build system for Java that’s currently under incubation at the ASF. Apparently they’re creating a better Maven, taking the proven concepts but none of its implementation. The jury’s still out on whether the average coder will be able to create effective, maintainable build scripts, but we’ll see. Having to learn Ruby may be a problem to some and an incentive to others.

So?

Poor build systems and development environments in general are a barrier for new project members and a frequent cause for frustration. I’ve worked on projects where all you had to do was to check something out from Subversion, type "mvn package" and you were ready to go. For other projects (both commercial and open source!) you needed a specialist’s support, a couple of days time and a lot of hand-holding to actually build the product and run it locally.

Unfortunately, only few development shops realize that a decent development environment is an asset and invest in their build infrastructure. A good project setup takes time, and if there’s no time or budget available, your build tool of choice won’t do any wonders on its own.

Advertisements
This entry was posted in java and tagged , , , , . Bookmark the permalink.

26 Responses to The State of Java Build Systems

  1. David Linsin says:

    >A good project setup takes time, and if there’s no time or budget available, your build tool of >choice won’t do any wonders on its own.

    That’s absolutely true and I invested a lot of time in my last project to make a decent and relatively easy to maintain ANT script.

    I don’t like maven, but concur with your statement that the idea is great, just the implementation sucks a lot!

  2. mafr says:

    Hmm, in this case you could still use the maven standard directory layout, IMHO one of maven’s strongest points. That combined with Ivy’s dependency management and you’re halfway there. Like that you could still switch to maven or buildr if the need arises and in the meanwhile, the learning curve for developers knowing maven is lowered quite a bit.

  3. David Linsin says:

    Well, I have to admit that I like to control my dependencies almost in a dictator like fashion. I’m really reserved when it comes to introducing 3rd party libraries. Maven makes your life so easy to just pull in a couple of more jars and if you are not careful you’ll get yourself into Jar-Hell with version conflicts, which only show up at run-time.

  4. mafr says:

    Yeah, you’re right, you have to be careful. However, the additional metadata associated with your referenced artifacts make it easier to find out where your transitive dependencies come from (I know this also works with OSGi). IMHO, a bunch of jars in a directory is more difficult to manage, especially when there are lots of them.

    I don’t like to check dependencies into source control, so maven provides a working solution for me. The problem is that in a professional environment you almost always need your own local repositories and you often have to manually exclude transitive dependencies for various reasons (conflicts, crappy metadata etc.).

  5. Teemu A-P says:

    One of my pet peeves with maven 2 is that third party dependencies (DOM files) are sometimes modified loosely without modifying the version number. This means that your build changes overnight without any modifications from yourself. In a sense the transitive dependencies feature means you’re distributing your build description, which is a bad idea, in my opinion.

    I’m still not sure if transitive dependencies makes sense in a build system. The work it takes to add a project with dependencies to your version control is surely minuscule when compared to the effort needed to learn and to use the project properly?

    It sure is a lot easier to add dependencies you know you need than it is to remove ones you do not need but somehow got added.

  6. Paul Keeble says:

    Gant is another possibility, but really its just ant done in Groovy instead of XML.

    Although Maven /Ant are not very good tools I think the world of releasing software to multiple environments is even more lacking. There is no one tool you can say that should be deploying your Java software, whether it just be a simple EAR file or something more complex such as various custom Java programs + a WAR and the config for everything.

    Maven is worse than Ant in this regard as its much harder to properly lay out the configuration, but I’m not convinced any of the release process should require a build to happen.

    So I’d love to see a cross between Ant and Maven, that does dependency management and uses a standard structure layout, but which is easy to customise unlike Maven, with Ant capable scripts (but please not ant!). But then we need a decent server to run this from for the build AND then some thing to do the install scripts also.

  7. Everyone’s looking for an idea of a new company/product to build. Here’s one where the marketplace is virtually screaming for a well built solution. And it doesn’t have to be open source, people will pay for a good solution. Something to think about…

    Btw, you might be interested in one of my latest posts. I talk about how, above an ant/maven build, there are some great ways to get a developer up to speed very quickly. Why should you need to install Eclipse, Maven, etc. every time someone new comes in. Why not automated that too! You can find the complete post at: http://www.followsteph.com/2008/07/29/how-to-quickly-setup-a-software-development-environment/

  8. Albert says:

    For our own java project Fiji (Fiji Is Just ImageJ) we have created our own build system. The whole idea being: that we can handle dependencies easily, and compile/run everything with a single JVM instantiation. We call it ‘fake’ and it uses a rule-based config file, not unlike a Makefile. See the main Fakefile for Fiji:

    http://pacific.mpi-cbg.de/cgi-bin/gitweb.cgi?p=fiji.git;a=blob;f=Fakefile;hb=HEAD

    Fake itself is GPL and contained in the fiji git repository above.

  9. Gregg Bolinger says:

    If you are looking at Buildr but are concerned with having to use Ruby you might want to take a look at Gradle.

    http://gradle.org/userguide.html

    I have only recently been reading the docs. I haven’t used it yet. But it seems to be a Groovy version of Buildr, somewhat. It uses Ivy for dependency management, which is a good thing.

  10. mafr says:

    Re. getting a developer up to speed quickly: At my previous company developers were given a VMWare image containing a complete development environment. Performance was ok (much better than you’d expect) and we all had the same setup. In other projects, some developers worked on Linux, others on Windows, some with JDK-1.5, others with JDK-1.6 and, as expected, it was a total nightmare.

  11. Berlin Brown says:

    Why is everyone so down on Ant? I believe the first poster is spot on and the post in general.

    Are there things that you want in your build system that isn’t covered by Ant or is it, Ant starts to get messy in larger projects and over time.

  12. Buildr is very nice, especially if you’re doing some non-standard tasks with your build script (e.g. building a LaTeX document). It’s a little low on some stability and usability issues though, not to mention a bit of performance optimization. It also shares one of Maven’s strengths/weaknesses: it completely takes over the build. This is nice because there is almost zero configuration to get a basic project in Scala, Java and or Groovy building and JARing, but if you want to do anything special, things get hairy.

    I like Ant because everyone has it and I can bend it to my will, but it’s both verbose and lacks dependency management. Ivy isn’t bad, but it’s like sugar-coating the dinosaur: it’s still verbose, clunky and annoying.

  13. mafr says:

    @Berlin Brown: I’ve seen Ant scripts that grew to several thousand lines of XML. Sure, that’s tiny compared to the actual code base, but there you have patterns and all kinds of abstractions that just aren’t really there in Ant. Compared to your program code, the XML is pretty verbose and hard to read and the tasks are too low-level for my liking.

  14. Steven says:

    i have to agree.
    however, we have ourselves to blame.
    we plug frameworks and 3rd party libs like crazy, and end up tangling ourselves in a big knot of dependency.

    it all comes down to static linking on moduled systems…

  15. Rob says:

    Rewrite Maven in Scala, using JSON/YAML/(some new Scala subset) for the POMs?

    Re Maven doc’n, the Sonatype book[1] is pretty good, but I agree that the doc’n for individual plug-ins can sometimes be lacking.

    1 http://www.sonatype.com/book

  16. It is not a long term solution, but if you’re frustrated with Ant then I suggest trying out Gant. I wasn’t that impressed with it at first because it doesn’t offer an abstraction layer on top of Ant… you still need to know about the core Ant tasks. But a Gant script is a lot easier to deal with because it is not declarative. If you need a function just make one and call it from anywhere. If you need a task/macro then just make a function. It easied deployment a lot for us.

    Like I said, I wasn’t that impressed at first, but after switching back to Ant I realized how much nicer it is.

  17. Dan Howard says:

    Hamlet is right. Gant leverages all the ant plugins (tons) and has a simpler syntax.

  18. Maven is powerful but still it sucks big time; I was trying to build plugin for Atlassian’s product JIRA for my organization and it was a nightmare using maven.
    But once setup it does alot of things for you.

  19. Geoffrey Wiseman says:

    Haven’t seen much about Maven 2.1 having a simpler syntax; ref?

  20. My general recommendation is to treat build scripts as important as the project they build. Let only the most a limited group of the experienced members of the team to work on them and you will be fine.

    Regards,

    Slava Imeshev

  21. Jing Xue says:

    Nice post. Ant+Ivy is still my build tool of choice. With Ant 1.7 you can come up with really nicely modularized scripts that are quite generic and usable. And Ivy’s dependency management is much more powerful than maven2.

    The biggest problem I have with maven2 is that it’s almost always a pain to deviate from the standard settings. For instance, I consider keeping resource files and source files under two separate base folders very unnatural and counter-productive – why should User.hbm.xml not be kept together with User.java?

    Of course Ant’s biggest problem started from day 1 – it was a bad idea to pick xml for the language. I’ve looked at a few other build tools – gant, gradle, etc. Some of them are very promising because of the powerful DSL they base on. But none of them have matured to the point where I’m comfortable with a complete migration.

    Also I’d agree with Slava. Build scripts should be treated first-class on a project. I’ve seen my share of crappy ant scripts. And all of them were produced by junior developers assigned to the task as an afterthought.

  22. Pingback: Blog bookmarks 08/06/2008 « My Diigo bookmarks

  23. panzi says:

    Keep in mind with Java you don’t have to go through autohell. ;)
    I think Waf supports Java, too. Dunno how good Java support is, though.

  24. ittay says:

    @Daniel: where do things get hairy? i’ve been writing a fairly large build (~90 modules and growing) which replaces an older one, so things are definitely not according to buildr standards and have written some functionality not provided by buildr. all went quite smoothly.

    @Hamlet: you can run ant tasks from buildr using AntWrap (and also any Java code you want)

  25. Anil says:

    I use a project template with a build file that downloads the maven-ant integration from the web. Most of the dependencies can be resolved using maven repos. Some libraries such as amazon webservice, i download them from my web server at the time of the build.
    It works most of the time as dependencies among the project changes little, after that its just cut paste.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s