Discussion On
Java Build Systems: A Sad State of Affairs

by in Books & Tools

47 Comments

  1. Brad
    Monday, October 444, 2010

    Just as a comparison, the “if” thing in workflow foundation’s xaml (which backs the TFS build system in 2010) would be something like:

    Where the Condition is a slightly VB-like syntax that can do property lookups on objects and all the basic comparison stuff (and, or, xor, bitwise and, etc)

    Reply
  2. u_r_so_right
    Monday, October 426, 2010

    u r so right dude..

    saw two “star” config/release manager come and go..
    one implemented his wet dreams with tens of build xmls with braindead string replacement and such
    the other lame arse was buzz-fan with all that “convention over configuration” shit.. and started writing his own plugins, which of course failed and he ended up reusing the ant tasks..

    now ruby/groovy/haskell/the next best hyping scripting language introducing new bs paradigms again..

    we needed a smart text editor and we got eclipse with plugins/repos/an empire
    we needed a building system and we got maven with plugins/repos/an empire

    format C: /S/N/M

    Reply
  3. Robert Massaioli
    Monday, October 438, 2010

    I agree that a new build system really is required. As for how to make your configuration file I love the way that the XMonad guys did it. Use the language itself to write the config file and keep that simple and dry.

    Reply
  4. Mikel
    Monday, October 417, 2010

    I agree XML is too verbose, but in what way does in violate DRY?

    Reply
  5. Matt Doar
    Monday, October 404, 2010

    It’s all true. Builds do take time to get right, and just as with any language you can shoot yourself in your foot without too much effort. I also see Java build tools rebuild more than they need to because “it’s only a few seconds” and because the dependencies embedded in class files are a pain to process. SCons does do the analysis better, but it’s not too speedy. I also don’t see why a build system should be language-specific any more than a documentation tool should be, but most of them are.

    Reply
  6. Dilantha
    Monday, October 411, 2010

    Buildr looks promising http://buildr.apache.org/

    Reply
  7. artsrc
    Monday, October 444, 2010

    Maven does seems to do a good job of describing a simple project to an IDE, which is an important use case to me.

    I think there are some very valid criticisms or Maven which have fairly obvious fixes:

    1. Things can change without bumping their version number
    2. Maven itself can change

    1. is a repository management problem. Well run repositories should refuse to update a (non-snapshot) published library without a version bump.

    2. Maven POM’s should indicate which version of maven to run with and run with a fixed set of plugin’s defined for that version.

    The ant “if” syntax was a deliberate attempt to not support a particular build file style. XSL is a more representative example of if in XML:

    http://www.w3schools.com/xsl/xsl_if.asp

    Reply
  8. Maciek
    Monday, October 454, 2010

    I agree XML is too verbose, but in what way does in violate DRY?
    Since the author states this as opposed to JSON, YAML, etc., I believe he means the repetition of opening and closing tags. This is, at best, stretching the no-DRY principle really thin.

    (Not that I disagree with anything else here: for a sophisticated enough build process, ant turns into a scripting language implemented in XML with bizarre sort-of-immutable variables, and I can’t think of anything more evil or insane)

    Reply
  9. bozoCranky
    Monday, October 446, 2010

    Errrr… a lot of what is mentioned I agree with…. ant, maven the constant debate amongst my architects…. we are now looking at Gradle which seems to bridge the gap between both build systems. Although we have not agreed that this is the best practice going forward… it looks like we will inevitably adopt Gradle as the preferred build strategy.

    As a note… I met the developers of Gradle at the Java One convention…. undoubtedly this was a highlight of the whole convention, these gents had such vigor and spark that it made me WANT to use Gradle… as a result I have had members of my team review it…. and… all the feedback has been positive… notably… much of the criticism highlighted in the article is addressed by the Gradle solution… In any event… I don’t want to over sell and would love to hear others options in order to find the best solution.

    1,
    R

    Reply
  10. Ben
    Monday, October 441, 2010

    Mickel:

    <tag></tag>

    how many times did I -really- need to say ‘tag’?

    Reply
  11. chris
    Monday, October 443, 2010

    I dislike build files which suffer from the inner platform anti pattern. To me this was Ant largest failing. Over the years each place I worked with ant had their own convention for how ant projects built, but each was complex and unique. For all of mavens failings the convention aspect is well enforced, and I’ve never seen a maven project that was horribly complex. My concern with using a touring complete language in a build env is that it opens itself up to making more company specific plaforms..ie back to ant land.

    Reply
  12. Meikel
    Monday, October 456, 2010

    Have a look at Gradle.

    Reply
  13. Mike Samuel
    Tuesday, October 558, 2010

    http://code.google.com/p/prebake/ is a build system designed to work well with Java, and it’s implemented in JDK 7.

    Reply
  14. Mark
    Tuesday, October 537, 2010

    Mikel:

    The post doesn’t say XML violates DRY, it says Ants implementation does, and then gives a specific example.

    Having said that, it is pretty clear that XML does force the repetition of the tag, and that does introduce a class of error that SEXPs, for example, don’t have:

    Uh oh...

    vs


    (tag1
    (tag2
    All clear here!
    )
    )

    I’m genuinely curious, where does the fear of turing complete build languages come from? I’ve occasionally been bitten by its lack, but never seen a hideous mess of a build file and thought “Gosh, if only this build language didn’t have arbitrary storage / unbounded recursion etc, this file would be so much more understandable.”

    Reply
  15. Jess Johnson
    Tuesday, October 518, 2010

    The ant “if” syntax was a deliberate attempt to not support a particular build file style.

    It seems to me that the better options are either don’t support “if” at all or support it in a sane way. As others have mentioned, there are ways to do conditionals in XML that are somewhat less ugly than the Ant example. What kind of logic are the Ant developers using if they are spending time purposely creating features that are ugly because they don’t want people to use them?

    Reply
  16. Mikel
    Tuesday, October 546, 2010

    DRY is about having a single authoritative place where something is defined or configured, not about syntax and grammar.

    You should try to make your point without appealing to DRY.

    Reply
  17. Dave
    Tuesday, October 506, 2010

    I don’t accept that Maven is as non-deterministic as you claim. Please head over and have a look at wakaleo’s “maven mythbusters” blog entries for a more level-headed look into issues such as those mentioned above…

    e.g.: http://www.wakaleo.com/component/content/article/246 (there’s another one somewhere there too – the rest of his blog entries are also pretty interesting)..

    It’s good to be looking for better alternatives – necessity is the mother of invention and all that – but you should try and do it with a level head and a critical understanding of what already exists, and where the real shortfalls lie. Unfortunately, discussions about build tools, like many things in the development world, seem to polarize people and this can make it hard to unpick the good points from the vitriol.

    I’ve personally found a combination of Maven + Ant offer everything that we want. We use Maven for libraries + some fairly large multi-module web-app builds. We use Nexus as a local repository for internal and 3rd party artifacts, and as a cache of the public maven repo. We use Ant with a set of standardized scripts (along with the maven ant tools) for building the one or two standalone deployable apps/artifacts that we have.

    This combination has so far worked very well for us – our ant build files basically boil down to a few includes + a few lines of config, the maven POM’s are similarly simple. New starters have the benefit of a completely standardised project structure which shortens the learning curve considerably from what would be required with a bespoke per-application build setup. Where I work, that counts for a lot. YMMV.

    Also, as far as Maven goes, tooling helps a lot. With M2Eclipse for example, you almost don’t need to bother editing the .pom files manually or worrying about the XML tag soup any more :)

    Reply
  18. cjh
    Tuesday, October 520, 2010

    It was my understanding that the reason make was not suitable for building Java code (at least when using javac) was due to the javac behavior of pulling dependencies into the compilation stage if they were needed to correctly build a class file, thus making some dependencies implicit, which make is rather bad at dealing with. See http://www.fudco.com/software/jdep.html for one solution to this problem.

    Reply
  19. Parsa
    Tuesday, October 533, 2010

    Gradle is the way to go: Groovy syntax, strong dependency management, scripting within build script, fully compatible with Ant, great plugins (jetty, eclipse, java, scala, etc.), good support for multi-project builds… I’ve been using it for a while now and I absolutely love it.

    Reply
  20. Bob McBob
    Tuesday, October 551, 2010

    Gradle is the natural successor. An absolute pleasure.

    Reply
  21. Andrei Ciobanu
    Tuesday, October 505, 2010

    +1 For Gradle.

    Reply
  22. Steph
    Tuesday, October 545, 2010

    I think the biggest issue is that it’s a hard problem to solve and there’s no money. Even as you expressed the issues, which I completely agree with, you didn’t offer a good alternative. And I don’t blame you, it’s hard. Many people have ideas on how to solve some specific aspects, but to overhaul it is a big task, and it too will probably encounter other issues.

    And again, above it being a hard problem to solve, there’s hardly any money into it. How many people and companies would pay for a good build solution? Most will still continue to force their developers to use ant/maven, even in the face of better solutions. It’s really hard to compete with free solutions that are pretty ingrained. It’s much like version control solutions, it’s a hard market to compete in, probably much harder since it’s not something you modify or use daily (by this I mean edit, modify, etc., not call a build file such as in daily builds).

    Reply
  23. bill
    Tuesday, October 531, 2010

    minor spelling correction
    >you will need to peak under the covers.
    peek.

    Reply
  24. Matt Doar
    Tuesday, October 551, 2010

    Steph – yup, that’s the classic “there’s no money in tools” problem. I’ve made my living running a small consultancy for 4 years helping people with software development tools. I’d say less than 20% of that time has been spent with version control systems and builds. Most of my work has been customizing Atlassian tools such as JIRA, which are used by a larger number of groups within most organizations so get more dollars spent on them. Builds tools are very specialized and don’t get the money they really should, in my opinion.

    Reply
  25. Benjamin Darfler
    Tuesday, October 523, 2010

    If you are going to write about build tools in October 2010 you really need to address gant, gradle, buildr, and simple build tool at the very least.

    Reply
  26. mark
    Tuesday, October 514, 2010

    The Buildr guys are appearing in January at the San Francisco Java User Group, fyi:

    http://www.sfjava.org/calendar/14894440/?from=list&offset=0

    Reply
  27. Matt Doar
    Tuesday, October 559, 2010

    Bill wrote: minor spelling correction
    >you will need to peak under the covers.

    Actually, “peaking under the covers” sounds rather more exciting than build tools!

    Reply
  28. Dean Schulze
    Tuesday, October 521, 2010

    One big reason holding back the adoption of Groovy-based alternatives like AntBuilder, Gant, and Gradle is the lack of any useful documentation. If you take a look at the AntBuilder documentation (http://groovy.codehaus.org/gapi/) you will not find anything about using javac, war, jar or any of the other Ant targets. Groovy uses dynamic method invocation so the appropriate Ant target call does not show up in the Javadoc – or anywhere else for that matter.

    Same situation with Gant. No docs on how to actually use it.

    Gradle is an astronautical approach to … something. I havent’ figured out what yet.

    If you want to convert the simplest Ant script to Gradle the Gradle docs are so bad that you can’t find how to do it. Here’s a thread on the Gradle mail list about converting a very simple Ant script to Gradle:

    http://gradle.1045684.n5.nabble.com/Example-or-tutorial-of-migrating-from-Ant-to-Gradle-td2852129.html#a2852129

    14 responses and 3 days later and I was almost there. Here’s a taste of what’s in store for you with Gradle. In Gradle your javac classpath is called a dependency, not a classpath, but they don’t bother explaining that basic fact anywhere.

    I don’t want to try to learn Gradle with an astronautical project. I want to transition to it starting with a simple project, but even that little bit is undocumented. (I dont’ think any of the projects I’ve worked on conform to Gradle’s convention, so their plugins won’t work.)

    Gradle is an example of how to make a simple thing difficult with Astronautical Architecture.

    I’m moving ahead with Groovy and AntBuilder. I’ll have to discover what the allowed method calls are and what parameters are required and what they are called, but it shouldn’t be that difficult. I’m hoping a good dive into the AntBuilder code will answer my questions.

    If the people on the Gradle project would just write some concise, useful documentation there would be a lot more adoption of Gradle. But the existing Gradle user guide is worthless.

    Lack of documentation is what is holding back adoption of Groovy-based alternatives.

    Reply
  29. Benjamin Darfler
    Tuesday, October 538, 2010

    @dean I’m surprised you had a lot of trouble with gant, its basically and where you turn your angle brackets in to parens. There is a fair amount of documentation and even an ant2gant script. The nice part of all of that is that you can leverage any ant tasks that third parties might supply since its mostly just syntactical sugar on top of ant.

    Reply
  30. lance bradley
    Tuesday, October 532, 2010

    We’ve been working on a build alternative for clojure called cake, which is heavily inspired by rake. It could also be a java solution if you don’t mind writing builds in clojure. It’s dependency based and compatible with maven repos, making heavy use the ant api and maven ant tasks internally. http://github.com/ninjudd/cake

    Reply
  31. Dean Schulze
    Tuesday, October 525, 2010

    @Benjamin,

    I didn’t spend any time with Gant once I saw there was no documentation. It looks like a wrapper around AntBuilder. I’ve noticed that Gant is bundled with some distributions of Groovy.

    Ant is adequate for building binaries. What I really need is a better way of starting and stopping servers and cleaning deployment directories. With Ant it is difficult to do things like

    check if a server is running
    Start it if it is not running
    Shut down the server if it is running
    Wait until the server is stopped to clean the deploy directories

    Anything requiring if … else logic is awkward in Ant.

    Groovy with AntBuilder should give me what I need for these basic tasks.

    Reply
  32. Matt B
    Tuesday, October 502, 2010

    @Dave is correct. The argument that this article makes about Maven builds being non-deterministic is a fallacy. This is only true if you do not specify a version for your dependencies and plugins. When you do not specify a version, you are telling Maven – and this is documented – “used the latest version you can find”.

    To produce a reliable (and sane) build, you should always specify the exact version of your dependencies and plugins.

    Reply
  33. Alex Boisvert
    Tuesday, October 513, 2010

    +1 for buildr (http://buildr.apache.org)

    Buildr is based on Rake, leverages the Ruby language nicely, integrates with Java via RJB and Antwrap. Runs great on JRuby.

    alex

    Reply
  34. Nick Burch
    Tuesday, October 501, 2010

    JavaScript is the answer!

    Use inline JavaScript in your ant build files. You’ll want Ant 1.7 + Java 1.6, but that’s not too hard a requirement these days. Then, every time you hit something hard to do in ant’s xml, drop into JS and code away in that. You get full access to all the ant tasks, targets and variables, plus you can set your own properties to be used by java when you’re done. It’s so much easier than the old way of using custom tasks and targets in java

    Reply
  35. Ton van Bart
    Tuesday, October 512, 2010

    @MattB +1 on the versions *but* I’ve had Maven dependencies disappear on me from Maven Central before. In our shop we use Nexus and force all dependencies to come from there – nothing comes in automatically.
    For the rest the I think article is spot on. Also note that some Maven plugins can be very poorly documented and/or the docs can be outdated. IMHO we still spend far too much time wrestling with our build tool to get it to do what we want.

    Reply
  36. Benjamin Darfler
    Tuesday, October 501, 2010

    @Dean, maybe, I’ve never played with AntBuilder but Gant is super easy, has documentation and is much better at dependency management than AntBuilder if that is a problem you need to address.

    Reply
  37. Mike
    Tuesday, October 558, 2010

    Actually, (GNU) make works fine under Window. The fatal problem, of course, is that for more than a decade, Windows has shipped with a completely emasculated shell environment (Eunuchs, as it were).

    If all of the time and effort that have been spent assembling pretty mediocre alternatives to make had been spent instead fixing Windows’ shell environment, Java programmers would have a nice clean powerful build system, instead of going blind trying to read and write huge XML files.

    Reply
  38. Nello
    Wednesday, October 615, 2010

    I’ve used all of these build systems in real projects now, and have been almost uniformly disappointed.

    Buildr looks good until you realise it doesn’t support transitive dependencies: i.e. it’s close to useless. Gradle looks great, but seems to excel at hiding information to the point where no-one really knows what is going on. Ant and Maven have been adequately dissected above (nicely put, btw.)

    My current team is using Rake, and for me it wins hands-down. It’s terse, and has a real descriptive language as its basis but still works as a modular declarative build system. It does remind me of Make, but only the good bits.

    Reply
  39. Englebart
    Wednesday, October 638, 2010

    We use a simple input XML document that lists projects plus which outputs are required for each project as an attribute.

    Combine this with an XSLT to generate the ANT script.

    o projects.xml is 6.6K bytes, 1/3 of which is a comment block. It describes 70 projects.
    It could also describe libraries, etc.

    o ANT.xsl is 26K, again lots of comments.

    o build.xml is 92K, lots of sections of 70 x project_dependent_definition

    We have 4 different project.xml variations, all of them use the same XSL.

    Most common:
    You add a new project, new library, etc. put one line in projects.xml, and run a transform. Done!

    Less common:
    You need a new application, create a new projects.xml. Run the transform. Done!

    Very rare:
    You need a new output from your ANT script. Update the XSL, Transform, Test!

    Using this approach preserved my sanity since the majority of the build.xml is highly repetitive.

    Reply
  40. Jordan Zimmerman
    Wednesday, October 611, 2010

    The problem with all of these tools is documentation. I don’t want to learn yet-another-tool with yet-another-xml-format with yet-another-religion of plugins, idioms, etc. Maven is fine once you understand it. Understanding it is the problem. Finding out how to accomplish a given task can be excruciating.

    What I want is great GUI front-ends for these tools. Before IDEs we had a mish-mash of command line tools. text files, etc. IDEs made life orders of magnitude easier. IntelliJ supports Maven somewhat. What we need is the IntelliJ equivalent for Maven (or whatever build/repository tool).

    Reply
  41. Andrew Wright
    Wednesday, October 603, 2010

    Also worth checking out:
    leiningen – build tool built in clojure (http://github.com/technomancy/leiningen)
    sbt – Simple build too. Scala based. (http://code.google.com/p/simple-build-tool/)

    Reply
  42. Joe H.
    Wednesday, October 648, 2010

    -45,000,000 for quoting the worst ever temper tantrum by Kent Spilner. I stopped reading the article at that point because anything else you might have to say about anything is ruined by the fact you would believe and quote such a childish rant.

    Reply
  43. Dimitar
    Saturday, October 922, 2010

    I wrote an article some time ago and I reached somewhat different conclusions: http://foobarbazqux.blogspot.com/2007/11/building-tools.html

    Overall, many projects and companies are building hugely successful products using Make, Ant and Maven. Also, huge number of contractors churn VB applications with nothing more than Visual Studio.
    We should really focus on what are the current problems and how to solve them better, rather than search the mythical ‘best tool’.

    Reply
  44. john
    Saturday, October 905, 2010

    I think maven has some fault for a non-reproducible build structure, but not all. Maven 3 is just released and has solved many problems of its predecessors. Outside of maven’s scope of control though is this concept of a global dependency tree. Over this past decade, open source projects have been transformed. No longer is it simply to build a stand-alone component. Projects have dependencies on other public projects. The job of tracking dependencies is complex and needs to be segmented. Identifiying a project’s dependencies belongs to the projects producer, not consumer. Component producers must be brought to task for weak dependency definitions. Any dependency resolution tool suffers from the fault of the component producers who neglect their own dependency tree.

    Reply
  45. Bryan
    Sunday, October 1053, 2010

    I chose Gradle as well over Maven and the others. Someone mentioned poor docs for Gradle which is baffling to me. Gradle has very good docs. Gradle has the obvious benefit of Groovy, Ivy based dependency management, and full access to Ant tasks and extensions for backward compatibility. Another big win is project dependencies where you can specify dependencies to other projects on the file system. If you make a change in one project and compile the other, everything will be compiled as needed. Maven is a nightmare if you have multiple projects that are being actively developed together.

    Reply
  46. vasya10
    Wednesday, November 1205, 2014

    Gradle is one good answer to CI build systems, where Maven miserably fails. Gradle has learnt from the mistakes of Maven and similar tools, it has an awesome DSL, it is programmable and pragmatic. I have successfully worked on projects migrating from maven to gradle and have been very happy. And developers are happy, not having to fight with a monster called xml.

    Reply
  47. jerome angibaud
    Thursday, December 1056, 2015

    There’s also Jerkar http://project.jerkar.org which is pure Java.

    Reply

Leave a Reply