Java Builds with Gradle

By Benjamin | Published: January 10, 2011

I’ve been working on Java projects for years now and the two primary build systems for such projects have been Ant and (more recently) Maven. Both have given me headaches for various reasons and lately I just find Maven increasingly cumbersome to use. Yes, the dependency management features are great (most of the time) and the multi-project support works well, but sometimes I need it to do more (or in some cases less) and that ends up being harder than it should be. Then there’s all that XML. Hundreds to thousands of lines of it with what seems like more noise than usefulness. Maybe it’s just me, but I expect my build script to look more like … well, a script.

So I’ve been looking for alternatives and it looks like there are a few to choose from. The Apache Software Foundation has one called Buildr which is Ruby based and therefore outside of my current comfort zone. The other big frontrunner is a tool called Gradle. This is a Groovy-based DSL that combines the good things about Ant and Maven into a concise and extensible syntax that is backed by a dynamic language. I could tell you more about it, but the Gradle website does so much better than I could. Besides, I think a demonstration is much more useful.

The Example

Let’s say we have a multi-module project that uses Maven conventions. The three modules are ‘core’, ‘engine’ and ‘webapp’ which result in two jar files and one war file. Furthermore, the engine depends on the core and the webapp depends on the engine. With me so far? Good. So how do we build such a project in Gradle? First, we need to create a file called settings.gradle in the root project folder to tell Gradle what the subprojects are. Here’s what goes inside it for our hypothetical project:

include "core", "engine", "webapp"

One line, zero XML. Life is good. Now we need to create a file called build.gradle that will contain common information for the subprojects.

subprojects {
  apply plugin: 'java'
  apply plugin: 'eclipse'

  version = '1.0'
  projectName = 'ballista'
  group = 'com.ioglyph.ballista'

  repositories {
    mavenCentral()
  }

  dependencies {
    compile "org.slf4j:slf4j-api:1.6.1"
    runtime "org.slf4j:slf4j-log4j12:1.6.1"
    runtime "log4j:log4j:1.2.16"
    testCompile group: 'junit', name: 'junit', version: '4.+'
  }

  jar {
    baseName = projectName + '-' + baseName
  }
}

I admit, I’m throwing a lot at you at once so let’s take a look. We’re telling Gradle that our modules use the ‘java’ and ‘eclipse’ plugins (there are other plugins available, including support for groovy and scala source code). All the modules are version 1.0, the base project name is ‘ballista’ (that’s my own custom variable, not anything like a Maven artifactId) and the group for Maven repository purposes is ‘com.ioglyph.ballista’. Yes, Gradle supports downloading from and uploading to Maven repositories. The ‘repositories’ block illustrates that – we’re telling Gradle to pull dependencies from the Maven central repository. Additional repositories could be added to this section if needed. Next, we define our ‘dependencies’ block. Each line is equivalent to an entire ‘dependency’ tag in a Maven pom.xml file. We have various scopes in use here and I’ve shown two different ways to define the artifact. The first three use the format “group:artifact:version” whereas the the fourth explicitly delineates those three properties.

At the end you’ll notice we’ve added a bit to the Gradle ‘jar’ task. The subprojects will get the basename for their jar files from the name of the subdirectory they are in, so our jars would end up being called core-1.0.jar and engine-1.0.jar which isn’t what we want. Instead, we’re telling Gradle that the basename for the jar should be prepended with the value of the ‘projectName’ variable defined earlier. Now they’ll end up being called ballista-core-1.0.jar and ballista-engine-1.0.jar. We’ll still need to address the name of the war file for the webapp, but we’ll get to that.

Now that we have the basic information for the project, it’s time to work with the subprojects. First off, the core/build.gradle file.

dependencies{

}

That’s it. Yes, it’s essentially empty, but if the core needed any dependencies not defined in the parent build.gradle we would add them here. Next, the engine/build.gradle file.

dependencies {
  compile project(':core')
}

Here we’ve defined the dependency between the ‘core’ subproject and the ‘engine’ subproject. As mentioned previously, any dependencies not defined in the parent could be added here. Finally, we look at the webapp/build.gradle file.

apply plugin: 'war'
apply plugin: 'jetty'

dependencies {
  compile project(':engine')
  compile "org.apache.wicket:wicket:1.4.15"
}

war {
  baseName = projectName
}

sourceSets{
  main{
    resources{
      srcDir 'src/main/java'
      srcDir 'src/main/resources'
    }
  }
}

Now we’re getting somewhere. We’ve added a couple plugins, one to create a war file and the other to run the Jetty servlet container with our webapp deployed. Additionally, we’ve defined the dependency between the ‘engine’ subproject and the webapp. This will pull in the ‘core’ jar file as well unless we wanted to disable that transitive dependency. In this case we don’t. An additional dependency was added to the Apache Wicket project (if you aren’t familiar with it, I recommend taking a look after you’re done here).

We’ve extended the default ‘war’ task by defining the base name of the war file to equal the value of the projectName variable. You’ll noticed that variable isn’t defined in this file but it was defined in the parent build.gradle file. So now the final war file name will be ballista-1.0.war. Finally, we had to extend the source sets that Gradle moves during the build process. Under normal circumstances, only the Java source files are compiled and moved to the build directory. If you’re familiar with Wicket, you know that there are HTML files that need to move over as well. This ‘sourceSets’ extension takes care of that for us.

Now when we run gradle build from the root of the project, we see the following output (artifact downloads have been omitted):

:core:compileJava
:core:processResources
:core:classes
:core:jar
:core:assemble
:core:compileTestJava
:core:processTestResources
:core:testClasses
:core:test
:core:check
:core:build
:engine:compileJava
:engine:processResources
:engine:classes
:engine:jar
:engine:assemble
:engine:compileTestJava
:engine:processTestResources
:engine:testClasses
:engine:test
:engine:check
:engine:build
:webapp:compileJava
:webapp:processResources
:webapp:classes
:webapp:jar SKIPPED
:webapp:war
:webapp:assemble
:webapp:compileTestJava
:webapp:processTestResources
:webapp:testClasses
:webapp:test
:webapp:check
:webapp:build

Now in each of the subproject folders you’ll see the appropriate artifacts in the build/libs directories.  If you crack open the webapp/build/libs/ballista-1.0.war file, you’ll see the various dependencies have been included as expected. With the Jetty plugin applied we can run the webapp by issuing the command gradle jettyRun and navigating to localhost:8080/ballista to verify that the webapp runs as expected.

I’ve only scratched the surface of what Gradle can do as I’m still learning how to use it. Additional information can be found on the Gradle web site. There they have an extensive user guide that contains most everything you will need to make Gradle work for what you need it to do. I recommend checking it out and downloading the latest version. If you’d like to view the source code and build files used in this post, you can find a link below (you’ll need to have Gradle installed to run the builds).

gradle-sample.zip

Posted in Java | Comments closed

Stop Doing Agile

By Benjamin | Published: November 10, 2010

I mentioned some time back that where I work we do Agile software development. This phrase, though accurate in some sense, shows a certain misunderstanding of what Agile is. Sure, you might ‘do’ Scrum, or ‘practice’ XP or Kanban or Lean or whatever, but you don’t ‘do’ Agile. You adopt these processes as a starting point so you can be Agile, and being Agile is worlds more important than doing Agile.

Doing Agile implies the mechanical application of some set of practices with no understanding of or commitment to the principles and values underlying them. Frequently this comes about because the decision to do Agile is made from the top down rather than the bottom up. If it isn’t fully embraced by the team with energy and commitment then they’re just going through the motions. Yes, they’re doing something and in some ways it might work, but they aren’t being Agile.

The issue can be further complicated when ‘doing’ some Agile process is considered more important than ‘being’ Agile. I refer to this as ‘process attachment’ and it likely happens with Scrum more often than with XP or the other Agile variants (because Scrum has a broader adoption base and more of a project management focus). This attachment manifests as a viewpoint or environment where all other methodologies are disallowed and deviation from the One True Process is frowned upon. What those fixated on the rigid adherence to the letter of a certain process miss is that this attachment to methodology is precisely one of the reasons the Agile movement started in the first place. That’s why point one of the manifesto is “Individuals and interactions over processes and tools”. Process attachment runs counter to the core of Agile because the process is of minimal concern compared to the project and the team.

By all means, start with Scrum or XP or Kanban or Lean, but after a number of iterations there should be some drift. This is because fully agile teams will find a steady-state in a zone with maximum productivity and minimum process. Agile incorporates a feedback loop every iteration for this very purpose. Retrospectives are the crucible of change for how you operate. Iterations aren’t just about the project, they also apply to refining and improving the process.

“Hold on, Ben,” those with process attachment might say. “We need to standardize on one process so we can move resources around and everyone is on the same page about how things work.” No, no, no. These resources you speak of are people, and people can’t be abstracted down to cogs in a development process machine. People bring too many variables and different strengths, weaknesses and personalities. Different teams of people have different dynamics which yield different optimal environments. Good teams can succeed with almost any process, but they can deliver better and faster when they define how they work. Process for the sake of process at that point is (at best) waste and (at worst) counter-productive.

Agile methodologies are useful tools, but Agile itself is more of a philosophy. Find energetic teams of people that are committed to the principles and values and let them decide how they work best. If the organization requires certain metrics or data points, let them know and they’ll deliver. Beyond that, leave them alone and let them build the solutions. At the end, it may not be exactly Scrum or XP that they’re doing but they will be as efficient and effective as possible. All because they’re being Agile, not doing Agile.

Posted in Agile | Comments closed

TechZing – Embrace the Morning Commute

By Benjamin | Published: November 4, 2010

I mentioned last time that I was going to try listening to podcasts as part of an optimized information intake process. I found a few that looked interesting and downloaded several episodes of each. I happened to start with one called TechZing with Justin Vincent and Jason Roberts (and really, with a tag line like “If you’re a hacker, you’ll probably like our show”, how could I start anywhere else). Turns out, it was a good podcast to start with.

TechZing has episodes that discuss a variety of topics related to programming, applications and tech entrepreneurship along with interviews of some interesting people in the industry. While I’m a Java developer in an enterprise IT department, the topics Jason and Justin bring up almost make me wish I was running a small consulting company or lean Start-up with no cash and a pantry half-full of Top Ramen. Unfortunately, Ramen noodles aren’t exactly a favorite of my wife and kids, hence the ‘almost’ qualifier. In any case, their conversational style is genuine and entertaining. I’d love to work with either of them except that would mean coding in PHP (yes, that’s a joke). As an added benefit, they discuss the more interesting posts and topics that I would have found on Hacker News, so they act as a good tech content filter.

I plan on trying out the StackOverflow podcast and probably This Developer’s Life, but I repeatedly find myself downloading previous episodes of TechZing that look interesting. With 82 episodes and topics like Modeling Disruption, AI, NoSQL, Erlang, Long Tail Optimization, UFOs, HTML 5, an almost disturbing number of references to Twitter, and a great deal more, I might be listening to back episodes for a while. I recommended it for hackers, start-up entrepreneurs, developers (unless you’re a Ruby-ist that considers PHP to be the scourge of programming) and those who just enjoy tech and entertaining discussion.

Posted in Reviews | Comments closed

Intake Optimization

By Benjamin | Published: October 31, 2010

Time is one of the few constants of human experience. At a fundamental level, our time is the only thing that we can really own or truly spend. We can trade it for money which we can in turn trade for things, but our lives ultimately come down to our very small and finite amount of time and how we use it. Distractions abound in our lives, many of them empty vacuums where we lose hours of our most precious resource.

There is a spectrum of fulfillment, meaning and utility that everyone must define for themselves. For me, most of the web lives at the low end of that spectrum around the same area as television. Video games and movies rank slightly higher if only because of potential storytelling value. Above all of those are books and the intelligent and meaningful content that writers have shared on their websites. Even at the high end of that spectrum, I realized I was spending too much time reading and not enough time doing. Creating is ultimately a more meaningful and rewarding endeavor than consuming.

So I realized it was time to attempt some optimizations. Television was the first (and easiest) thing to cut. The few TV shows that I did watch regularly will eventually be out on DVD where they can be watched commercial free and on my terms (strange as it might seem, I’ve never owned a DVR). Admittedly I didn’t watch much TV to begin with, so this only freed an hour or so a day. It was slightly more difficult to cut several decent feeds from Google Reader, but I set myself a limit of 20 and made sure to remove the post-heavy aggregation sites. DZone and Hacker News feeds easily reach 100 posts a day, though I admit I will likely visit the front page of Hacker News on occasion.

An additional optimization is to replace radio with podcasts during my commute. This might seem to go the other way in terms of consuming more content, but it’s already time lost driving back and forth to work each day. Not sure which podcasts to start with, but there are certainly plenty to chose from on a variety of topics. I considered going with audio books, but when listening to them previously I found it very difficult to turn them off when I reached my destination. Because podcasts are shorter, they are a better fit for the time-slice.

So what will I do with the extra time that these optimizations free up? My plan is to put some much needed work into this site, practice coding, spend more meaningful time with friends and family, exercise more and put some time to other ideas that I can work on. I still plan on reading quite a bit, watching movies and maybe playing the occasional video game. The downtime is useful and perhaps necessary as we can’t (or at least shouldn’t) spend all of our time creating. As with all things, it is all about balance.

Posted in Uncategorized | Comments closed

The Agile Analyst

By Benjamin | Published: October 28, 2010

Traditional software development has many roles – architects, systems analysts, quality assurance, developers, project managers, the list goes on. Agile development doesn’t define nearly as many roles, simply that teams are cross-functional groups of five to nine people with some type of facilitator role. Most Agile variants assume that most of the team will act in a developer role. What then becomes of those other people that aren’t developers?

The Analyst

I’ll get to the other roles another time. For now, I’m going to focus on the role of the Analyst. Prior to Agile, Systems Analysts did a lot of talking, listening and writing. In a Waterfall world, they produced copious amounts of documentation, primarily in the form of gripping requirements. These would clearly spell out what the software would do if only those reading could stay awake long enough to make it through all those pages of directives with a clearly defined numeric scheme. You see headings like Requirement 6.7.4.1.3. How we produced anything beyond a collective sleep-over remains a mystery.

Enter Agile

Agile is theoretically lighter in documentation than earlier approaches. Where then do the analysts go? Typically back to the keyboard to write User Stories. If the Agile process is being misapplied, these frequently look a lot like requirements, especially for analysts that have done waterfall. Sure, the numbering scheme might have dropped a couple subsections, but they’re still writing requirements. User stories are not requirements, they are indefinite placeholders for discussion and understanding. Furthermore, they should be written by the customer (not the analyst) as the primary influence mechanism for the project.

False Separations

In some organizations, you frequently end up with two types of analysts. Business Analysts and Systems Analysts. I don’t appreciate this dichotomy because there is only one type of Analyst that we should care about and that is the Problem Analyst. The Problem Analyst works ahead of the developers in an effort to find the problem that needs to be solved. This is harder than it sounds, because while your users might be able to go on at length about ‘A’ problem, even they may not know what ‘The’ problem is.

The Agile Analyst

The Agile (Problem) Analyst should:

  • Ask why. Ask it five times in fact. Users know they need something but they may not really know what the heart of the problem is. Asking why in an iterative fashion can reveal the true problem and it may have a simpler solution. The Five Whys is a powerful technique for enabling this discovery. It is usually applied to root-cause analysis of a defect but it is a problem solving technique and software development is about solving the right problem.
  • Answer questions. Given the depth of their knowledge, analysts will know and understand the business better than anyone in the room (including the users in some cases). Developers will have questions and the Analyst should be able to answer them or be able to quickly find the answer.
  • Work with the onsite customer, developers and testers to identify and maintain the acceptance criteria. If these tests can be formalized in automated integration tests, even better. There are tools that make these types of tests maintainable by non-developers and such maintenance may fall to the Analyst. If so, embrace it because this can be an exceptionally powerful tool.
  • Act as the loudest user experience advocate in the room. The final product should solve the problem in the best way possible given the project constraints. The analysts knows the needs of the user and how they work day to day and should make certain the customer will find the final deliverable usable and effective. Just remember, most developers interface with computers in ways that most users will find incomprehensible (I know, I’m a developer and most of us would probably deliver command line applications with esoteric control languages if left to our own devices).
  • Work with the team and the customer to control scope and find acceptable compromises. In a way, this is the flip side of user experience advocacy. Some of this work may fall to the team facilitator (assuming that isn’t already the Analyst), but the Analyst is the natural axis point for this conversation given their work with the users.
  • Work with the Product Owner or Onsite Customer to develop and maintain high level functional specs with UI mockups or story boards. Don’t worry about how code will work internally, just how it is expected behave in the hands of the user. The functional spec is a living document. It is also not a technical requirements document. Joel Spolsky of Joel on Software discusses his take on functional specs in four parts here. While some of it may be overkill for the Agile process, it is a good starting point.

Agile methodologies leave the role of the analyst largely undefined. The unfortunate side effect of this is that it leaves organizations transitioning to Agile with analysts who are uncertain about their role (or worse their future).  This is unnecessary as there remains plenty of work that analysts are suited to doing in an Agile environment to help their teams deliver quality software.

Posted in Agile | Comments closed

Going Forward

By Benjamin | Published: October 25, 2010

As I mentioned in the last post, I’ve been doing some reflecting recently, both on how this site is going (or, of late, not going) and about life in general. In doing so, I’ve learned a bit about myself. Of course, you’d think after being so close to me for all of my life that I’d know me pretty well. To a degree, yes. But things change and I think it’s important to periodically examine your life, where you are and where you want to be. That process and my answers to it are largely beyond the scope of this post which focuses on the tangible and intangible things I will do with ioglyph.com.

Structure and Design
I can make some improvements. My plan is to play around with some ideas and make some changes. When I set this site up, I borrowed some ideas from other sites that I plan on revisiting. The projects page might change and other areas can be moved around or removed entirely. The color scheme could even be changed – given that I’m color blind that could be dangerous but it should convey just how far I’m willing to go in this endeavor.

Project
I need a problem. Not just any problem, but a problem that speaks to me. One that can be solved with code that I want to write. It’s a lot easier to work on a project that you love that solves a problem that you love than one that is missing one or both of those aspects. My previous problem was building this site and once it was up and running the problem was no longer there. So it’s time to find a new problem. I’m still brainstorming different ideas, but once I’ve settled on one or three, I plan on writing code and talking about it on here. Which brings us to the next point.

Knowledge
I work in a knowledge-based industry as part of a Java-centric development group. I think it’s time to expand my cognitive horizons. Working on this site keeps my HTML and CSS skills reasonably sharp (or at least sharper than they would be not working on this site), but more work with JavaScript and jQuery is probably a good idea. Beyond that I’m going to spend some time focusing on Python followed by a functional language, likely Erlang.

Content
I’ve written a few blog posts and I have several that remain in some half-done limbo on my local machine. Most have been about software development to some degree, but there’s more to me and this site than that. I plan to explore that more as I’m considering the design changes. I have several interests and ideas that I will write about. No idea if they’ll be included in my blog or somewhere else. We’ll see how things turn out.

Focus
Distraction is mental entropy. Everyone is prone to it and it requires effort to not allow distraction to consume all of our free time. Some distraction is okay, probably even necessary, but not all distraction is created equal. Of late, I’ve spent a good deal of time reading and more time than I’d like watching television. I consider the reading to be time better spent. Even so, I can likely get by reading fewer pages, blog posts or articles. I’ve realized that I need to commit to being focused on what interests me in order to accomplish what I want. This means taking time every day to deliberately work on either the web site or one of my numerous other interests.

Courage
Fear is the mind-killer. Everyone has their own unique set of fears and worries. One of mine is the fear of not being good enough to meet my personal expectations or that my best efforts will result in failure. Time to move past that. Not trying is certainly easier, but letting fear win usually is. I may not be as great as I’d like to be, what matters is being better than I am and that means working through the cognitive resistance and trying. Courage is not the absence of fear, it is the ability to do what must be done in spite of it.

Posted in Uncategorized | Comments closed

Reflections

By Benjamin | Published: October 22, 2010

It’s been three months since I started this site and quite some time has passed since my last post. Over the past few days I took some time to reflect on why that is. I wish I could say “I’ve been busy”. While I have been somewhat, it’s no busier than I was when I was pouring hours into the site trying to get it up and running. At first, I thought it was simple writer’s block. Then one week turned into two which became a month and then I had to wonder what happened to the energy I had when I started working on this project.

So I’ve given it some thought during this whole Socratic self-examination that I’ve been doing. Turns out, I’m not very happy with the site. Now admittedly, there are a number of things in my life that aren’t going the way I’d like. So it goes. This site, however, should be different. The key difference being that this site is completely within my control whereas the rest of reality is not (as of yet). In any case, there are a number of reasons why I haven’t done much with ioglyph in the last month or so.

First off, it was starting to feel like a job and I don’t want that. I expect it to feel like work, but (for most people) your work is different from your job. This site is supposed to be about my own personal development, finding my voice and writing about things I care about. These are things that I’m passionate about and want to work on. So why would it feel like a job? Could be for a couple of reasons. I’m not writing about the right things or taking the wrong approach. The work that I’m doing is too close to what I do during the day. The design, feel and content aren’t as complete or as good as I hoped it would be. It’s too much like other sites and not enough of my voice and vision comes through.

On a more personal level, it comes down to three things. Fear, boredom and (for lack of a better term) ennui. Or it could also be characterized as a lack of three things – courage, motivation and fulfillment. These demotivating factors have been present in my life beyond the site, both in my personal and professional life (hence the self-examination mentioned earlier). I think it is time to face them, figure out what to do about them and move past them.

What’s to be afraid of? I suppose that depends on the person. For me, it boils down to a couple things. The first, and what turns out to be the lesser of the two by orders of magnitude, is the fear that other people won’t find it interesting. By other people I don’t mean the great majority of the web population, but the smaller, technically-oriented subset of people that are my virtual peers. What if you build a site and no one shows up? I’m not usually the type of person that worries about the opinions of others, but we are social animals so this fear arises from the recesses of our primitive ancestry. This is a nuisance that I’m able to ignore most of the time. The greater fear is that I’ll find that I’m not as good as I should be. Whether in design, or writing, or code, whatever the work, I have expectations for how well I should be able to do them. The fear of not living up to your own expectations can stop you from trying. The fact that I set my expectations for myself as high as I do doesn’t help. We should want to be good at what we do. Hell, we should want to be great at what we do. Yet that drive can so easily be subverted by the fear of failing to be great.

As to boredom, it’s difficult to say when that started and it’s entirely possible that it is closely linked with the ennui / apathetic moodiness that the modern world makes all too easy for me. I simply haven’t found the site as interesting as I thought I would or that it was even worth it to work on it. External influences have contributed to this, but I find those to have less of an impact than the internal responses that arise in reaction to them. Compound that with the fact that it’s all too easy to allow the many distractions that surround us to consume what free time we have and you have a recipe for wasted time and lack of fulfillment.

So there are some of the reasons that ioglyph.com has not seen a lot of activity of late. Next time I’ll talk a bit about this Socratic inward journey I’m on, what I intend to do about the issues that I’ve let prevent me from staying focused and what I plan to do about this site.

Posted in Uncategorized | Comments closed

Classic Blunders

By Benjamin | Published: September 2, 2010

Today I almost fell prey to one of the classic blunders. No, I’m not talking about land wars in Asia or going in against a Sicilian when death is on the line. This was one of the classic software development blunders. There are several of these, but the one I’m referring to is building something now because you think it might be needed later. Fortunately, I came to my senses in time to see what I was doing.

Premature enhancement is second only to premature optimization as the root of all software evil. It’s a deceptive trap, and it usually manifests as follows. You’re designing and building software and you see something in the user stories, or wireframes, or cave drawings, or requirements, or whatever it is that you use to convey the framework of your solution. This something catches your eye because it appears to be future malleable. This something could easily turn into That something, so why don’t we go ahead and build That something now to make it easier in the future. The temptation is especially strong when That something seems like a more interesting problem than This something. Stronger still when That something appears to be only a little bit more work beyond This something.

This is a problem because the goal of every pragmatist in software development should be to reduce or manage complexity in the most effective way possible. Adding functionality that will not be used does the opposite. Nevermind that it might get used in the future, it’s not worth the additional trouble in the present. One reason it isn’t worth the trouble is that the change you’re expecting might never be necessary. If it never comes to pass, you have unnecessary complexity that will make the code harder to understand, both for yourself and other developers. Since we only have a finite amount of conceptual space to work with, it’s important that we aim for simplicity and elegance over complexity and future-proofing.

Or perhaps that change you’re envisioning might become necessary (it could happen though it often does not). By the time it does, additional requirements or a change in your way of thinking could invalidate your previous attempts at a premature enhancement. A better design might come to mind and what you built before turns out to be wasted effort. Or worse, your attachment to what you wrote previously prevents you from seeing a better design. The premature enhancements you made put unnecessary and limiting constraints on the solution space.

Premature enhancement isn’t just about complexity, it is also about time. Every hour spent building something that might be needed later is taking time away from what is needed now. We’re paid to deliver solutions to problems, the real and immediate problems that are in scope – not the potential problems that may arise in the future. Delivering effectively means focusing on what is essential, building what is essential in the best way possible and getting it in the hands of the user.

I’m not saying we should ignore future needs. By all means, think about what might change, consider how such changes might impact your design, identify the extension points that might be necessary, discuss how likely it is with your customer. But don’t build it unless you need it for what you are working on now. To use the storytelling metaphor, premature enhancement is like writing a sequel to a story you haven’t finished yet. It is gambling on a possible future at the expense of the known present. Do yourself a favor and don’t take the bet.

Posted in Development | Comments closed

Enterprise Viscosity

By Benjamin | Published: August 26, 2010

Agile software development includes a set of terms which convey the forward progress of team. These include concepts like ‘Velocity’ and ‘Burndown’. One day, quite by accident, I was saying something witty or sarcastic and introduced the phrase Enterprise Viscosity. While I don’t advocate making it an official part of any parlance, it’s a useful idea and something to look for when projects proceed slower than expected.

Enterprise Viscosity is the sum of all factors great and small that do not contribute to moving a project forward. It is the amount of friction, drag, resistance or impedance that slows projects down by nature of the environment, culture or people. Larger organizations are more likely to experience Enterprise Viscosity than smaller ones by nature of the communication overhead and the intersections of various disparate groups with different functions (hence the ‘Enterprise’ qualifier). Even so, smaller groups can experience high levels if the circumstances are unfortunate enough.

There is no such thing as a software project that encounters zero problems. You can’t completely eliminate friction regardless of how smooth you try to make things. This applies in software development just as much as in physics. There will always be some level of resistance and that’s to be expected. The goal of any group or company should be to reduce enterprise viscosity to the lowest level possible.

Enterprise Viscosity can take many forms and left unchecked can have a devastating impact on team morale, productivity, job satisfaction, motivation and all the other factors we expect to see in remarkable people and teams. Below are some examples of unnecessary drag at varying degrees of severity that can occur:

  • That weekly staff meeting that takes an hour to deliver 10 minutes of useful information.
  • Office politics of any kind that reaches the team level.
  • Mandates that force the use of inferior tools.
  • Needless bureaucracy.
  • Management ‘dropping by’ frequently to see if the project is ‘on track’.
  • Days broken into 30 minute or 1 hour chunks due to meetings.
  • Shuffling people on or off teams that are working well.
  • Urgent or critical issues that turn out to be neither urgent nor critical.
  • Referring to people as resources (or worse treating them that way).
  • Placing more emphasis on process than results.
  • Committing to Agile methods without committing to Agile philosophies.
  • Any meeting that starts with no agenda, lasts longer than twenty minutes or ends with no action items or resolution.

The common themes above are interruptions, lack of focus and poor management. If you are an Agile Manager, one of the primary metrics of your effectiveness must be how you reduce Enterprise Viscosity – wherever and however it manifests. Remove the resistance and get out of the way so your teams can do what you hired them to do.

Posted in Agile | Comments closed

The Metaphors We Build By

By Benjamin | Published: August 20, 2010

The practice of software development is frequently described in terms of metaphors. The point being to establish a non-obvious connection or analogy between the creation of abstract logical structures executed by computers and other pursuits that are somehow similar. Metaphors are powerful cognitive constructs by which two words or ideas are linked by association or identity even if they seem dissimilar.

If someone said, “her hair was spun gold in the morning light”, they would be making a metaphor (perhaps a bad one, but a metaphor just the same). A connection is made between dissimilar things, hair and gold in this case. Metaphors work because they allow our mind to look at the objects in question from a different perspective.

Metaphors abound when it comes to computers. Recycle bins where deleted files go are in no way ‘green’. Files themselves aren’t actual files like you would find in a filing cabinet and the directory tree that you find those computer files in do not germinate from seeds and grow in forests. Processes can be widows, orphans, or zombies, even though none have ever had a spouse, appeared in a Dickens novel, or been killed by a man named Ash sporting a chainsaw for a hand. These metaphors have been in place for so long that we operate with the metaphors as models until something interferes with the connection[1].

Using metaphors in this fashion shows just how much we connect abstractions and how these connections can act as point of reference or a way of working with complex subjects. Extreme Programming recognizes the power of metaphor by advocating a System Metaphor that can actually aid in the design of software and answer questions about implementation during development. This can be powerful, if such a metaphor can be found.

As with all mental models, metaphors do break down at some point. If we’re using them as points of reference or filtering thought processes through them we need to recognize the limitations. This applies even to the metaphors we use for the creation of software.

Writing

One of the oldest metaphors for writing software is that it is, well, writing. In a way, this evokes of images of a developer sitting down at a desk with a pen or pencil in hand to turn out code in calligraphic strokes on heavy bond paper. While that image isn’t entirely accurate, it does have its uses. While in college, I frequently wrote functions or entire source files on paper when I didn’t have access to a computer. This forced me to think through the logic more than I would with a keyboard in front of me. While it was a slower process, I frequently found that the code I wrote on paper first was less prone to have errors because of the mental walkthroughs I had to do. Obviously, this is not the most efficient way to create software. One thing the writing metaphor did promote was code readability. It also carried notions of programming style and an emphasis on originality. This metaphor is likely the reason that the individual writing a piece of software is called the ‘author’ rather than the ‘developer’ or the ‘programmer’.

Construction

Viewing development as software ‘construction’ is one such metaphor that is very compelling and effective. So compelling and effective that it has been the most enduring metaphor for development. Steve McConnell strongly supports this metaphor in his book Code Complete. Scaffolding, tools, design, plumbing, foundation, all these terms have meaning in both construction and development (astute readers may also note that architects and those implementing the designs may have disagreements in both fields as well). These same conceptual alignments occur when you use the term ‘software engineering’ (also a metaphor). While these metaphors are powerful, they ignore a couple of important points.

  • Software is not the same kind of thing as a building. Buildings cannot be easily changed except in superficial ways, but there is the perception that software is highly malleable.
  • Similarly, software isn’t a physical entity. It is an abstract and highly complex logical structure. The conceptual effort required for complex software is much higher than that required for construction.

Organic – Growing metaphor

With the rise of agile development, a new metaphor has increased in popularity. Software is ‘grown’ and development is a more organic process. Given the iterative nature of Agile practices, this metaphor is appropriate. Working software starts small and grows in size, functionality and aesthetics as a project progresses. While it is an effective metaphor for how development is done, the extent to which it can be leveraged is limited. We don’t ‘weed’ code bases, ‘harvest’ unit tests or ‘trim’, ‘prune’ or ‘plant’ much of anything.

Craft

Another popular metaphor identifies the creation of software as a ‘craft’. Historically, craftsmanship has been viewed as a creative endeavor that blends utility and art in the hands of a skilled practitioner. The term implies a level of quality and value that is difficult to find in a world of mass production, assembly lines and throw-away goods. Craftsmanship implies the use of good tools, a certain amount of artistry, and pride in the finished work. It also carries with it the idea of cyclical mentoring – the master helps the apprentice become a journeyman who will one day become a master that will take on an apprentice, and so on.

Storytelling

The final metaphor I’ll mention is that software development is storytelling. I admit, this is my own personal metaphor, but it works for me and that’s what matters. Storytelling is itself a craft, one of the oldest there is since it has been a part of human culture since the very beginning. It is a craft that takes place primarily in the mind and imagination of the writer, the telling or writing is simply a storage or transfer mechanism. It does overlap with the writing metaphor, but storytelling is very different from just writing. Good stories are built over time. They start with an idea and are gradually built into coherent and connected structures that depend on each other to work. Parts may be changed or rewritten if they don’t work well enough with the rest of the story. Stories and their components should be beautifully written or told. Many stories use common character types or devices (archetypes) that are very similar to Patterns. Good stories give something to the people to whom the story is being told. They provide entertainment, new ideas, or new ways of looking at the world. Good stories are powerful works of art that have the potential to change the way people think and operate.

Many other metaphors have been used for software development – it is science, art, herding cats, nailing jello to a tree, a bazaar, a cathedral, slaying werewolves, a process, a waterfall or a factory. The term ‘development’ is itself a metaphor. When we do software development, it isn’t the same kind of ‘development’ that kids go through as they grow (though I’m sure we’ve all seen code that misbehaves like a rebelling teen-ager). It isn’t breaking news or what we do to land when we build shopping centers or subdivisions. Correlations between the ideas can be seen with all the metaphors I’ve mentioned, but the depth to which that correlation can be pushed is variable. Taking any metaphor ‘too far’ has negative value. Similarities may exist but we must remember that they are fundamentally different types of things. The creation of software is precisely equal to nothing except itself.

Computer pioneer Edsger Dijkstra pointed out that programmers have to work with nine orders of magnitude between the lowest level of detail and the highest. One mechanism to cope with this level of complexity is to view the creative process as a metaphor and identify the correspondence between the two ideas. Metaphors are mental models and those mental models are useful tools. As with all tools, they are useful so long as they are used properly, provide value and don’t break. Ill-chosen metaphors, metaphors that are extended past their limit of utility, or those that are of very limited applicability are like choosing the wrong tool to accomplish a task. The point is to find the right metaphor for the job.

[1] Neil Stephenson has a great essay called ‘In the Beginning was the Command Line’ that uses the term ‘metaphor shear’.  You can download it here: [http://www.cryptonomicon.com/beginning.html]

Posted in Development | Comments closed