There are many project management tools designed to support Agile methodologies such as Scrum and eXtreme Programming (XP), including several open source options. This article compares the most compelling open source options on the following criteria:

• Feature set
• Usability
• Viability
• Suitability for large projects and products

Background

Agile development methods, including Scrum, XP, Crystal, FDD, and DSDM, have gained widespread adoption and acceptance in recent years. IBM’s Rational Unified Process (RUP) has been tailored to create the Agile Unified Process (AUP). The agile approach has gained traction even in large organizations with complex projects and geographically distributed teams, including IBM. While the Agile Manifesto favors “individuals and interactions over process and tools”, a well-crafted tool for managing agile projects has clear advantages, and is essential as agile teams and organizations grow to geographically distributed locations and enterprise scale.

Agile software and open source have enjoyed a tight relationship through the years, with several open source projects being inspired by and integral to the agile movement.

• Unit testing: XUnit (JUnit, NUnit, etc.), various Mock Objects frameworks
• Acceptance testing: Fit and FitNesse, HttpUnit, etc.
• Integration tools: Cruise Control, Hudson
• Behavior driven development tools…

The market for agile project management tools is now mature and saturated, with dozens of offerings from both small and large vendors, and more products hitting the market regularly. Market leading commercial offerings include Rally, VersionOne, Thoughtworks Mingle, and Danube ScrumWorks. A few open source tools have been around for some time, and several have emerged more recently. Compared to the open source offerings, the leading commercial tools offer better features for large organizations and large scale products and projects. They also tend to offer more integrations with third-party applications. Those extras come at a price, of course.

Contenders

This article compares only open source agile project management tools, and (with one exception) only considers projects under active development that have demonstrated significant usage (measured by a combination of download volume, discussion activity, and volume of internet references to the tool).

• Agilefant
• IceScrum
• Agilo
• eXPlainPMT
• XPlanner*

*It’s true that XPlanner is no longer actively developed, but it gained widespread usage years ago, with over 45,000 downloads of the latest version (0.7b) alone. We included it in this comparison because it provides a good baseline to determine how the tools have matured in the past few years.

Some open tools that didn’t quite make the cut due to lack of apparent usage include Agilito, Scrinch, and FireScrum.

Comparison Summary

This section provides a high level comparison of the five contenders. A deeper discussion of each tool follows this section.

Legend:

√ Feature is included

X Feature is not included

√¹ Superscript number: see notes below the table for more information

*      1 star rating: poor

**    2 star rating: acceptable

***  3 star rating: good

***  4 star rating: excellent

Notes:

1. Agile allows only pseudo-Fibonacci numbers for story points.
2. IceScrum’s iteration burndown chart is flawed; the horizontal axis shows only days elapsed so far in the sprint rather than full time range of the sprint.
3. Agilo’s iteration burndown chart is flawed in the online demo version; the vertical axis scale is off.
4. eXplainPMT has a burndown chart for the whole project, not for each iteration.
5. Agilo’s backlog hierarchy is not easy to use; it requires associating items by item number.
6. eXplainPMT has “initiatives” that are similar to epics, but they are only an attribute of a story.
7. Agile uses the term “milestones” in place of “releases”.
8. Agilo’s roadmap feature shows all sprints and a lot of scrolling is required to see all releases.
9. Agilo has a separate bug backlog.

Agilefant

URL: http://www.agilefant.org/

Version reviewed: 1.6.2  [Note: Version 2.0 is promised “soon”.]

Technology: Tomcat 5.5, MySQL, Java 1.6

License: MIT

Concepts

Products are the highest level construct, and each deployment may have multiple products. Each product may have one or more projects, which are essentially releases. Each project may have one or more iterations. Each product, project (think “release”), and iteration has its own backlog, which contains stories. Stories can be moved to any other backlog, for example from the product backlog to an iteration backlog. Stories may consist of zero or more tasks. Projects can be prioritized in the Portfolio view.

The tool supports multiple concurrent iterations, which allows larger organizations to use the tool effectively.

Agilefant supports multiple users and users can be grouped into teams. However,  it does not support any user roles; a user is a user, with no differences in permissions or access to features. Teams can be assigned to specific iterations.

Version 1.6.2 does not include the concept of higher-level features or epics, although this feature is promised in version 2.0 which is supposed to be released soon. It does support the concept of themes, which are an attribute of stories, and this does provide a simplistic way to associate a group of stories.

Viability, Support & Documentation

The product documentation is adequate but not excellent. The product is intuitive enough that not much user documentation is necessary. The installation documentation is sufficient. Forums and email support appear to be fairly good. Development is active with version 2.0 supposed to be released soon, although no particular date for the release has been published.

Usability

Agilefant is intuitive. We were able to figure out how to use all features without relying on documentation. It lacks “drag and drop” features, but it otherwise makes good use of client-side features that make the user interface more responsive.

Strengths

• Rich feature set.
• Suitable for larger organizations and projects, except for the lack of “epics” or story hierarchies. (Version 2.0 should fix that shortcoming.)
• Reasonably intuitive and easy to use.
• Portfolio planning feature.
• Timesheet feature.

Weaknesses

• Stories can be estimated in hours only, not points.
• Stories cannot be ranked absolutely; they can only be prioritized on a scale of 1 to 5.
• No “drag and drop” re-ordering of stories.
• No “epics” or hierarchy of stories. Note that a story hierarchy is promised for version 2.0.
• No “task board” or “whiteboard” view.
• No differentiation between user roles.

Overall Rating

Agilefant is a very capable tool with a rich feature set and a few weaknesses. It is better suited for large projects and large organizations than any of the other tools reviewed in this article, but the lack of a hierarchy of story/requirement levels (a.k.a. “epics”) is a significant drawback for large projects. The lack of epics or a hierarchy of stories, however, is a big weakness to overcome for large scale projects.

Screenshots

Above: Tree-style navigation in Agilefant.

Above: Objects that a user can create in Agilefant.

Above: The Iteration Details view in Agilefant.

Above: The list of stories and tasks within an iteration in Agilefant.

Above: The Iteration Burndown chart in Agilefant.

Above: The Daily Work view in Agilefant.

Above: The Product view in Agilefant.

Above: Agilefant’s Development Porfolio view allows projects (releases) to be ranked.

Above: Agilefant’s Timesheets feature provides a report of how users have spent their time.

IceScrum

URL: http://www.icescrum.org/

Version reviewed: 2#13

Technology: Java 1.5 with servlet engine (e.g. Tomcat). Bundled with HSQLDB, but can be configured to work with other DBMs such as MySQL.

License: GPL

Concepts

Products (also called projects in some places) are the highest level construct, and each deployment may have multiple products. Each product has a single backlog and a roadmap. A backlog contains features (similar to epics), user stories, defects and technical stories. A roadmap contains multiple releases, each of which has a single release plan. A release plan consists of multiple sprints. Each sprint contains stories, which in turn contain tasks and acceptance tests. Impediments can be tracked for each product.

Unlike Agilefant, IceScrum includes a task board/whiteboard view of the iteration, allowing tasks to be dragged and dropped. It also allows drag and drop ranking of user stories in the backlog. Many options are available only from a right-click context menu, which isn’t obvious at first but is easy to use once you notice it.

IceScrum is the only product reviewed that has a planning poker feature. This feature allows a distributed team to play planning poker for estimating features and user stories.

IceScrum allows only a single release and single sprint to be active at one time (for a particular product), making it unsuitable for larger organizations that need multiple concurrent sprints with multiple teams running in parallel for a single product.

IceScrum users may have any of the Scrum roles (Product Owner, Scrum Master, Team Member, Stakeholder) plus custom roles may be created. It does not allow users to be grouped into teams.

Beyond a burndown chart, IceScrum does not have any reports or reporting API.

Viability, Support & Documentation

The product documentation is adequate but not excellent. The installation guide is written only in French. Forums and email support appear to be fairly good. Development is active.

Usability

Some features are available only from right-click context menus. This is fine once you discover it, but it’s not obvious to new users. It has a relatively rich user interface with drag and drop capability in several places.

Strengths

• Rich feature set.
• The Sprint Backlog view resembles a physical task board quite effectively.
• The Roadmap, Release Plan and Sprint Plan views support multiple levels of planning.
• Acceptance tests can be recorded for each story. A BDD-inspired template for acceptance tests is available.
• Includes a planning poker feature.
• Supports absolute ranking of stories by drag and drop, and estimates in story points.

Weaknesses

• The story card layout would make it difficult to rank a large product backlog.
• Some features are not intuitive. Drag and drop works in some places but not in others, even though the mouse cursor make it seem that drag and drop should work. The right-click context menu is not obvious, but it’s easy to use once you discover it.
• IceScrum is not suitable for large projects with multiple teams working on a single product; only a single release and single sprint can be active at one time.

Overall Rating

IceScrum is a very capable tool with a rich feature set and a few weaknesses. Although it supports multiple products (projects), it is suitable only for small projects with a single team working on one sprint at a time per product.

Screenshots

Above: IceScrum’s Feature view. Features are high-level requirements akin to epics.

Above: Creating a user story in IceScrum’s product backlog.

Above: IceScrum’s Product Backlog view. Each story card can be dragged and dropped to change ranking. Features (epics) are associated with a story by dragging the feature from the left side of the page onto the desired story.

Above: Right-clicking on any story card gives a context menu.

Above: IceScrum’s Sprint Backlog view. To add a story from the product backlog to the sprint, drag the story from the left side of the page and drop it in the story area of the sprint.

Above: IceScrum’s Roadmap view shows the timeline of releases and sprints within the release.

Above: IceScrum’s Release Plan view. Product backlog items can be dragged and dropped onto any sprint.

Above: IceScrum’s Sprint Burndown view. The burndown is flawed, showing only the actual days elapsed so far on the horizontal axis rather than the full time range of the sprint. In the screen shot above, the sprint is 14 days, but the burndown shows only the first 2 days.

Agilo

URL: http://www.agile42.com/cms/pages/agilo/

Version reviewed: 1.0.2 Pro (demo instance at https://agilo.agile42.com/login with login: demo/demo)

Technology: Python (2.4, 2.5, or 2.6), SQLite (also PostgreSQL or MySQL). Built on Trac 0.11 (open source ticketing system built on Python).

License: Apache License 2.0

Concepts

Each deployment can have multiple milestones (releases). The backlog contains requirements, user stories and tasks. These three item types support a hierarchy by reference: requirements can reference stories, and stories can reference tasks. This hierarchy is difficult to establish and utilize, however. Agilo has a separate backlog for defects, which makes it difficult to prioritize defects relative to user stories. It also has a separate impediment backlog.

Users can be given the roles of Scrum Master, Product Owner, or Team Member. Users may also be grouped into teams.

Agilo includes an excellent and intuitive task board/whiteboard view of the iteration, allowing tasks to be dragged and dropped. This is by far the best feature of the tool; other features are less untuitive. Perhaps Trac users would find it more friendly?

Agilo has a simple dashboard with a sprint burndown chart, a graph of stories and tasks (total, planned, and closed), and allocation of tasks to team members by percent. In addition, it allows custom queries and reports to be created and saved.

Viability, Support & Documentation

The product documentation is adequate but not excellent. The forums are very active and seem to be a good place for support. The Pro version includes professional support for about €8.50 per month.

Usability

I found some of Agilo’s features to be unintuitive and difficult to use, requiring many clicks and pages to accomplish common tasks. The task board/whiteboard feature, however, is excellent and very easy to use.

Strengths

• Excellent task board/whiteboard iteration view with drag and drop functionality.
• Supports absolute ranking of stories by drag and drop, and estimates in story points.
• Good reporting features, including user-customized reports that can be saved.

Weaknesses

• Some features are not intuitive. It feels like Trac didn’t support agile concepts very well so they had to be force-fit into Trac’s model.
• When creating a task, it can be associated with a sprint, but not with a story.
• Many operations require lots of clicks to complete.

Overall Rating

Although the task board/whiteboard view is intuitive and excellent, other features are less intuitive and often not very easy to use. The sprint burndown in the demo instance was flawed, showing 1200 hours when only 37 hours of tasks were planned.

Screenshots

Above: Agilo’s Product Backlog view.

Above: Agilo’s Sprint Backlog view. This view is confusing because tasks don’t appear underneath their stories. In the example above, 2174 and 2176 are tasks for story 2173. They can be dragged and dropped to an intuitive position, although in some views I was unable to do even that.

Above: Agilo’s Sprint Burndown. This chart was flawed, showing 1200 hours on the vertical axis when only 37 hours of work were planned.

Above: Agilo’s Whiteboard view. This is one of the best features in the tool. It’s quite intuitive, with drag and drop capability. Add a new task to a story by clicking the “+” symbol. Tasks can be edited inline by right-clicking.

Above: Agilo’s Roadmap view. This view shows all milestones (releases) plus a summary of each sprint in each milestone. With multiple sprints per milestone, a lot of scrolling is required to see the big picture, which makes this feature rather ineffective.

Above: Agilo’s Bug Backlog. Bugs are tracked in their own separate backlog, which is a drawback since they can’t be prioritized relative to other product backlog items such as user stories.

Above: Associating requirements (epics) with user stories and stories with tasks in Agilo. This feature is not easy to use since stories must be separately looked up by number, then individually entered on this page.

Above: An Agilo user story showing references to a requirement (a.k.a. epic) and a task.

Above: Agilo’s View Tickets page, which lists built-in and user-defined reports.

Above: A sample report from Agilo  – active tickets by milestone.

eXplainPMT

URL: http://github.com/explainpmt/explainpmt/tree/master

Version reviewed: No version specified at demo site: http://github.com/explainpmt/explainpmt/tree/master (login: demo/demo)

Technology: Ruby, RDBMS (SQLite, PostgreSQL or MySQL)

License: GPL

Concepts

The highest level construct is the project, and eXplainPMT supports multiple projects. Each project has one or more releases, and iterations. Note that iterations are associated with projects, not with releases. A project has one backlog, and a backlog contains stories. Stories have tasks and acceptance tests, and can be assigned to a release. Stories may also be associated with an initiative, which is somewhat like an epic, except initiatives cannot be ranked or prioritized. Surprisingly, tasks do not have estimated or actual hours; they have only two states: incomplete or complete.

Users in eXplainPMT do not have specific roles; all users are the same. Users may be grouped into teams.

Viability, Support & Documentation

eXplainPMT has no user guide and only a brief README file for an installation guide. It has no forums, nor any email support. It is unclear whether development is still active; the last update was November 2008.

Usability

The user interface is straightforward and intuitive, but some very basic features are lacking. For example, after creating a story in the backlog, it cannot be moved into an iteration. In addition, eXplainPMT does not support drag and drop nor the richer experience of modern browser applications.

Strengths

• Intuitive interface.
• Supports the definition of acceptance tests for stories.

Weaknesses

• Cannot estimate or track task hours!
• No task board/whiteboard view.
• No iteration burndown chart.
• No releases or roadmap.
• Development status is unclear; the last update was November 2008.
• Support and documentation are almost non-existent.
• Not suitable for large projects since it allows only one iteration to be active at a time.

Overall Rating

eXplainPMT is missing many important features, the development status is uncertain, and support and documentation are almost non-existent. It does not support large teams. Agilefant, IceScrum, Agilo and XPlanner are all likely to be better tools for most agile teams.

Screenshots

Above: eXplainPMT’s Projects view.

Above: eXplainPMT’s Releases view.

Above: eXplainPMT’s Iteration view.

Above: eXplainPMT’s Backlog.

Above: eXplainPMT’s Initiative view. Initiatives are logical groupings of stories.

Above: eXplainPMT’s Story view.

Above: eXplainPMT’s Project Burndown. The burndown, measured in points, is for the whole project only, not for iterations.

XPlanner

XPlanner is included in this comparison because at one time it had a large user base and was the primary tool in this category. Although the project is dormant now, it serves as a good baseline for seeing how tools have changed in the past few years.

URL: http://www.xplanner.org/

Version reviewed: 0.7b (beta)

Technology: Java 1.5, Tomcat 5.0 (not 5.5), MySQL

License: LGPL

Concepts

The highest level construct is the project, and XPlanner supports multiple projects. It does not include releases. Each project has one or more iterations. It does not include a product backlog, but a workaround is to create a special iteration to use as a product backlog. Iterations contain stories, and stories contain tasks. Tasks may be of type feature, debt, defect, ftest (functional test), atest (acceptance test), or overhead. Stories are estimated in hours, and tasks are estimated and tracked in hours.

Users in XPlanner can be assigned four permission levels: Viewer (read only), Editor, Admin., or Super Admin. XPlanner does not have a team construct for grouping users.

Viability, Support & Documentation

The last update to XPlanner was in May 2006, so it’s safe to say the project is dormant. The project does have forums for support that contain lots of useful help but they are inactive now; the forum averages two new topics per month with a few replies each. The installation documentation is sufficient although it’s only a README file in the latest version 0.7b. User documentation is minimal, but the tool is intuitive enough that little documentation is necessary.

Usability

XPlanner’s UI is straightforward and intuitive. However, it lacks most of the client-side (AJAX-based) usability features of most modern browser applications.

Strengths

• Intuitive interface.
• Rich built-in reports and charts.
• SOAP API for integration.
• Customizable notation for simple URL linking with external systems such as defect tracking.

Weaknesses

• No releases or roadmap.
• No task board/whiteboard view.
• No story points.
• No “epics” or hierarchy of stories/features/requirements.
• The project is dormant, with the last update in May 2006.
• Support is very limited since the project is dormant.

Overall Rating

Despite its shortcomings, XPlanner is still a good tool for small teams. Since the project is dormant and other tools have better features now, we recommend Agilefant, IceScrum or Agilo over XPlanner, but recommend XPlanner over eXPlainPMT for most teams.

Screenshots

Above: Defining a story in XPlanner.

Above: XPlanner’s Iteration view. Stories are ranked by manually renumbering them. You can move a story to a different iteration with the 3^rd action icon.

Above: XPlanner’s Story view

Above: Recording time spent on a task in XPlanner.

Above: XPlanner’s Iteration Progress chart, showing estimated and actual hours.

Above: XPlanner’s Iteration Burndown, showing hours remaining in the iteration.

Above: A variety of built-in charts in XPlanner’s chart view.

Above: XPlanner’s Metrics view.

Above: XPlanner’s accuracy metrics, showing the accuracy of estimates.

Conclusion

Overall I give high marks to Agilefant, IceScrum and Agilo. XPlanner has significant drawbacks but is a very capable tools the feature sets meet your needs. eXPlainPMT is missing too many key features and its development status is uncertain, so I don’t recommend it. However, each organization should consider its own situation and feature needs before choosing a tool. Particular features may be very important for one organization but not for another.