Skip to content

Software Tool Evaluation Factor 1: Functionality

Functionality

In my first blog entry I was talking about the TOP 10 Factors when evaluating software tools. Today I want to cover the factor "Functionality". For many evaluators it is the most important factor and for many others it is the only factor to take into consideration during an evaluation.

My experience tells me, that "Functionality" is important for every evaluation, but only one of many different factors you should take in consideration.

Why this? Let´s go back and ask yourself some basic questions:

Why are you evaluating software tools?

  1. We never used a specific tool for ABC, but we´re expanding and we need the functionalities of a tool to handle ABC
  2. We are using the tool XYZ and want to replace this tool as it does not have the functionalities we expect
  3. We are using tool XYZ but this tool is too expensive regarding maintenance, so we want to replace it by a cheaper tool with better functionality
  4. We are using tool XYZ but this tool is very complex and needs daily administration, so we want to replace it with a tool with less administration overhead
  5. We want to have more functionalities and use a more up-to-date tool, as the tool XYZ we´re using is outdated and upgrading to the latest version too expensive
  6. We want to streamline the tool chain and replace several tools with one having at least the same functionalities as now
  7. We want to check, whether we´re still using the right tool or whether there are better tools on the market with better and improved functionality

If your answer is one or more of these topics, you´re not on the right page.

STOP THE EVALUATION

Stop the evaluation here as it will be based on functionality only. Find out the real reason behind the evaluation. Ask yourself or your management the following question:

What is the reason behind the tool evaluation?

  1. Increase velocity
  2. Improve our product quality
  3. Improve user experience
  4. Meet regulatory prerequisites
  5. Be ready for expansion
  6. Have a better ROI
  7. Save costs

Now you should get more than one of these 7 topics as an answer and you can start building your criteria around these topics, create the functionality list and the key points based on the second answer.

  1. Process support
    • Support for our current process and workflow?
    • Support for our legacy processes?
  2. Methodology support
    • Support for our development methodology (Agile? /Waterfall? etc.)
  3. Traceability
    • Can we trace our data through all stages?
  4. Templates
    • Is an out-of-the-box template available for our industry?
    • Can we use the templates as a starting point?
  5. Existing Ecosystem reuse
    • Can we import our legacy and current data?
    • Can we convert this data from the existing system?
    • Can we export our data for usage in our other existing tools?
  6. Level of Customization
    • Can we customize the tool for our specific usage?
    • We need to do XYZ, is that possible?

These are a few examples, not a complete list. A complete list must be much more detailed and much more specific.

STOP THE EVALUATION HERE AGAIN

These are questions about functionalities, but do you really need all these functionalities? To better find out, you should not ask for specific functionalities but create requirements, use cases or usage scenarios in your specific context.

Examples

  1. As a user I want to store all our product issues reported by customers
  2. As a user I need to reuse all existing data stored in a relational database (attach excerpt)
  3. As a user I must be able to create reports in existing format (attach report)
  4. As a user I must be able to trace from an issue back to the feature request

These are again only a few examples, not a complete list. A complete list must be much more detailed and much more specific.

Put your requirements in a checklist (which you later want to send out to tool vendors), it does not make sense to ask whether it is possible or not. Better ask for screenshots or a short video showing your data in the context of the specific tool. Why? All vendors will answer yes, it is possible as otherwise they will lose you as a possible customer without talking to you and making you an offer.

Don´t prepare a checklist with 100 functionalities, better prepare a requirements catalog with the 20-30 most important requirements and ask for close explanation how these requirements are handled in the tool.

What you should consider as well is the fact how important are these requirements. Before starting a checklist, you should internally qualify your requirements in categories.

Categories for Evaluation Requirements

When qualifying your software tool requirements into categories, you should set a score for every requirement as well. When the requirement is fullfilled you can add this score to your overall result score.

Category Score
Show Stopper / Must have 5
Important 3
Nice-to-have 1

Why only these three categories? Because it makes the categorization easier and you avoid categories like "not so important", "medium", etc. You don´t need this. Either you must have a functionality to achieve the goals or the functionality is important but not mandatory to achieve the goals or it is a nice-to-have, which means, we would appreciate such a functionality but we can achieve all goals without fullfilling this requirement.