Skip to content

5 Stages of Software Tool Evaluation

5 Stages

Today every company needs software products and tools to be competitive. Using paper and pencil won´t bring you advantages in today´s digital world. But how to find the right product, the right tool? Sometimes the management simply decide: My friend’s company uses XYZ, so let´s use this tool as well. Other decide by looking at competition: Our competitor uses XYZ, let´s use it as well. Another possible solution is: Let´s use the market leader. These are examples, where no evaluation is needed and you can stop reading.

But in that case, you will miss the best try: Find the best solution by evaluation. It´s a fact, the evaluation and selection of appropriate tools is essential to the success of the company.

Before you even start thinking about a tool evaluation, you should think about the goal, your budget and how to proceed. What´s next, when to develop the criteria, when and how to decide? Evaluation is a complex process where different people from different departments are involved and you need to plan the evaluation in different steps. Let´s name the steps: It´s a process with different stages.

You can divide the process in 5 stages:

5 Stages

STAGE 1: Plan the evaluation

  • Feasibility
  • Goal
  • Identify stakeholders
  • Find an effective team, who is involved
    • End users
    • Technical and domain experts
    • Stakeholders
  • Gather management commitment
  • Do the evaluation team have decision authority?
  • Specify the criteria
  • How do we measure?
  • Budget
  • Timing for all stages
  • Create a charter
  • Possible results:
    • Stay with the old tool
    • Make your own tool
    • Start evaluation

STAGE 2: Perform a market analysis

  • Information gathering
  • Depending on Stakeholders and knowledge of evaluation team
  • How to collect information
    • Google Search
    • Use Forrester Wave or Gartner Quadrant to identify the candidates
    • Trust the documentation
    • Reviews in internet
    • Independent analysis
  • Define requirements incl. priority and weighting
  • Result:
    • "Long List" with 4-7 tools

STAGE 3: Analyze the tools

  • Invite vendors on the list for a presentation and live demo
  • Finalize your requirements
  • Send your requirements as a check list to all vendors on the "Long List"
  • Collect the answers and make it comparable
  • Analyze the results
  • Perform a gap analysis
  • Use hands-on techniques
  • When necessary take product probe
  • Refine the requirements
  • Detect unrealistic expectations
  • Make and document recommendations
  • Gather data to improve future evaluations
  • Result:
    • "Short List" (2-3 tools)

STAGE 4: Pilot Project

  • Define a pilot project with real world scenarios
  • Have your real-world scenarios prepared
  • Create infrastructure and use a prototype of your system
  • How should be the pilot conducted? In a lab or at work place
  • Who should be involved?
  • Short list should not be more than 3 tools, I always suggest 2 tools
  • deeply evaluate the criteria based on real usage and not theoretical answers
  • Install the tools in question in parallel
  • Result: Decision

STAGE 5: Go Live

  • Define the first project and use the tool there in production for 3 months
  • Configure product to fit the needs of the project which goes in production with the new evaluated tool
  • Integrate the tool in your whole ecosystem
  • Result:
    • Rethink the decision
    • Lessons learned