Solution Assessment & Validation: A Complete Guide Tutorial For FREE | CHECK-OUT

Last updated on 08th Jul 2020, Blog, Tutorials

Solution Assessment and Validation is the procss of ensuring that the solution built matches the requirements.

Solution Assessment

Once requirements are handed to the solutions team the business analyst will be expected to assess the design returned to the project team. The Business analyst maintains much of the project’s intelectual property – especially about requirements and is the most able person to assess the apropiateness of the solution design in terms of alignment and quality. (Others such as the sponsor and project manager may have more ability to asess value for money.)

Often the Requirements Traceability Matrix and Requirements specifications documents are used as tools to assess the solutions comprehensiveness.

Solution Validation

Solution validation is the activities of explaining the solution’s appropriateness to stakeholders and sponsor. This often involves explaining technical concepts to lay-people. Often a business analyst will be required to get a document approved by one or many people.

Solution validation also involves inspecting or managing the test plans and activities. Often documents are created to govern the testing process including a test strategy and test plan.

At each of these quality checks the BA should be in a position to highlight problems or acknowledge that the solution has been built to the appropriate standards. The V-Model expands upon the testing activities of the SDLC further.

This article considers the BABOK Knowledge Area, Solution Assessment and Validation, about which there is much confusion and inaccurate information on the Internet.  I have been intrigued by this topic for some time and truly believe it has great potential to deliver significant business value.

According to BABOK, the Solution Assessment and Validation Knowledge Area describes tasks performed to ensure that solutions meet business needs and facilitates successful solution implementation. These activities are not just limited to application software, but address a variety of other areas such as business processes, organizational change, outsourcing agreements, and any other component of the solution. The knowledge area consist of six tasks, which are:

• Assess Proposed Solution
• Allocate Requirements
• Assess Organizational Readiness
• Define Transition Requirements
• Validate Solution
• Evaluate Solution Performance

Let’s examine three of these areas where significant confusion exists:

• Assess Proposed Solution,
• Validate Solution, and
• Evaluate Solution Performance.

The confusion among these areas is amazing. I read one article that described Evaluate Solution Performance as “performing various tests to confirm that the solution meets performance related goals such as response time, availability, and scalability.”  Wow—maybe it is time for IIBA to change the name of this task. Other articles seem to think these goals are all the same.

Assess Proposed Solution (Pre Solution Acquisition)

According to the BABOK, the purpose of Assess Proposed Solution is to “assess proposed solutions in order to determine how closely they meet stakeholder and solution requirements.” 

After requirements have been defined and prioritized, the proposed solution with its set of requirements is analyzed to determine whether the solution will deliver sufficient business value to justify its implementation. This can be viewed as kind of a checkpoint or gate to make a go/no go decision before the solution is acquired or built.  Solution assessment may be performed on a single solution or be used to compare multiple proposed solutions to recommend the best course of action to proceed.

The output of this task is the Assessment of Proposed Solution and its purpose is to assess the value delivered by each proposed solution. If multiple options are available, a recommendation of the best solution should be made. A recommendation to terminate the initiative may be given if no solution delivers enough value to justify being implemented.

Validate Solution (Solution Implementation)

According to BABOK, the purpose of Validate Solution is to  “validate that a solution meets the business need and determine the most appropriate response to identified defects.”

Solution validation is required to ensure that the project and its outputs deliver a proposed solution that meets defined requirements.  Problems identified through solution validation are reported and prioritized for resolution. When a problem is identified with the solution such as a failure to meet a stakeholder need or requirement that was not correctly specified, the business analyst helps the team determine the most appropriate action to resolve the problem.

Defects are generally identified through participation in lifecycle events. Below are some examples of lifecycle events for waterfall projects:

• Design Review
• Code Review and Demonstration
• System and Integration Test
• User Acceptance Test
• Organizational Change Plan
• Solution Deployment Plan

The process also applies to agile projects, although the lifecycle events may differ.  Examples of lifecycle events for agile projects include:

• Product Backlog Prioritization
• User Story Sizing and Estimating
• Sprint Plan
• Demonstration
• Sprint Retrospective
• Release Plan

Evaluate Solution Performance (Post Implementation)

According to BABOK, the purpose of Evaluate Solution Performance is to “Evaluate functioning solutions to understand the value they deliver and identify opportunities for improvement.”

Solution performance evaluation involves analyzing how a solution is functioning after it has been deployed.   Assessments are performed to determine if the solution is achieving the expected benefits in the business case.  Solutions are adapted and modified by end users, often using workarounds, recording additional information in spreadsheets, or adopting informal policies and procedures to resolve problems.

Performance is evaluated based on various Key Performance Indicators (KPIs) that are generally documented in the business case. The goal is to assess whether the solution is achieving the expected benefits and, if not, to determine why it is not and then assess what can be done to fix the problem.  Fixing the problem may involve making additional system changes, making organizational changes, providing additional training and support, or simplifying the workflow. These assessments are generally performed every quarter.  The solution, along with the proposed optimizations, is reevaluated the next quarter to ensure the expected improvements were realized.

In our Business Analysis Methodology, the Requirements Excellence Framework,™ we refer to this task as Benefits Realization, as we have found this term to more widely applied than the term Evaluate System Performance used in BABOK. The diagram below shows how Benefits Realization works in our Framework.

According to various research studies, organizations that have implemented a good process for benefits realization management have been able to increase the return on investment by 3 times.  Benefits realization management is very important for all projects, but seems to be particularly important for acquisition and implementation purchased packages such as ERP or CRM systems.

Solution assessment and validation has several objectives:

• Evaluate organisational preparedness to the proposed change
• Evaluate performance of existing solutions to support enterprise analysis
• Assess the proposed solution prior to its selection
• Validate the designed solution to ensure that it meets the requirements
• Determine transition requirements for moving from the current state (“as is”) into the target state (“to be”).

Assessment of organisational readiness to changes plays a critical role in a project’s success. The results of the assessment determine the scope of the project in terms of actions to be taken to train personnel, modify business processes, change certain activities and reporting, not to mention changes to behaviour and beliefs. These results also support a definition of transition requirements which describe how the solution should be implemented and released into production with minimal disruption to the existing solutions and processes.

A business analyst performs assessment of the designed solution to ensure that the solution meets the acceptance and evaluation criteria and meets the approved requirements.

When the solution has been constructed, the business analyst validates the solution to ensure that defects are identified, their causes are known and the defects are communicated to the responsible stakeholders.

The business analyst carries out an evaluation of solution performance multiple times within the project. Firstly, an evaluation of performance of the existing solution helps identify weaknesses of the solution and supports the investigation of causes of poor performance. This information feeds into gap analysis conducted within enterprise analysis. Secondly, the business analyst evaluates performance of the solution once it’s ready for deployment in order to ensure that the solution delivers the expected value to the business.

In practice, the business analyst works closely with solution architects, vendors, software developers, testers, technical personnel and change managers, acting a bit like a conductor of a big orchestra to ensure the satisfaction of the listeners.

Click on the image below to download the visual map of the solution assessment and validation section of the BABOK:

Three Business Analysis Techniques for Solution Validation

If you find yourself performing solution validation as part of your project, remember that there are three general techniques that the BABOK® Guide  recommends that you use when validating that solution and addressing any defects or issues that you might find. Some of these techniques occur early in the project life cycle when you are defining what you will validate, while others are used closer to the end of the project life cycle when you are actually performing your validation efforts. Let’s step through each of those three techniques in a bit more detail.

Acceptance and Evaluation Criteria Definition. Early on in your project, you and the team need to make sure that you define the set of requirements that will be used to validate the resulting project’s solution. This set of requirements must have defined, measurable acceptance criteria for you to use in your validation efforts.  If you can’t measure these acceptance criteria, it will be difficult to prove that the solution is delivering what was promised.  I always tell my project teams to think about the end of the project when our key users and stakeholders are tired of us, and to put together measurable and objective acceptance criteria that we can use to prove the solution performs as defined.  That way, we are good to go whether our stakeholder love us or dislike us at the end of the project. 

Problem Tracking.  Never, ever forget that problem tracking is your formal vehicle for identifying and tracking identified defects in your solution. This technique ensures that the defects found during solution validation are addressed and resolved. It is funny how the problems, defects or issues that get written down, analyzed and have agreed-upon fixes actually get fixed, isn’t it? I find myself using this technique later in the project life cycle, but the mechanism for identifying, logging and tracking project problems, defects or issues should be available very early in the project life cycle, just in case.

Root Cause Analysis. Root cause analysis allows you to determine the underlying reason for a problem or defect in your solution. Identifying the real cause of the problem is a significant step in addressing and fixing that problem. This straightforward and effective business analysis technique allows you to correct the actual cause of the defect versus correcting a symptom of that defect and not the defect itself.  It can be all too easy to fix the symptom of a problem and miss the real problem that is causing your solution to behave badly.