Technology Scorecard for Application Portfolio Rationalization (Application Portfolio Management)

1.     Introduction

To understand Application Portfolio Rationalization (APR), we need to understand what is Application Portfolio Management (APM)? APM is akin to Financial Portfolio Management but it is applied to inventory of IT applications / systems present in an enterprise or segment of business within the enterprise.

“In many organizations, applications are deployed and then run forever leading to an endless proliferation of IT systems that is not sustainable. The practice of adding new software applications without retiring old ones will eventually cause the entire IT budget to be consumed by the maintenance and operations burden of legacy applications.” – Burton Group

To avoid the condition described above, Application Portfolio Management function is required. Application Portfolio Rationalization (APR) is a means of APM.  APR needs to be undertaken periodically for a sound APM. APM is performed in two ways as suggested by Wikipedia, “There are two main categories of Application Portfolio Management solutions, generally referred to as 'Top Down' and 'Bottom Up' approaches”.  The top down approach is when the application or system’s function, design etc is taken in to consideration to arrive at the rationalization decision; whereas bottom up is then, when the system / application’s source code is taken into consideration to see if the system is capable of fulfilling its functions.

This paper discusses a unique scoring technique designed to score the rationalization decision based on a Technical Evaluation Criteria for the System under Consideration (SuC). The technique proposed here is mostly based on the Top Down approach but few low level details such as number of lines of code (to assess the application size), main  modules & dependencies etc. have also been considered at times.

As the most important input - the scoring system proposed here requires a sound judgment on the functional significance of the systems under consideration. A business analyst will make that decision based purely on the business value that the system is trying to deliver and it’s TCO.

The business analyst team may use a system’s business value & TCO as a starting point to arrive at rationalization decisions and identify potential target states for the systems under consideration.

Once the tentative initial decisions are in place, the technical scoring model proposed here will suggest how easy or difficult or if at all it is feasible to meet the Rationalized Target State (RT State). The end result of this scoring technique can be captured in a chart. The chart will capture the feasibility of attaining the rationalized state based, i.e. Rationalization Feasibility Score on one axis and  Business value of the system on the other axis. The chart can be divided into four quadrants namely – Merge, Retain, Retire, Enhance.

The Rationalization Score indicates the relative effort, cost & risk involved in rationalizing the systems based on the target states defined for each system/application. Higher the score, the easier (relatively less effort, cost & risk are involved) it is to rationalize the systems and vice versa.

2.     Rationalized Target State & System

 Typically rationalization decision that a business analyst will make (based purely on functional analysis) can be one of the following:

1.        Retire

2.       Merge

3.        Retain

4.       Enhance

These four types of decisions can be treated as potential Rationalized Target States and will be referred throughout the paper as “RT States”.

Retire

The system under consideration is not in use and does not serve any relevant business function currently and unlikely to do so in future and therefore it is best to be retired. However there is another possibility, the system being retired is not serving any significant business purpose, but it still has some minor functionality that is active and will remain useful in foreseeable future.  This type of case can be handled by moving the useful business function that the system being retired carries out to another closely aligned system. If this is the case then the decision would fall under the “Merge” RT State and not “Retire”.

It is also very important to note that once it is decided to retire a system a very critical question needs to be answered – what will happen to the data? Of course, the data will be archived. While this is the case most of the times, there are occasions when a system retires, but data does not and it could still be used for reference purposes if not for transactions.

Merge

There are two or more systems that have some duplication/overlap in the functionalities that they fulfill or deliver. Obviously we need to keep one of the systems and retire the other(s) while merging the non-overlapping functions together into one of the systems. This would be the “Merge” state for rationalization.

There is a likelihood that not all of the functions of the systems being retired will be available on the system being retained, in this case the system being retained will need to be upgraded / enhanced to build the missing functions. It is also possible that the system being retained does not have sufficient capacity to accommodate additional traffic / transactions; in this case it will need to be upgraded for additional capacity.

Retain

 The system fulfills a relevant business function and it is likely to be useful in foreseeable future and no change is required either in the functionality or capacity. It is therefore best to “Retain” the system as is.

Enhance

The system only fulfills part of the function expected from it and the missing functions are not delivered by any other existing system. As such it should be “Enhance”d by building additional functionality to it or upgrading its capacity if it is unable to serve all of its clients without degradation in service levels. There could be another scenario where a system might be fulfilling the function expected from it but it can be a candidate to add some new function which is aligned with its present function and the system has spare capacity.

The four Rationalized Target States (RT States) described above are tentative and these only suggest the intent and not a final plan of action. However these actions can only be materialized if we have identified Target Systems (RT Systems) that will facilitate implementing the rationalization decision (i.e facilitate migration to RT State) e.g. in order to retire a system that is mostly unused except some minor function that is still relevant, we will need to identify a Target System that can accommodate the surviving functions. This target system should be functionally aligned with the system being retired so that we can move the surviving function from the system being retired. The target system would be referred to as Rationalization Target System (RT System).

The functional analysis team will not only identify the Rationalized Target States (RT States) but shall also identify the Rationalization Target System (RT System).

3.     Technical Evaluation Criteria

While these decisions are being made i.e. when the systems are being analyzed to assess their functional fitness by a team of business analysts; the technology team will undertake similar activity i.e. the technical analysis of the Systems under Consideration (SuC). The system that is a candidate for rationalization will be referred to as System under Consideration (SuC) henceforth in this paper.

The technology team will need to record the data about the technical aspects of the systems under consideration. The architectural aspects listed & discussed below should be considered for evaluation of the SuC from a technical point of view. This will be used towards assessing technical fitment of the system towards achieving its respective Rationalized Target state. 

  1.     Alignment with Enterprise / LoB Architecture vision
  2.     Non-functional Requirement Coverage
  3.     Stability (special case of 2)
  4.     Application Architecture
  5.     Data Architecture
  6.     Deployment Architecture
  7.     Availability of Documentation

Alignment with Enterprise / LoB Architecture Vision

Most enterprises have an architecture vision defined at the enterprise level or at least at the LoB level. The Architecture Governance Board or the Enterprise Architecture Group maintains & regulates these guidelines. Adherence to such vision has been given due weight in the overall scoring model. This category includes criteria such as approved Architecture & Integration styles, Technology Platforms and Products & Frameworks. These EA alignment parameters should be recorded in a table, such as one shown under “Alignment with Enterprise Architecture / Line of Business Vision” in Appendix.

Non-functional Requirement Coverage

The present non functional requirements that the system fulfills; should be recorded for assessing the fitment of the system for migration to the RT state and also to assess whether the SuC is able to fulfill the quality of service expected from it currently and to judge overall technical fit. The non-functional requirements coverage considered includes evaluation parameters such as performance (response times on various interfaces), availability, scalability, security, load, operation hours etc.  These should be recorded in a table as shown under “Non Functional Requirement Coverage” in Appendix.

Application Stability

Stability as a special case of NFR: Stability which is one of the non functional requirements was considered as a special case so as to be able to assign weight independently to it. Under Stability factors aspects such as recurring issues / tickets, open issues, redundant features and most desired features have been considered. It was felt that these parameters add a different dimension to the overall assessment and hence considered independently of rest of the systemic properties. Application stability evaluation parameters should be recorded as shown under “Application Stability” in Appendix.

Application Architecture

Some of the criteria evaluated under this category are same as in 3 (alignment with Enterprise or LoB Architecture vision) and some others at a more granular level such as number of lines of code (for size estimates), system interfaces, protocol & data exchange format on each interface etc. These should be recorded in a table as shown under “Application Architecture” in Appendix.

Data Architecture

Under this criteria data model related aspects such as entities, batch jobs, interfaces to the database, supported roles & users have been considered. These parameters should be recorded in a table in Appendix under “Data Architecture” in Appendix.

Deployment Architecture

The category considers deployment aspects such as the tiers in each layer, hosting locations, processor & speed, memory consumption & availability etc to arrive at a score on how well the system under consideration is suited to meet the expectation from the target state. These parameters should be recorded in a table as shown under “Deployment Architecture” in Appendix.

Availability of Documentation

Under this criteria the availability of documents such as requirement specification, design, user manual or production run book, data model schema etc. have been considered. The criteria under this category will be used to understand or assess the effort required in migrating to the target state effectively.  These parameters should be recorded in a table as shown under “Availability of Documentation” in Appendix.

4.     Scoring Technique

The technique involves placing the attributes of the SuC & RT System side by side and then comparing to see if there is a match. A complete match leads to a high score whereas no match to a low score. 

In some case such as for RT State – Merge, the RT System will be known, whereas for a RT State such as - Enhance the properties of the RT State will need to be derived from the requirements (technical) the target state is expected to fulfill / implement when ready.  

The systemic properties will be placed side by side in the table shown below so as to facilitate comparison and to derive the scores.

Category	System Property	System Property Values		Comparison Score 0 - Not Feasible 5 - Extensive effort 10 - Minimal  effort 15 - Readily Feasible	Attribute Weight 10 - High 5 - Medium 1 -Low	Weighted Score
		System under Consideration (SuC)	Rationalization Target System  (RT System)
Application Architecture	Enterprise / LoB Architecture Vision Alignment	Fully Aligned	Not Aligned	5 (Extensive Effort)	10	50
	Open / Closed System	Open	Hybrid	5	5	25
	Architecture Style	Layered	Pipers & Filters	0	5	0
	System Interfaces	D&B, Check Imaging	D&B Check Reader, NAVDB	5 (Extensive Effort)	10	50
	Technology	Java	Mainframe	5	10	50
	Interface Technology (Protocol & Data Exchange Format)
	Licenses & Validity Period
	Application Size
Stability (Special Case of NFR)	Recurring Issues (tickets)
	Unused or Redundant features
	Must Have Features
Non Functional Requirements  Coverage	Availability / MTBF
	User count
	Simultaneous Users
	Load (No. of requests / transactions)
	Operation interval (peak and offpeak)
	Response times (each interface)
	Security (User / Role repository, SSO, Acess Manager etc)
Deployment Architecture	Hosting
	Tiers & Clustering
	Shared or Exclusive Hosts / Nodes
	Processor count & speed
	Memory Consumption & Availability (RAM)
	Network Speed (MBPS)
	Storage (Size)
Information Architecture	Schema documentation availability
	batch job count
	Interfaces
	Archival / Purging / Retention policy
	Audit Logging policy
	DR Strategy
	Back up
	Supported Character set
	Supported users & roles
Documentation	Requirement Specifications / Use Cases
	Architecture & Vision document
	Design (HLD & LLD)
	User / Support Manual
	ER model / Data Dictionary / Schema Documentation
			Total Score:			1865

Deciding the weight for each systemic property requires judgment based on the priorities defined for the enterprise. The priorities will generally be decided based on factors such as existing investment in software licenses, infrastructure, architecture vision and guidelines, future plans etc. However once the priorities and also the properties of the system under consideration and its target are in place, the rest of the technique is straightforward, so straightforward that it can be automated.

Once we have the score worked out for each set of systems, the scores can be grouped in to gradient such as “Very High”, “High”, “Moderate”, “Low” & “Very Low” based on the average score range or percentile.  The systems with scores High & Very High would be easy candidates for rationalization and would take least time, effort & risk to consolidate or merge.  The High scores indicate ease of consolidation whereas the low scores indicate that the consolidation may be relatively effort intensive and therefore costly and may involve risk, accordingly right planning decisions can be made about the execution & budget for the rationalization projects.

5.     Conclusion

The rationalization scoring technique proposed here is not only easy but it is also plausible & very rational so as to convince most of the stakeholders about the rationalization decisions and eventually to justify any project decisions. The technique relies on hard facts as it is required to collect an extensive set of data for all the systems (the system properties) involved and therefore there is very less room for any error. However judgment is also involved at two stages, first the input from business analysts about what could be the potential target systems for the systems under consideration for rationalization and secondly deciding on the weight of each system property for arriving at the weighted score. Overall this technique and the accompanying scorecard provide a systematic & easy to use approach to consolidation of systems identified as part of rationalization.

Appendix

 

Evaluation Criteria

In order to evaluate the systems under consideration for the potential rationalization states, the system properties noted in the table below should be captured for each system. The evaluation parameters are based on 7 important dimensions such as Alignment with Enterprise Architecture, Non-functional Requirement Coverage, Application Stability, Application Architecture, Deployment Architecture, Data Architecture & Availability of Documentation. Under each of these 7 categories, several properties or parameters have been suggested, each aiding in the decision towards rationalization. The parameters are noted in the tables below under respective categories.

1.      Alignment with Enterprise Architecture or Line of Business Vision

Alignment with Enterprise Architecture / Line of Business Vision
Sr. No	Criteria	Observation	Remarks
1	Approved Architecture Styles	Choose one or more: SOA, Component Based Design, Pipes & Filters (batch jobs), EDA / Messaging Bus, Layered / n-tiered, Cloud, Client-Server(Thick Client)
2	Approved Integration Styles	Choose one or more: Messaging, Web Service, SOA with ESB, SOA, RPC, FTP, Shared Database, Shared Drives, ETL
3	Approved Technologies (Technology platforms,  Products & Frameworks in all Layers )
	Technology platforms	Choose one or more: Java, .NET, Mainframe, Other(Specify)
	Presentation Layer	Products and/or Frameworks
	Service/Business Layer	Products and/or Frameworks
	Integration Layer	Products and/or Frameworks
	Database Layer	Products and/or Frameworks
	OS	Products and/or Frameworks
	Security	Products and/or Frameworks
	Others BPM , BRM, CRM, ERP, VPN, Development Tools (IDEs), Source Code Control, Build & Release, Monitoring tools, Testing tools etc	Products and/or Frameworks

2.      Non Functional Requirement Coverage

Non Functional Requirement Coverage
Sr. No	Criteria	Observation	Remarks
1	Average System Availability or MTBF	current: 95% required: 99%
2	Total Users	current: 450 projected: 700
3	Concurrent Users	current: 150 projected: 350
4	Load - Peak & Offpeak (no. of requests / transactions per day) (Scalability)	current: 500 projected: 800
5	Operation Hours  - Peak & Off peak	Choose one or more: Business Hours /  24 *7 /  12 am - 5 am
6	Response Time (on each Interface)	current:   UI : 1 - 5 sec   Messaging: XXX ms   Database: XXX ms                      required:     UI : 1 - 5 sec   Messaging: XXX ms   Database: XXX ms
7	Security	Specify Authentication: Authorization / Roles: User Profile Repository: Access Permission Manager Single Sign on SAML

3.      Application Stability

Stability
Sr. No	Criteria	Observation	Remarks
1	Top recurring issues
	Ticket count & Frequency	1. 2. …
	Root Cause (Functional & Nonfunctional)	1. 2. …
	Module (where root cause lies)	1. 2. …
2	Unused or Redundant Features (Functional & Non-Functional)
	Feature:	E.g. 1. No more takes 1000 requests a day; load down to just 300 a day  2. Interest rate calculation is not required anymore
	Module / Component (implementing this feature):	1. 2. …
	System / Application(already providing this feature):	E.g. 1. CapOne 2. System CapOne provides interest rates …
3	Most Desired Features (Functional & Non-Functional)
	< Feature Name>	1. 2. …

4.      Application Architecture

Application Architecture
Sr. No	Criteria	Observation	Remarks
1	Is this an Open System - (UNIX, Oracle, Java, .NET etc.) or Closed System - (Mainframe, Natural, AS400, S90 etc)	Choose one or more: Open Closed Hybrid
2	Architecture Style - (Client server, layered, SOA, Component Based, Messaging etc)	Choose one or more: SOA, Component Based Design, Pipes & Filters (batch jobs), EDA / Messaging Bus, Layered / n-tiered, Cloud, Client-Server(Thick Client)
3	Licenses & Validity Period
4	Technology platforms, Products, Frameworks (each  layer)
	Technology platforms	Other(Specify)
	Presentation Layer: <Products & Frameworks>	E.g. JSP, JSF, VB, Flex etc
	Service Layer:  <Products & Frameworks>	E.g. Web Services, EJB, DCOM,
	Integration Layer:  <Products & Frameworks>	Choose one or more: Messaging – e.g. JMS, Web Service, SOA with ESB, SOA, RPC – e.g. .NET Remoting, FTP, Shared Database, Shared Drives, ETL
	Database Layer:	<Products & Frameworks>
	Others:	<Products & Frameworks>
5	Coupling between Layers	Choose one or more: Local calls, RPC, Messaging, Shared Database / Files, Web services Batch
6	System Interfaces
7	Protocols & Data Exchange Formats  for Each Interface	E.g.  Presentation Layer - HTTP, Thick Client Integration Layer -  Web services / RPC (.Net Remoting / RMI / EJB), Flat files etc Database - ODBC / JDBC / Native API
8	Main Modules / Components (e.g. search module, DAO etc)	1. 2. 3. …
9	Business Rule Implementation Where are Business Rules Maintained?	E.g. Hardcoded, file, rule engine, database,
10	Workflow Are there any workflow and how are they implemented?	E.g. product(e.g. webmethods), code, NA
11	Batch/offline processes names & count Are there any application level batch / offline processes?
12	Number of lines of code (LOC), if available	LOC

5.      Data Architecture

Data Architecture
Sr. No	Criteria	Observation	Remarks
1	List of entities	Attach a list or Mention here
2	List of batch jobs  /  ETL
3	Interfaces
4	Archival / Purging / Retention policy
5	Audit Logging policy
6	DR Strategy
7	Back up
8	Supported Character set	ASCII / Unicode
9	Supported Users & Roles

 

6.      Deployment Architecture

Deployment Architecture
Sr. No	Criteria	Observation	Remarks
1	Hosting	Choose one or more: On premise / DR /  Vendor hosted / Cloud
2	Tiers & Clusters (Nodes in each layer ) (Sun V Fire 500, 2 node, active - active load balanced cluster)
	Presentation Layer:
	Service Layer:
	Integration Layer:
	Database Layer:
	Security:
	Other:
3	Shared boxes or Exclusive	Exclusive / Shared
4	No. of Processors & Processor Speed? (CPU count & speed)
5	Memory Used  Memory Available (RAM Size)
6	Disk Storage Size (current & projected)
7	Network speed (GBPS/sec)

 

7.      Availability of Documentation

 

Documentation
Sr. No	Criteria	Observation	Remarks
1	Requirement Specifications / Use Cases	Yes / No / Partial
2	Architecture & Vision document	Yes / No / Partial
3	Design (HLD & LLD)	Yes / No / Partial
4	User / Support Manual	Yes / No / Partial
5	ER model / Data Dictionary / Schema Documentation	Yes / No / Partial

 

 

 

 

 

 

Case Study

 

Client Background

Client is a major retail bank.  Its business spans across segments such as retail, small business, corporate, and investment banking.

 

Project Background

§  A third party vendor carried out Application Portfolio Rationalization for 200 applications from the corporate banking portfolio.

§  Author was engaged to carry out further analysis for a set of applications from two business segments to identify concrete consolidation opportunities

 

End Result

§  Author’s team  prepared a detailed report that consisted of recommendations, estimation, value analysis, feasibility analysis, To Be state architecture & roadmap

 

Scope & Implementation

§  Business value analysis of the application portfolio.

§  As  is state technical analysis.

§  To be state architecture

§  Technical feasibility analysis. This was based on a unique technique, the “Technology Scorecard for Rationalization” that made rationalization decisions easier by quantifying the feasibility .

§  Roadmap definition

§  Recommendations &  value analysis

§  Estimation for implementing recommendations & target architecture.

§  Cost benefit analysis based on the 5 year TCO and  potential savings accrued

 

Technology

·         Analysis of Open and Legacy systems.

·         Open systems included - Java, .NET, UNIX, Oracle, SQL Server, Ab Initio, Webmethods ) and legacy systems included Mainframe, DB2 V8

 

Approach

•       The total team size was 4; consisting of 2 Architects & 2 Business Analysts.

•        One set each of an Architect & Business Analyst was based out of onsite while the other was at offshore.

•        The onsite team carried out  some 30 interviews with all the SMEs for each application

•         This was followed by detailed analysis & preparation of recommendations

•        The recommendations were presented in a 3 – 4 workshops to reach a consensus between SMEs, key stakeholders and business sponsors.

 

Benefits

•       The client‘s management got a synopsis of the current portfolio TCO and savings that can be accrued based on the implementation of the proposed architecture and thus facilitating rationalization decisions.

 

References:

1) http://en.wikipedia.org/wiki/Application_Portfolio_Management

2) http://www.burtongroup.com/Consulting/Engagements/ApplicationRationalization.aspx