See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/224136792
Why do public sector IT projects fail Conference Paper · April 2010 Source: IEEE Xplore
CITATIONS
READS
2
61
3 authors, including: Javed Ashraf
Naveed Khattak
National University of Sciences & Technology
National University of Sciences and Technology
3 PUBLICATIONS 19 CITATIONS
20 PUBLICATIONS 138 CITATIONS
SEE PROFILE
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
Security in Software Defined Networks and IoT View project
All content following this page was uploaded by Javed Ashraf on 09 January 2016. The user has requested enhancement of the downloaded file. All in-text references underlined in blue are added to the original document and are linked to publications on ResearchGate, letting you access and read them immediately.
Why do Public Sector IT Projects Fail Javed Ashraf College of Signals National University of Sciences and Technology Rawalpindi-46000, Pakistan
[email protected] Abstract Adoption of Information Technology (IT) can make a valuable contribution to development of public sector organisation. A reasonable size of budget is allocated every year to IT projects in public sector in both developed and developing countries. However, at present, the majority of public sector IT projects fails either totally or partially. The percentage of failure is even more in developing countries.
A number of research studies have been conducted to identify the reasons of failures of IT projects in private and public sector in developed countries. However, very little work of similar type has been done in developing countries. In Pakistan also no such research work has been published. Based on case study of three projects, this paper studies the reasons leading to failure of IT projects initiatives in public sector of Pakistan.
Keywords: IT projects, public sector, eGovernment, software engineering, project management, developing countries, risks.
1. Introduction In past few decades IT has seen explosive growth; however the field of IT itself is not as mature as compared to other fields and disciplines. This trend is more pronounced in developing countries where the IT industry is still in the stages of infancy and sufficient maturity level has not been achieved by most of the companies. The IT companies in developing countries are facing the challenges of extreme shortage of skilled and experienced human resource and higher employee turnover. As a result the IT industry in Pakistan is not capable of providing appropriate support to private and public sector organizations. Much has been written about the extent and causes of IT project failure and numerous studies have discussed a range of recognised risk factors. The famous and most quoted studies are [1], [2] and [3]. These research works have been mostly conducted in public and private sector
Athar Mohsin College of Signals National University of Sciences and Technology Rawalpindi-46000, Pakistan
[email protected] of developed countries. Very little research has been done to carry out such study in developing countries. Moreover no consolidated data is available on software projects performance in public sector of developing countries. This paper focuses on identifying causes that contribute towards failure of IT projects in public sector of Pakistan. In Pakistan, Electronic Government Directorate (EGD), which was established under Ministry of Information Technology in 2002, is responsible to plan and prepare Electronic government projects and to provide technical advice & guidelines for implementation of E-government projects at the Federal, Provincial and District levels [12]. This is a directorate with small size staff which can not meet the requirements of all federal, provincial and district level organisations. Thus there are serious governance issues related to implementation of E-government projects which will be discussed in this paper. This paper is structured as follows. First we describe three categories of issues that contribute in success and failure of software project. Then we overview the study of three projects in public sector organisations of Pakistan and describe its main findings. Then we give important conclusions from the findings that are peculiar to developing countries and not so common in developed countries. We also give implications of our study for research and practice.
2. Related Work As mentioned earlier, there are a large number of studies done to identify the causes of failures of IT projects. Examples are: The Chaos Report (1995) [1], The OASIG Survey (1995) [2] and The KPMG Canada Survey (1997) [3]. However, not much of the similar type of research work has been done in developing countries. To our best knowledge no such research work in Pakistan has been published. While, as this study will show later, there is a lot in common in our study and above mentioned studies; there are some causes of failure of IT projects which are peculiar only in developing countries in general and Pakistan in particular which will be discussed later in the paper.
3. The Study To explore the arguments set out above in a deep and meaningful manner, a case study approach was considered to be suitable [4] [5] [6]. We chose three different public sector organisations located at Islamabad, the capital city of Pakistan, for our study. Three project managers and two client managers agreed to participate in this study. All these participants were collectively responsible for delivering the software product. This research was complemented with interview with some of end users - who were the receivers of the service provided by the respective organisation. Initially, unstructured interviews were conduced with managers and end users using set of questionnaire prepared based on best practices in three categories of issues. Based on this, questionnaire was improved to include some of the important things which we missed and which managers and end users thought were important in failure of the projects. Follow-up structured interviews were thereafter arranged with the same managers and end-users in order to confirm the results and clarify any unclear information [6]. Since the focus of this research was to identify causes of failure in three different categories, discussed below, the questions were fairly focused. Notes were taken during the interviews in a logbook (interviews were not tape-recorded as requested by the interviewees) and later transcribed into the computer. The interviews were combined with observation and review of organisations documentation (two of the three organisations shared the documentation), which allowed us to verify and validate the empirical findings. Finally, data analysis was done by comparing the different findings against each other and initially forming themes, which were later merged/divided and categorised into appropriate headings.
3.1
Organization and Projects Profile
Three organizations in the study varied significantly in size, ranging from small (200 employees) to large (3000 employees). All three projects (we name them as project A, B and C) involved development, deployment of software at customer premises and training of end users. Project A involved integration and some customization of about 15 COTS products plus automation of business functions of various departments. Projects B and C were aimed at development of software for automation of some of the business functions of concerned organizations.
3.2
Division of Issues in Three Categories
Detailed analysis of the projects in the study uncovered wide range of causes of failures. In order to segregate various types of reasons, we have divided them in three categories as under.
Project Management and Risk Management Issues. This category includes issues related to project management including risk management. The PMBOK [7] has been taken as reference for preparing questionnaire. Some questions have also been taken from [8]. SW Engineering Issues The issues related to noncompliance of software engineering best practices. We have used SWEBOK [9] as reference for preparing questionnaire for this category. Governance and Funding Issues. Issues other than discussed in above two categories, like governance, funding, culture, politics, etc.
4. Findings The scores are awarded as under. For yes/no questions, yes=10 and no=0. For other questions scores are given from 0 to 10. Exceptions are mentioned with the concerned questions. Scores are mentioned for each project in the respective column with their average in the right most column.
4.1
Project Outcome
All three projects were delayed and could not complete the desired functionality. Two projects were also overbudgeted. Project B did not over budget because organisation did not agree to enhance the funds for the projects. Summary is shown in Table 1 (figures are in %age). Table 1: Project Outcome Questions A How much was the project delayed? 50 How much was project over- 55 budgeted? How much of the desired 80 functionality was completed?
4.2
B C Avg 60 40 50 0 210 88.3 70 50 66.7
Governance and Funding Issues
This category includes issues related to governance, funding, culture, etc. We have included those questions only which contributed considerably (more than 5%) towards failure of the projects. Organisations have dedicated departments and their heads to manage finance, HRM, operations etc. Similarly, it is the job of the Chief Information Officer (CIO) to direct the activities of the IT department to optimize the usage of all the resources in place, while setting out a strategic vision for the organization to keep-up with the incredible pace of change. Without an IT strategic plan, and a CIO to create and enforce it, organisations will be like a ship without charts or
a captain, drifting aimlessly and expensively in the information ocean. But with a CIO, organisation data and information resources can be managed and utilized efficiently and effectively, and organisation will be poised to take full advantage of the opportunities provided by the electronic revolution [10]. In most of the developed countries, all large and medium sized organisations have separate IT department and a CIO to manage it; whereas, small organisations hire consultant for this job. Pakistan, however, has not yet realised this requirement for its organisations. None of the organisations in this study had IT strategic plan and only one organisation had appointed CIO or equivalent, who too was not directly reporting to CEO. In fact he was reporting to third level management and hence did not have required influence towards implementation of project in the organisation. Other two organisations even had no IT staff to manage, analyse, validate and approve various documentation, plans and code related to projects. Thus, the quality of requirements specifications, design, coding and testing was left to contractor managers only. Summary of our findings related to governance and cultural issues are mentioned in Table 2 below. Certain questions were only related to organisations and others were related to IT companies. Table 2: Governance issues Questions A Does your organisation have a CIO or 10 equivalent? (Yes=10, No=0) Does your organisation have IT 5 Strategic Plan? Did your organisation hire consultant 10 for this project? Did the project report to steering 8 committee? How committed was top 6 management? How involved was top management? 5 Did the project team feel they had 5 executive support? Was feasibility study carried out? 8 Was this project aligned with 7 achievement of some defined business objectives? Do you think change of senior 10 leadership was one of the main reasons of project failure (yes=10, partially agree=7, no=0)? Were there conflicts between different 10 departments? Was the project adequately resourced? 1 What was the skill level of end-users? 7 Did users resist change? 8
B C Avg 0 0 3.3 0
0
1.7
0
0
3.3
5
0
5.3
5
3
4.7
3 3
2 1
3.3 3.0
6 7
4 6
6.0 6.7
7
0
5.7
8
7
8.3
4 4 9
3 2.7 4 5.0 9 8.67
Funding Issues. Budgets for public sector IT projects are allocated for one year whereas projects approval, bidding, selection of vendor and contract signing takes about 8-10 months. Hence it is highly unlikely that IT project could complete in one financial year. Therefore funds allocated for the project had to be surrendered at the end of financial year. The concerned organisations had to take up the case for re-allocation of funds which would typically take 5-8 months, leaving very little time to plan progressive payments to the contractors. Thus project A, which was initially planned to be completed in two years time, took about 4 years to complete about 80% of the desired functionality. Project C was over-budgeted by 210%. Summary of issues and findings related to availability of budgets are discussed in Table 3. Table 3: Funding Issues Questions A B C Was budget approved for the project? 10 10 0 (Yes=10, No=0) Was appropriate budget set? 7 8 10 Was budget made available at 5 6 6 appropriate time? Was project over-budgeted? (What 55 30 210 %age)
Avg 6.7 8.3 5.7 98.3
4.3 Project Management and Risk Management Issues PMBOK [7] defines project management as “the application of knowledge, skills, tools and techniques to project activities to meet project requirements. The project manager is the person responsible for accomplishing the project objectives.” Thus project management and project manager play important roles in success or failure of IT projects. Issues in this category mostly relate to IT companies (solution developer) and their project managers. However questions were put across both client and contractor managers to get fairly good idea about issues related to project management from two perspectives. We received contradicting responses from client and contractor managers due to obvious reasons. Here we are mentioning the average of the scores. Table 4 shows the summary of our findings. Most of the questions in this section have been taken from [8]. Risk management. Project Risk Management includes the processes concerned with conducting risk management planning, identification, analysis, responses, and monitoring and control on a project; most of these processes are updated throughout the project. The objectives of Project Risk Management are to increase the probability and impact of positive events, and decrease the probability and impact of events adverse to the project [7].
Table 4: Project management issues Questions A B C Did the project have a full-time 10 7 8 project manager? Was the project manager experienced 10 5.5 10 (minimum three successful projects)? Was the project manager PMP 0 0 0 certified? How well was the project managed? 5 3.5 3 Did the project manager have a clear 5 6 4 vision of the project? Did the project manager change 0 10 10 during the project? How effective was project planning? 3 5 2 Was a realistic schedule defined? 2 5 4 How appropriate was the 2.5 6 4 methodology used? How well was progress controlled 2 5 4 against the plan? Did the project manager know the 6 7 8 project’s status most of the time? How well were issues handled during 3 4 1 the project? Did the project have effective change 6 6.5 4 control?
Avg 8.3 8.5 0 3.9 5.0 6.7 3.3 3.6 4.2 3.7 7 2.7 5.5
Questions and responses related to risk management are shown in Table 5 [8]. The responses indicate that organisations in the study tended not to use a specific/formal risk management methodology, and were quite unclear about the prioritization of risks. These responses are curious considering that all projects encountered unanticipated, indicating the importance of having strong risk management. Table 5: Risk management issues Questions A B Did you use a specific risk 3 2 management methodology? How well were risks identified as the 3 2 project’s start? How well were risks managed 4 3.5 throughout the project? How well were risks prioritized when 4 3 identified? Were mitigation/contingency plans 3 1 pre-determined? Was responsibility assigned for 3 1.5 monitoring risks? Did unanticipated problems arise 7 9 during the project?
4.4
C Avg 0 1.7 0
1.7
0
2.5
0
2.3
0
1.3
5
3.2
10 8.7
Software Engineering Issues
Software engineering is the application of a systematic, disciplined, quantifiable approach to the development,
operation, and maintenance of software; that is, the application of engineering to software [9]. Software engineering processes are well defined and documented. However there were a number of issues related to this process. In this section we include only those issues related to software engineering discipline which contributed substantially in failure of the projects. Table 6 gives summary of our findings. It is important to note that Requirement Specification (RS) document was not signed off for any of the projects. For project A two parties did agree on about 70% of the requirements in RS document and the differences for rest of 30% were never resolved. Developing skills in new technology delayed project A and C by 33% and 30%, respectively. Table 6: Software Engineering issues Questions A B C Was feasibility study done for the 7 6 0 project? How well was requirement elicitation 7 5 5 and analysis done? Were all stakeholders involved for 6 6 9 the requirement elicitation? How well users participated in 6 4 3 requirement engineering process? Was requirement validation done? 5 4 9 Did you sign off requirement 5 0 0 specification document? How often requirements changed 7 8 10 after requirements engineering process? How effective was requirements 7 6 4 change management process? Did you use specific “quality 6 5 7 analysis and evaluation” technique for design? Did you use specific “measures” to 6 5 7 assess the quality of software design? How satisfied were client managers 6 5 6 on quality of software design? Did software coding involve 10 0 10 developing skills in new technology? How much (in %age) did the 33 - 30 developing skills in new technology contribute towards delaying the project? How well was unit and integration 7.5 7 0 testing done? How do you grade SQA process of 7 6.5 3 the contractor? In your opinion what %age of total 80 70 45 functionality completed by the contractor.
Avg 4.3 5.7 7 4.3 6 1.7 8.3
5.7 6
6 5.7 6.7 21
4.9 5.5 65
5. Important Conclusions Previous section showed various findings related to three categories of issues. We note that, mostly, the causes of failure of IT projects are common in developing countries including Pakistan and developed countries. Although we are not providing any evidence for this claim about other developing countries (as we could not find any such type of study), we say this keeping in view many commonalities in developing countries, such as political instability, economic conditions, lack of education and computer skills, lack of certified (CMMI, etc) software companies, lack of IT awareness, reluctance to use IT to automate business functions, etc. However there are some issues which are more applicable to developing countries in general and Pakistan in particular. These issues are discussed below. Change in government/ head of organisation. Project A and B could not be fully deployed at user premises due to change in head of organisation. Though these projects were over-budgeted and over-scheduled, the projects did meet 80% and 70% of the desired functionality, respectively. These software applications could have been used by respective organisations for their benefit. Lack of top management commitment to the project. Project A had little commitment from top management whereas projects B and C had almost no commitment from head of the organisations. This is not common in developed countries. Availability of funds. Non availability of funds at appropriate time contributed in delay of the project A by about 10 months. Projects B and C were also delayed due to this reason by about 4 and 6 months, respectively. This shows that the procedures for allocation and release of funds need to streamlined and the bottlenecks required to be removed. This problem is again more pertinent to developing countries. Absence of dedicated IT Department and CIO (or equivalent). In Pakistan, most of the large public organisations have small cell comprising a network manager, a DBA and a web developer, managing so called IT systems. Medium and small organisations do not have even these small cells and they mostly rely on few technicians. As discussed earlier all large and medium organisations of Pakistan should have dedicated IT department and a CIO. CIO will provide advice and other assistance to the organisation head and other senior stakeholders to ensure that IT is acquired and information resources are managed for the organisation in a manner that implements the policies and procedures of the organisation consistent with the priorities established by the organisation head. EGD should work on this issue and should
recommend government of Pakistan to issue necessary notification to have a dedicated IT department and CIO to lead and manage it for all large and medium size organisations. This issue is again more relevant to developing countries. Non seriousness of end users towards participating in requirement engineering process. Due to lack of senior management support and interest, the end users did not show interest in the requirement engineering process. End users of one organisation (project A) did show some response; however other two organisations were noncooperative. Absence of Project Risk Management Process. This factor has been highlighted by studies in developed countries as one of the causes of failure. However in Pakistan, as is evident from results shown in Table 5, this area is almost completely neglected by both IT companies and public organisations. Users’ computer skill level. Most of the clerical staff and some of the senior management in three organisations found lacking in their ability to use computers. Most of them use computers for typing purposes only. Again, this problem is almost non-existent in developed countries. Lack of maturity of IT companies. As mentioned earlier, the IT companies in Pakistan are facing the challenges of extreme shortage of skilled and experienced human resource and higher employee turnover. Due to this and other reasons, these companies are not yet matured to handle the issues mentioned in this paper. Out of about 1161 IT companies registered with Pakistan Software Export Board (PSEB), only two companies CMMI level 5, three level 3 and thirteen are level 2 certified [11]. Resistance to change. Due to lack of computer skills and fear of transparency in their working, end users of the three systems resisted the implementation and deployment of the systems. For them IT is an additional work.
6. Implications 6.1
Research Implications
Since very little research work of this type has been done in developing countries and none in Pakistan (to best of our knowledge), there is a lot that needs to be done. This work can form the basis for further and more focussed research. One of the objectives of this research was to apply the findings of previous research done in developed countries on the public sector IT projects in Pakistan. This study identified that there are certain issues which contribute more towards failure than these do in developed countries. This implies that public sector organisations and IT companies in Pakistan need to focus
more on these issues. Absence of such research work in Pakistan indicates that public sector organisations have not analysed their failures and thus have not learnt from their mistakes. This work emphasizes that policy/project continuity, senior management buy in and involvement from start till end, good governance, project management including risk management and matured software engineering process are key elements towards making IT projects in public sector successful.
6.2
Practice Implications
There is a lot that can be taken by organisation IT, operational and financial staff from this study. First, organisations need to take IT seriously. Not using IT to automate or support business functions is no more an option now. Second, Pakistan need to have a dedicated CIO (or equivalent) for all medium and large size public sector organisations as is common in developed countries and must in some of the countries like, USA and UK. Third, formal and matured project governance is necessary for success of IT projects. Finally, risk management should be part of project management and formal risk management process should start at the time of feasibility study and continue till the closure of the project.
6.3
Limitations
8. Conclusion Despite numerous studies and well documented causes for failures of IT projects, it is alarming that significant numbers of IT projects still fail to deliver benefits on time, within budget and to expectations. At the same time number of IT projects has succeeded in both developed and developing countries. This study has attempted to identify causes of failures in IT projects of public sector in Pakistan. By addressing the issues and mitigating the risks identified in this paper, developing countries can increase the chances of success of IT projects.
References [1] The Standish Group, 1995, CHAOS, The Standish Group International. [2] The OASIG Survey (1995) [3] The KPMG Canada Survey (1997) [4] Pettigrew, A. M., (1990), Longitudinal Field Research on Change: Theory and Practice. Organization Science, Vol 1, No. 3, pp.267-92. [5] Walsham, G. (1993), Interpreting Information Systems in Organisations, Wiley, Chicester. [6] Yin, R K (1994), Case Study Research - Design and
This research has few limitations therefore its findings may be interpreted cautiously. First, it is based on study of only three public sector projects. Second, it is based on only failed projects. For better analysis, the study could have included some of the projects which could be called successful. Third, responses to questionnaire were based solely on the perceptions of study participants from three organisations and contractors (IT companies) which can not be verified. Finally, all three projects were outsourced projects. Inclusion of some of the in-house software developed projects in this study could have further made our analysis more interesting and comparative.
Methods, Second Edition, Sage Publications, London. [7] PMI. A Guide to the Project Management Body of Knowledge (PMBOK Guide). Project Management Institute, 2004. [8] Paul L Bannerman. Software Project Risk in the Public Sector, IEEE computer Society, ASWEC’07, Australia 2007. [9] SWEBOK. Software Engineering Body of Knowledge, 2004 Edition, IEEE Computer Society.
7. Future Work As mentioned earlier, since not much of this type of research work has been done in developing countries, there is a lot that can be done in this area. Some of the areas for future work include: extending and refining the study by examining a wider and more balanced range of projects; need for risk management in IT projects; need of CIO in public sector organisations; “outsourcing” versus “in-house” software development; how to improve requirement engineering in public sector organisations. This study only identifies causes of failure; further work may be done to suggest some formal methodology to address the issues mentioned in this paper.
[10] Chip Collis. Does Your Agency Need a CIO? http://www.lib.niu.edu/2000/ip000717tc.html [11] PSEB, List of CMMI rated companies in Pakistan, http://www.pseb.org.pk/item/cmmi [12] A Profile of Electronic Government Directorate (EGD), http://pakistan.gov.pk/e-government-directorate/about/ about-dir.jsp
