Factors, frameworks and theory: a review of the information systems literature on success factors in project management

Introduction

This review article provides a critical evaluation of the literature on project success factors in information systems, with a particular focus on organizational information systems. An organizational information system can be defined as 'any of a wide combination of computer hardware, communication technology and software designed to handle information related to one or more [organizational] processes' (Flowers, 1996 cited by Yeo, 2002, pp. 241-242). This definition excludes personal information systems such as those deployed by pilots in the cockpits of modern fighter jets (see, for example, Kopp, 1981/2005). Systems that fall into this latter category are not subject to the issues that arise when technology is implemented in an organizational context and are thus beyond the scope of this evaluation of the literature.

Organizational information systems can vary significantly in terms of their scale, complexity and functionality, as can their host organizations and end-user populations. These are both internal, for example functional groups charged with performing organizational processes such as human resource management or production, and external to the host organization, for example customers and suppliers. Examples of organizational information systems include enterprise resource planning systems, Web-based e-commerce systems and customer relationship management systems.

The specific focus of this review paper falls at the intersection of two concepts (1) organizational information system development projects and (2) success factors. Defined generically, a project is 'a temporary endeavour undertaken to create a unique product or service' (Project Management Institute, 2008, p. 5). An information systems project represents a specialized form of a project. It may be conceived as a temporary endeavour performed to provide an information system for a host organization, or to significantly update or upgrade an existing implementation.

Information systems projects can be further classified as (1) implementation-only or (2) development projects. Implementation-only information systems projects do not include any significant software development (although they may still involve lesser degrees of software development for system installation, data transfer, minor customizations, etc.). They comprise the implementation of commercial packaged software products for a host organization. In contrast, development projects involve a significant amount of software development to create the information system prior to implementation. In this paper, development projects are referred to as organizational information systems development projects to accentuate (a) the organizational aspect of the project's deliverable (the information system hosted on the hardware), and (b) the software development process used to create it.

It is acknowledged that the term project success is difficult to define and interpret (for example, Baccarini, 1999, p. 25; Cooke-Davies, 2004, p. 99<; Davis, 2014, p. 189>; Ika, 2009, p. 8; Standing, Guilfoyle, Lin and Love, 2006, p. 1149; Thomas and Fernández, 2008, p. 733). In addition, those who discuss project success factors often neglect to provide a detailed explanation of the concept of the broader term in their work. Ika (2009) suggests that this issue is ignored because it is presumed that 'everyone knows what is meant by project success' (p. 7), even though success in the context of project management is multi-dimensional, time-dependent and is very much determined by the perspective of the stakeholder who is making the judgment of success (or otherwise). For example, for a business owner, project success may be considered as a positive impact of the work completed on the business in question. It is also important to differentiate project success from project management success. The former refers to the satisfaction and benefit accrued by the host organization. The latter is concerned with management performance, typically whether the project adhered to the requirements of the schedule and budget.

Taking the above into account, for the purposes of this paper success is considered as an expected outcome of all projects (as identified by, for example, Hall, Beecham, Verner and Wilson, 2008, pp. 31-32; Nicholas and Hidding, 2010, p. 152; Project Management Institute, 2008, p. 5; Subramanyam, Weisstein and Krishnan, 2010, p. 137). We argue that whether or not a project is successful may be judged by assessing the extent to which the project in question meets its intended aim. At the same time we recognize that others (for example, de Wit, 1998) have argued that to objectively measure the success of a project is impossible.

Project success factors are conceived here as the conditions, circumstances and events that contribute to the success of a project. This definition is in alignment with terminology used in Ika's study (2009, p. 8), which is regularly cited alongside Jugdev and Müller (2005) as one of two literature reviews that provide a generic analysis of project success factors. It is important that project success factors are distinguished from project success criteria (Cooke-Davies, 2004, p. 99). The latter are the principles and standards by which project success can be judged, i.e. the measures that indicate that a project has been successful (Lim and Mohamed, 1999, p. 243). In the research literature these two components of project success are sometimes deployed loosely, occasionally giving the impression of misinterpretation and/or confusion (for example: Bernroider and Ivanov, 2011, p. 326; Davis, 2014, p. 189; Gingnell, Franke, Lagerström, Ericsson and Lilliesköld, 2014; Lim and Mohamed, 1999, p. 244; Turner, Ledwith and Kelly, 2009, p. 292).

In the domain of information systems, the burgeoning interest in project success factors has created a major research stream with findings generated mainly from quantitative studies (Larsen and Myers, 1999, p. 397). Project success factors have attracted the attention of many researchers over the years, as noted by a number of authors (for example, Bryde, 2008, p. 800; Cooke-Davies, 2002, p. 185; Christensen and Walker, 2004, p. 39; Söderlund, 2004a, p. 186; Söderlund, 2011, p. 159; Thi and Swierczek, 2010, p. 570) and their study holds a prominent position in the field (Lu, Huang and Heng, 2006, p. 295). Articles on project success factors feature in project management journals such as the Project Management Journal and the International Journal of Project Management (Ika, 2009, p. 11; Söderlund, 2004a, p. 189). Practitioners have also felt the influence of project success factor research. This is evident in the way that its output has been codified into standards, for example in project management bodies of knowledge (Papke-Shields, Beise and Quan, 2010, p. 660).

Although other researchers have previously addressed the question of project success factors in their work, their contributions have typically been at a generic level and, to date, a literature review with a specific focus on organizational information systems development has not been published in any established peer-reviewed journal. In addition, although they are valuable to information systems research from a broad perspective, the two most recent generic analyses of project success factors (Jugdev and Müller, 2005; Ika, 2009) are now somewhat dated. Much has changed in the working environment in the past few years, especially in the domain of organizational information systems development with, for example, the uptake of mobile technologies, the proliferation of new breeds of software and the move towards agile information systems development methods. Our contribution is thus a timely analysis that responds to the question of how project success factors are treated in the organizational information systems development literature and where future research endeavours should be directed to enhance theoretical development in the domain.

As well as presenting an up-to-date and novel articulation of the treatment of success factors in an area of information systems project management practice that has previously been unexplored in detail, this paper also has practical value to information management, information systems, and project management researchers and practitioners. Since there is a paucity of literature reviews in the domain (despite the proliferation of articles that discuss project success factors per se), it is perhaps unsurprising that members of the first of these audiences - those who research information systems project management - have been accused of lacking a grasp of the literature (for example, Morris, 2010, p. 140). Our work addresses this issue by presenting a much-needed evaluative summary that can be accessed by the information systems project management research community at large. In addition, given that organizational information systems development projects are also renowned for high failure rates (as noted, for example, by Gingnell et al., 2014, p. 21; Glass, 2006, p. 15; Walsh and Schneider, 2002), the content of a paper such as this can help improve understanding amongst a wider practitioner audience.

The main contribution of this work is found in the identification of the four broad research themes that have emerged from the detailed analysis of the body of literature on the topic of project success factors in information systems development. These themes, and the main findings as related to them, are summarised in Table 1.

Our articulation highlights where further opportunity lies to build on extant knowledge and to enhance understanding in this area of information systems research, as is presented in the account below.

The paper responds to recent calls for more literature reviews in the information systems literature. For example, in 2014 the editor of the European Journal of Information Systems appealed for 'literature reviews that offer the most solid foundations for theory building and research landscaping' (Rowe, 2014, p. 242). By summarising the prior research and critically examining its various contributions, we identify some thematic gaps and possible future research directions.

Methods and presentation of findings

Our main findings as related specifically to organizational information systems development derive from an analysis of fifty-six papers of core relevance published between 1979 and 2012. These are listed in the Appendix. They include papers from the leading, peer-reviewed conferences and journals in the fields of information systems, information management and project management. This core set of papers represents just over a third of the material cited in this paper (56 out of 163 publications in total).

The papers analysed in our review were identified through extensive literature searches using commercial online databases such as ABI/Information Complete (ProQuest), ACM Digital Library (ACM), Emerald Journals (Emerald), Expanded Academic ASAP (Gale) and IngentaConnect (Ingenta). Google Scholar was also used to identify a number of articles. The initial searches were conducted using combinations of the terms shown in Table 2.

Many of the papers are published in information systems, project management and information management titles including the European Journal of Information Systems, Information Research, Information and Management, the International Journal of Information Management, the Journal of Strategic Information Systems, International Journal of Project Management and the Project Management Journal. Since many phenomena in information systems research are interdisciplinary, titles from other related fields, such as business management and software development, were accessed to provide relevant material, as is established practice in the domain (Rowe, 2014, p. 247).

In recognition that software development has been described as a practitioner-led discipline (Glass, 2006), and that important advances in project management research have also been made in publications outside the realm of academic journal articles (Jugdev and Müller, 2005, p. 25), various other sources such as books and commercial articles, written by academics and practitioners alike, were deemed appropriate for the analysis presented in this paper and are thus also cited in the analysis below.

The literature search also involved citation pearling to identify further material from reference lists found in the papers identified. This proved to be particularly important, as a number of papers that are key to the theme under discussion, such as Moynihan (1996), would not have been identified simply by using the search terms shown in Table 2.

The content of the material accessed was then classified according to:

• the project type under discussion: for example, decision support system, executive information system, software development project.
• the description of factors listed: for example, success, failure, risk.
• the classification of the information systems project(s) examined: development, both development and packaged system implementation, or unspecified.
• the perspective from which the account of the project(s) is presented: client, in-house, supplier (or unspecified).
• the context of the work described, to include details of geographic location and industry sector where specified.
• the data source on which the analysis presented is based: for example, survey data, anecdotal evidence.

An overview of the classifications as applied to the papers is provided in the Appendix table. The classification process contributed to a broader analysis that led to the identification of the four main research themes in the extant literature, as explored in further detail below.

The structure of our paper allows for the broad context of success factor research to be presented first. It is against this that the specifics of project success factor research as related to information systems in general, and to organizational information systems development as a particular sub-category of information systems, are then set. We then present the detail to support the articulation of the four themes summarized, exposing where there are overlaps between the general and specific literatures, and gaps particular to organizational information systems development. We conclude by arguing that there are many opportunities to develop theoretical insight in an area that is ripe for research and that such endeavours will be of value to both information systems research and practice in general, as well as organizational information systems development project management in particular.

The development of project success factor research in the domain of project management

With its focus on organizational information systems development as a sub-category of information systems, this paper contributes to a larger body of literature on project success factor research. Interest in this area in general (i.e. independent of fields such as information systems and project management) grew in the middle of the twentieth century. The popularity of an article in Harvard Business Review (Daniel, 1961) on the development of executive information systems spawned initial interest. This work argued that companies are typically subject to between three and six industry-specific success factors (Daniel, 1961, p. 116). Almost two decades later, in 1979, a further article was published in Harvard Business Review on the same theme (Rockart, 1979). This later contribution became regarded as seminal in the study of success factors (Fortune and White, 2006, p. 53). It describes an interview method to assist chief executive officers in establishing the information needs of organizations. The method was later extended to include (amongst other developments) success factor classification constructs (Bullen and Rockart, 1981). These ideas were subsequently popularized in the project management literature, thus seeding the stream of research on project success factors.

In the period up until the mid-1980s empirical research on success factors in the field of project management was largely based on limited data sets and/or anecdotal evidence from single case studies, and derived from small samples (Cooke-Davies, 2004, p. 101; Jugdev and Müller, 2005, p. 24; Söderlund, 2011, p. 160). Although thought-provoking, the output of these studies cannot be considered as generalizable. Over time more rigorous studies were undertaken, drawing on data from larger samples (Söderlund, 2011, p. 160). The aim here was to identify generic project success factors applicable to all types of project, as opposed to a specific context or domain of application (Fortune and White, 2006, p. 53; Söderlund, 2004b, p. 659). While the majority of the findings from these later studies were generated from quantitative surveys (Ika, 2009, p. 12; Larsen and Myers, 1999, p. 397), a few in-depth case studies were also published (Söderlund, 2011, p. 160).

The work on project success factors described above is noted as belonging to a tradition labelled the factor school (Söderlund, 2011, p. 158). Typically such studies identify ten or more items in the format of project success factor lists (i.e., a larger number than the three to six of Daniel's 1961 work on general success factors noted above). Added to this empirical research are other studies relevant to success factor research which have been labelled as theoretical. These either derive their findings from secondary data, or make suggestions about project success factors that have yet to be tested empirically (Fortune and White, 2006, p. 56).

A reading of the more recent project management literature as related to project success factors points to the long-held belief (or at least the assumption) that the identification and subsequent use of project success factors will lead to improved project performance (Papke-Shields et al., 2010, p. 660; Pinto and Prescott, 1988 cited by Söderlund, 2004a, p. 189). It is therefore unsurprising that efforts continue to be directed towards explaining which conditions, circumstances and events contribute to the success of a project. Recent studies reported in the information systems literature cover, for example:

• the impact of political behaviours and game-playing in information systems project development (Chang, 2013)
• the role of management support in multiple project environments (Elbanna, 2013)
• means of quantifying success factors through deployment of an expert-based Baysian model (Gingnell et al., 2014)
• the influence of cultural and external pressures on IT project performance (Gu, Hoffman, Cao and Schniederjans, 2014)
• the relationship between management styles and control in a financial services information systems implementation (Gregory and Keil, 2014)
• the part that organizational change plays in project success (Hornstein, 2015)
• critical skills for managing information technology projects (Keil, Lee and Deng, 2013)
• risk in information technology project performance (Liu and Wang, 2014)
• uses of management control systems in information systems projects (Sakka, Barki and Côté, 2013).

Further incentive for such research is the common practice of attempting to achieve organizational goals through project work (Dvir, Lipovetsky, Shenhar and Tishler, 1998, p. 915; Söderlund, 2004a, p. 186). Another driver of these efforts is to understand why organizations that adopt this way of working continue to suffer poor project performance (Mir and Pinnington, 2014, p. 202; Söderlund, 2011, p. 159).

The general view is that with enhanced knowledge of project success factors, managers can focus their energies on those that matter most, then, it is hoped, watch as project performance improves (Andersen, Birchall, Jessen and Money, 2006, p. 129). The desire to pin down the most important project success factors perhaps explains the steady stream of research publications on this theme (as noted by, for example, Bryde, 2008, p. 800; Cooke-Davies, 2002, p. 185; Christensen and Walker, 2004, p. 39>; Söderlund, 2004a, p. 186; Söderlund, 2011, p. 159; and Thi and Swierczek, 2010, p. 570). This also accounts for the calls for more research into generic project success factors (for example, Bryde, 2008, p. 800; Davis, 2014).

Project success factors in the information systems and organizational information systems development literature

The focus of the evaluation presented in this paper is previously published research on success factors in a particular area of information systems project management, i.e. projects related to organizational information systems development. Having set the general context of current priorities in success factor research in project management, the remainder of this review article considers the coverage of project success factors in this area by characterising its main themes and by highlighting gaps in knowledge to be addressed in future research.

The fifty-six papers of core relevance to software development, competences, activities or artefacts published in the past four decades (Table 2) (henceforth referred to as the core papers) are authored by academics, practitioners, commercial organizations and, in one case, a professional body. The majority derive from the USA. The body of work under review was published across a wide range of journals. The scattering of the literature, which is also evident in closely related areas such as enterprise resource planning projects (Ngai, Law and Wat, 2008, p. 549), may be considered an indicator of the disparate nature of research in this domain. Such disparity impedes knowledge development (Savolainen, Ahonen and Richardson, 2012, p. 10) and perhaps explains in part why there are still many differing opinions on project success factors (Andersen et al., 2006, p. 130), as will be elaborated below.

Despite the difficulties associated with researching an extant literature that is located across many titles, some authors have clearly made efforts to access and use it as a starting point for the identification of candidate success factors to explore in their own studies. In contrast, a number give the impression that their research has been carried out in near isolation. As a consequence, the core papers, when taken as a collection, indicate that there has been little opportunity for learning from previously published studies. This issue has been identified as a shortcoming of project management publications in general and often highlighted at project management conferences and in project management journals (Reich et al., 2013, p. 938). A further reason why the more recent authors appear not to have benefited from prior research is the lack of detail on research methods made available in the earlier work. For example, many accounts fail to discuss the limitations of the work presented (for example, there is regular use of localised convenience samples); the validity and reliability of findings; and the lessons learned from the studies reported. Nor do these authors offer recommendations for further research. In some cases, the research methodology is not made clear at all. This is a criticism that has been levelled at project management research as a whole (Smyth and Morris, 2007 cited by Morris, 2010, p. 143).

The content of the core papers covers both in-house and supplier software development projects. Even so, in over half the cases (33), this distinction is not always clear from the written accounts of the studies. The most common perspective presented is that of the host organization. Again, however, this detail is not always made explicit in the text of 30 papers. Three papers (Cerpa and Verner, 2009; Hartman and Ashrafi, 2002; Sharma, Sengupta and Gupta, 2011) consider the client and supplier perspectives together, and one focuses exclusively on the supplier perspective (Moynihan, 1996). The project settings comprise both public sector and commercial organizations. The findings from the studies derive predominantly from the analysis of quantitative data collected by survey. Some qualitative research is also presented, based mainly on structured interviews.

The detailed evaluation of previously published research on success factors related to organizational information systems development projects presented below considers the characteristics of the literature according to the four themes presented above in Table 1.

The identification and exploration of project success factors and the generation of project success factor lists: quantity of factors and lack of agreement

Consideration of individual project success factors in information systems research

It has been established that general research efforts related to project success factors stretch back several decades. Even so, the identification of individual factors is still highlighted as a priority area. For instance, there is a need to explore in greater detail the meaning of brief, high-level abstract ideals of factors that frequently feature in factor lists. As illustration, executive support is often named as a project success factor, but simply learning that this is important does not tell a project manager how to support a project more effectively (Zwikael, 2008a, p. 387). Another area that requires attention is the inappropriate grouping of factors. In some cases, project success factors are abstracted to such a level that they become composites (or groups) of two or more individual factors. Although it has been argued that there can be merit in grouping success factors together (Clarke, 1999, p. 140; Gingnell et al., 2014, p. 25), this can also present problems. For example, the monitoring and control process has often been noted as significant (for example, Taylor, 2000, p. 25; Walsh and Kanter, 1988, p. 19), yet these are actually two distinguishable elements and should be considered as such (Gardiner and Stewart, 2000, p. 252). Separating them out helps identify the effects of each individual factor and the possible interactions between them. A third limitation of previous research is that it has not identified which factors can (and cannot) be transferred from one type of project to another. Part of the problem is that the extant studies have contributed to a project management literature that, in general, is predominantly normative (Ahlemann, El Arbi, Kaiser and Hexk, 2013, p. 43): it describes how projects should be managed, rather than how they are actually managed (Nicholas and Hidding, 2010).

Over the past decade a number of studies have considered in detail individual success factors for information systems projects. Unlike the remainder of organizational information systems project success factor research, such studies often employ qualitative methods and concentrate on factors related to particular actors, namely:

1. Project manager competences (for example, Bloom, 1996; Müller and Turner, 2007; Skulmoski and Hartman, 2010; Seiler, Lent, Pinkowska and Pinazza, 2012; Thite, 2000)
2. Executive management support (for example, Beckley and Gaines, 1991a; Beckley and Gaines, 1991b; Wight, 1983; Young and Jordan, 2008; Zwikael, 2008a; Zwikael, 2008b)
3. End user involvement (for example, Hsu, Lin, Zheng and Hung, 2012; Petter, 2008; Subramanyam et al., 2010).

Ample opportunity remains to enhance understanding here (see, for example, Petter, 2008, p. 708; Hsu et al., 2012, p. 9). For example, with the exception of the three areas listed above, few other actor-based success factor studies exist. Research into other project actors, such as the project team, is merited.

A few published studies of project success factors in the extant literature consider processes, such as, for example, project success factors for the software development life-cycle (for example, Jones, 1996; Smuts, van der Merwe, Kotzé and Loock, 2010). As well as those who highlight the dearth of the coverage of processes (for example, Hofmann and Lehner, 2001, p. 58), there are others who point out the difficulties of examining them when it is often the case that the processes prescribed are routinely not followed in practice (de Bakker, 2009, p. 3; Fortune and White, 2009, p. 37). Thus there is merit in investigating how alternative behaviours and activities of individuals impact organizational information systems development project processes (Kutsch and Hall, 2005, p. 595).

The deployment of project success factor lists in information systems research

The broader literature shows that since the 1960s project success factors have typically been presented in lists (Fortune and White, 2006, p. 53). Many lists have been produced. Indeed, the literature of project management is replete with them (Alojairi and Safayeni, 2012, p. 17). Furthermore, existing lists readily spawn others. For example, Gingnell et al. (2014) have recently summarized and consolidated nine lists from the information systems literature into a new one that features twenty-one factors (pp. 23-25). Most of these lists are named as project success factor lists. Other labels have also been applied: for example, check lists (Alojairi and Safayeni, 2012, p. 17), top ten lists (Klakegg, 2009, p. 500) and hit lists (Gingnell et al., 2014, p. 23). A continued lack of agreement across these published project success factor lists demonstrates that it has not yet been possible to generate a comprehensive list of the factors that are applicable to all projects (Altuwaijri and Khorsheed, 2012, p. 38; Fortune and White, 2006, p. 54; Söderlund, 2004a, p. 186). It should also be noted that project success factor lists often do not distinguish between the most commonly cited project success factors and the most influential project success factors. Those that attract more attention may not be the most important (Klakegg, 2009, p. 500). A further issue is that the lists that resemble the top ten format negate the possibility of the influence of a far wider range of factors on any particular piece of work. Belassi and Tukel (1996, p. 142) and Jugdev and Müller (2005, p. 24) note another concern related to the sheer quantity of candidate project success factors in the information systems literature. The factors listed are not grouped or classified in any coherent manner in the published studies; an exception to this in the organizational information systems development literature is the work of Chow and Cao (2008, p. 964).

Individual project success factors and lists in the organizational information systems literature

Our analysis of the core papers reveals that, on average, each identifies twelve project success factors, thus reflecting the pattern in the wider literature as noted above. In contrast, two previous studies that form part of the set present a high number of factors: Moynihan (1996) derived 113 risk constructs for project success (or failure) and Bannerman (2008) refers to (although does not elaborate upon) over 300 artefacts that 'appeared to be relevant or important in enabling or inhibiting the performance and/or outcome' of organizational information systems development projects (p. 2123).

Our audit revealed 488 unique project success factors related to organizational information systems development. Although further consolidation might be possible (for example, by additional effort to resolve near-synonymous terms), this number does not reflect the critical nature of a small number of success factors as originally defined by Daniel (1961, p. 116), nor the spirit of a top ten list or similar as noted above. This indicates that the success of organizational information systems development projects is the function of a very large range of factors.

This also points to a possible reason for the failure of project success factor research to provide a deep understanding of real-world projects, as highlighted above: the majority of success factor studies do not acknowledge, let alone address, the large number of factors relevant to organizational information systems development projects. This does not, however, imply that individual factors are not commonly shared, nor that individual projects are so dissimilar that nothing from one case could be applied to another. Indeed, our means of describing project success factors as entity-characteristic pairs as shown in Table 3 demonstrates that there is sufficient commonality to support the transfer of certain factors between projects of different types.

In some instances the relative rankings of the importance of project success factors identified is presented in the lists published in the information systems literature (Kwon and Zmud, 1987, cited by Larsen and Myers, 1999, p. 397). In keeping with the presentation style employed by the majority of such studies, Table 5 shows the top twenty-four project success factors in the core papers. It is worth highlighting that none of these factors is exclusively restricted to organizational information systems development. Indeed, most are applicable to generic projects and their management. The exceptions are end users' involvement and the maturity of the project deliverable's technology.

Two findings from this analysis are of particular interest. First, Table 4 highlights differences in the level of detail that can be found in success factor lists. Project requirements, for example, has four detailed entries in the list, each with different characteristics: (1) stability; (2) clarity; (3) completeness; and (4) fitness for purpose. In contrast, end users' involvement can be considered a much vaguer term. It does not distinguish, for example (1) the project activities in which the end users are to be involved, nor (2) the form that this involvement is to take (for example, full or part-time). This is likely to be indicative of a literature that has tended to focus on processes rather than socio-technical issues (as also noted by Hornstein, 2015, p. 291), and the relative maturity of the consideration of these two perspectives. Here we highlight an opportunity for contributions from information systems researchers working in areas such as socio-technical studies and social informatics to make a contribution to widening the scope of studies of information systems project success factors.

A second observation is the low level of agreement on the ranking of project success factors across the papers, reflecting the case in the broader literature noted above:

• the top two factors appear together in only eight of the papers
• the top three factors appear together in only three papers
• the top four factors only appear together in one paper
• the top five factors do not appear together in any single paper.

We argue that this may be accounted for by the different contexts of the studies reported in the core papers (although it should be noted that the provision of details of context is often poor in the core papers, as can be seen in Table 2). Others have previously called for research into context-specific project success factors, for example by specific project type, geography or culture (Söderlund, 2011, p. 159), where significant differences have previously been identified (Pinto and Covin, 1989, p. 49). Equally it has been argued that the influence of organizational dynamics, particularly associated with information systems and information technology projects, is often ignored (Gauld, 2007, cited by Altuwaijri and Khorsheed, 2012, p. 38) yet merits particular attention as an important contextual factor. Since it is rare that a particular set of success factors holds the same importance across multiple studies (Kwon and Zmud, 1987, cited by Larsen and Myers, 1999, p. 398), investigations into the relative value of project success factors in similar environments is likely to furnish more meaningful comparisons and assessments of the generalizability of conclusions from research reported. For example, Smuts et al. (2010) identified almost fifty individual success factors related to the outsourcing of software development work. However, their findings derive from a single, South African case study and this is highly unlikely to be representative of project success factors in the development life-cycle of other contexts. This work would also address, to an extent, the question of applicability of top project success factors in particular contexts, yet also recognize that a single unified list is unlikely ever to be established (Ika, 2009, p. 9) due to the unique nature of each project.

Time should also be regarded as part of the context. This is because the influence of any project success factor is temporal: it varies at different points across the life-cycle of a project (Larsen and Myers, 1999, p. 398; Pinto and Covin, 1989, p. 49 cited by Jugdev and Müller, 2005, p. 26; Pinto and Covin, 1989, p. 59; Pinto and Prescott, 1988, p. 5; Pinto and Slevin, 1988 cited by Söderlund, 2004a, p. 189). In addition, the project life-cycle itself is not static, but a dynamic phenomenon (Altuwaijri and Khorsheed, 2012, p. 38; Ginzberg, 1981 cited by Larsen and Myers, 1999, p. 398; Paré and Elam, 1997 cited by El Sawah, Tharwat and Rasmy, 2008, p. 260; Söderlund, 2004a, p. 189; Söderlund, 2011, p. 159-160). A further element of time, extraneous to the studies under review but important to the methods deployed in our study, is the period over which the core papers were published: much has changed in information systems in the thirty-three years that the collection spans. Therefore, it is perhaps inevitable that the attention paid to individual factors over the years is varied.

Contributions of individual/group project success factors to project success (or failure): much research, little agreement, weak coverage in the organizational information systems development literature

Some have claimed that there has been limited research into the influence of project success factors on project success (or failure) per se (for example, Andersen et al., 2006, p. 128; Thi and Swierczek, 2010, p. 572). However, this review of the literature demonstrates that several papers on this theme have been published in the information systems literature since 2000, as noted in the examples listed in Table 5.

With the exception of a few qualitative studies (for example, Christensen and Walker, 2008; de Bakker et al., 2012), research into the influence of project success factors on project success per se is predominantly quantitative, as is the case for success factor lists as discussed above. Typically the results derive from surveys coupled with descriptive statistical analysis. The studies tend to focus on just one factor, or occasionally two, where the factor under investigation is considered as an independent variable that directly influences the dependent variable of project success, as illustrated in Figure 1. Notably, project success is not normally considered in terms of specific success criteria, i.e. measures that indicate that a project has been successful. Some of the studies also give consideration to moderating variables, i.e. those factors that might have a bearing on the influence of the factor under investigation.

The vast majority of studies listed in Table 5 conclude that the factor under investigation is indeed a factor of project success, i.e. it has an influence. This suggests that future studies of the impact of other factors will demonstrate that project success is a function of a far wider range of factors than previously believed. This finding calls into question once more the validity of the top ten checklists highlighted above.

Although a detailed discussion of each of the papers listed in Table 5 is beyond the scope of this paper, the study published by Sauer et al. (2007), which conforms to the research model presented in Figure 1, merits further discussion. The authors raise an issue that is largely ignored in the published research on project success factors. They argue that if a project success factor is to be treated as an independent variable to be measured or assessed in some way, then there should be some agreement of the units of measurement to achieve this. Taking project size as an example, Sauer et al. (2007) recommend that project budget is an inappropriate unit of measurement. Instead, project size is better measured in terms of effort. This can be viewed as the product of project team size and project duration, and expressed, for instance, as person-months (Sauer et al., 2007, p. 82).

A further issue identified by Sauer et al. (2007), but not taken into account by the majority of reported studies, is the degree or magnitude of any factor that influences project success. Without this information, the logical assumption is that the magnitude of a specific factor is directly proportional to its influence on project success. Thus it might be assumed that the more success factors in the mix, the better. However, Sauer et al.'s (2007) study shows this is not necessarily the case. This demonstrates, for example, that the influence of project size on project success is not linear. Instead, it rises slowly from twenty-five person-months to 1000 person-months, at which point it starts to rise far more steeply (Sauer et al., 2007, p. 81).

A third issue related by Sauer et al. (2007, p. 87), and noted by others (for example, Andersen et al., 2006; de Bakker et al., 2012; Hsu et al., 2012; Tesch et al., 2009), is that the relationship between a specific factor and project success 'is [typically] not as simple or direct as many think' (Sauer et al., 2007, p. 87). In many cases, specific factors are found to comprise a number of other sub-factors (as noted, for example, by de Bakker et al., 2012; Gelbard and Carmeli, 2009; Verner et al., 2007).

It is also relatively common for studies of project success factors and their contribution to project success to acknowledge that the interaction between factors is relevant and that these interactions merit further investigation (for example, Bryde, 2008; Hall et al., 2008; Tesch et al., 2009). A number of studies (for example, Gelbard and Carmeli, 2009; Hsu et al., 2012; Tesch et al., 2009) also identify the moderating effects of intermediate factors. In addition, these studies reveal small, localized causal chains associated with the specific project success factor under investigation (for example, Hofmann and Lehner, 2001; Hsu et al., 2012; Jiang et al., 2002; Wallace et al., 2004). Any claim for a specific project factor to have a direct and distinct influence on project success is therefore difficult to justify. As Hall et al. (2008) put it: 'the impact of individual factors is complex' (p. 33).

The analysis of the literature thus reveals that despite the number of papers published on the influence of success factors on project success and given the high number of project success factors that can be identified from sampling the literature (488 from the core papers), there is clear opportunity for further investigations into the influence of success factors on project success. This is especially the case in terms of endeavours to identify the full range of factors at work and how groups of factors have an impact.

Causal interactions between individual/groups of project success factors and simulations of these: incomplete and untested causal models

The relationships between factors is a priority area for project management research. Indeed, it has long been argued that project success factors should not be considered on an independent basis (Clarke, 1999, p. 141). This is because 'the inter-relationships between factors are at least as important as the individual factors' (Fortune and White, 2006, p. 54). Knowledge of these relationships is of particular benefit to practitioners engaged in information systems development (Nandhakumar, 1996 cited by Fortune and White, 2006, p. 54). However, the success factor list approach tends to treat each factor as an independent variable and overlooks the interactions between the factors identified (Nandhakumar, 1996 cited by Larsen and Myers, 1999, p. 398). As a result relationships are left unexplained (El Sawah et al., 2008, p. 260; Ginzberg, 1981 cited by Larsen and Myers, 1999, p. 398).

The published research discussed in the previous section supports the contention that numerous factors have an impact on project success. A closer analysis of this literature on project success in information systems also reveals a low understanding of how these factors interact. The need to enhance the extant knowledge was identified almost two decades ago by Belassi and Tukel (1996, p. 150), and has been expressed by others since (for example, Kim and Pan, 2006, p. 73). The earlier work noted that a combination of a large number of factors along the project life-cycle leads to project success (Belassi and Tukel, 1996, p. 142). Equally, the means by which these factors promulgate their influence in practice is significant: the factors do not typically affect a project's outcome in a direct manner (Belassi and Tukel, 1996, p. 142). It is therefore necessary to recognize that the influence of any given factor, or combination of factors, differs at points along the project life-cycle (because time influences context, as noted above) and this artefact of factor behaviour needs to be considered when assessing the overall impact of any factor on the final project outcome.

In some respects it is not surprising that there is low understanding of how success factors interact in information systems projects. Such projects are multifarious (British Computer Society, 2004, p. 15; Williams, Klakegg, Walker, Andersen and Magnussen, 2012, p. 44; Xia and Lee, 2004, p. 69) and their management is 'a very complex undertaking in which a complex network of interrelationships and interactions exists' (Abdel and Madnick, 1983, p. 346). Indeed, complexity itself has been cited as a factor that influences project outcome, and normally one of failure, as noted in three of the core papers (Charette, 2005; Tiwana and Keil, 2004; Wohlin and Andrews, 2002).

As far back as the 1980s it was noted that an integrative model is necessary to make sense of the complex network as identified above, and this model should be flexible enough to represent the large number of factors in the complex network of interrelationships. In addition the model should be able to determine the dynamic behaviour of interactions between factors (Abdel-Hamid and Madnick, 1983, p. 346). Such a model would thus enhance understanding of information systems project success factors in two stages: first, by identifying the relationships between factors and, second, by making visible the dynamics of these relationships over the life-cycle of the project. The majority of research papers that have attempted to model project success factors address only the first issue. It should also be noted that most of these studies deal with a range of information systems projects, and not organizational information systems development projects specifically. Nonetheless, the factors considered in these studies are the same as those that feature in the success factor lists for organizational information systems development projects, for example executive management support and end user involvement. It is therefore likely that their findings are relevant to organizational information systems development projects.

Table 6 summarizes the characteristics of studies that have modelled the interactions of project success factors.

While these studies have enhanced understanding of the interaction of project success factors, they have limitations, and these may go some way to explain the low understanding in this area noted above. Most striking perhaps is that even though our detailed study of the core papers identifies 488 unique project success factors, the studies summarised in Table 6 above consider just a few key success factors each. A further issue is that only three sets of researchers (Akkermans and Van Helden, 2002; King and Burgess, 2006 and 2008; and Yetton et al., 2000) investigate and report upon (simple) causal loops in their work. A related issue is that the treatment of success criteria across the studies as a whole is superficial. As such, the papers present rather simple scenarios. A stronger portrayal of the complexity of the causal interactions between success factors in information systems projects in general, and organizational information systems development projects in particular, would result from the inclusion of a wider range of factors, relationships and discussion of project success criteria in future work.

A number of researchers have noted that in cases where project success factors have been modelled, the models presented are somewhat static (for example, Altuwaijri and Khorsheed, 2012, p. 38; King and Burgess, 2006, p. 67). It is thus interesting to consider the extent to which attempts have been made to simulate the dynamics of success factor relationships.

It has been argued that 'simulation is a useful tool for tactical management in software engineering. It provides a means to study complex phenomena in project development that cannot be carried out easily with actual cases' (Lee and Miller, 2004, p. 80). Added to this, the use of modelling techniques for projects in general has proved extremely useful in uncovering how complex projects behave, enhancing understanding of the actuality of projects (Cicmil, Williams, Thomas and Hodgson, 2006, p. 682-684).

Despite this, simulation remains an under-explored technique for understanding success factors in information systems projects, including those related to organizational information systems development (King and Burgess, 2008, p. 430). Some earlier publications point to computerized simulation modelling techniques as a means to extend knowledge of success factor dynamics (for example, Abdel-Hamid and Madnick, 1983, p. 346; King and Burgess, 2006, p. 62; King and Burgess, 2008, p. 430; Lee and Miller, 2004, p. 80). However, much of this work is tentative (for example, King and Burgess, 2008). The use of simulation models to enhance understanding of the dynamic relationships between success factors in organizational information systems development projects is thus another area ripe for research. There is thus significant scope for researchers to generate more comprehensive causal models to better explain factor influence in real-world organizational information systems development projects. In addition, it would be useful if the conceptual models already developed were validated, complemented, modified and/or extended through the work of others in the future.

Project success factor frameworks: dearth of frameworks for organizational information systems development projects

A variety of generic project success factor frameworks, in a range of shapes and forms, have been proposed to address the issues such as how project success factor may be grouped. These are considered below.

Discussion of frameworks is commonplace in the mainstream general project success factor literature (Ika, 2009, p. 11; Jugdev and Müller, 2005, p. 25) and some of the frameworks described have been tested empirically (Jugdev and Müller, 2005, p. 29). However, it has been noted that there remains a need for more inclusive project success factor frameworks (Ika, 2009, p. 11).

Early examples of general project success factor frameworks include Kerzner's model for project excellence (Kerzner, 1987, p. 33) and Slevin and Pinto's project implementation framework (Slevin and Pinto, 1987, p. 35). By the mid-1990s, Belassi and Tukel (1996, p. 144) had developed a more holistic framework. This encouraged the consideration of project success factors in terms of their classification and relationships. It also highlighted the implications of not taking particular project success factors into account (Jugdev and Müller, 2005, p. 25). The value of Belassi and Tukel's framework was that it was assessed by a range of study participants, some of whom were information systems practitioners (Belassi and Tukel, 1996, p. 149). Another general framework that was tested for its usefulness amongst information systems practitioners was that of Westerveld (2003, p. 415). This framework linked project success factors to project success criteria (i.e. the measures that indicate that a project has been successful). Evidence of the value of Westerveld's framework is shown in the assessment of an enterprise resource planning implementation (Westerveld, 2003, p. 417).

More recently, a number of success factor frameworks directly related to information systems (as opposed to general) projects have been published. For example, Vithanage and Wijayanayake (2007, p. 37) developed a framework for large-scale information systems implementations. Similarly, in 2010 Hawari and Heeks proposed a design-reality gap model for enterprise resource planning projects (p. 151). Although both of these studies are concerned with implementation-based projects, the factors that they encompass suggest that the frameworks in question may be relevant to the implementation aspects of organizational information systems development projects. Indeed Hawari and Heeks (2010, p. 155) state explicitly that there is no reason why their framework cannot be exported to other information systems applications. Fortune and White (2006) have also shown how a generic framing device can be applied to information systems projects (including one organizational information systems development project). The evidence provided in these studies thus suggests that generic project success factor frameworks can be applied to information systems projects in general, and possibly also to organizational information systems development projects.

The literature search conducted for the analysis presented in this article, however, uncovered just one framework that addresses factors for organizational information systems development projects in particular (Keil et al., 1998, p. 80). This framework focuses on risk, with particular attention paid to two of its dimensions: (1) perceived relative importance; and (2) perceived level of control. The general content of Keil et al.'s 1998 paper, including the framework described, is clearly useful in itself. However, its greater value in the context of a review of the literature such as this lies in its identification of the need for further development of project success factor frameworks for organizational information systems development projects. Despite the length of time since the publication of Keil et al.'s study in 1998, the analysis of the literature presented in this article supports the currency of this view.

Conclusion

Our goal when preparing this review was to generate a critical evaluation of the literature pertinent to success factors in organizational information systems development projects within the broader context of the information systems literature. It was anticipated that the output would highlight where future research endeavour could extend the existing knowledge base on project success factors in organizational information systems development projects. This goal has been met by considering four broad research themes that emerged from an analysis of extant literature as summarised in . Although we cannot claim comprehensive coverage of all literature belonging to the domain – this is often impossible in this kind of work (Rowe, 2014, p. 246) - the collection of work reviewed is comprehensive enough for identifying (1) where previous research effort has been invested and (2) possible future research directions.

Perhaps the most prominent finding here is just how much opportunity there is to enhance the understanding of project success factors in organizational information systems development so that better decisions can be made in practice, and those decisions can be translated into actions that are properly resourced. Indeed, it is striking how a research stream that has been in existence for several decades and has produced plenty of output in this time (despite claims to the contrary, for example, Belout and Gauvreau, 2004) has failed to generate any convincing theoretical models. What is found here is an abundance of factors and a paucity of frameworks. This apparent immaturity of the domain is also reflected in the superficial treatment of some topics in studies that lack rigour. This is exhibited, for example, in the reluctance in some studies to acknowledge that project success factors cannot be considered as independent variables. This superficiality is also demonstrated in cases where there appears to be general ignorance of certain topics, such as the question of causal loops.

While it is useful for practitioners to have an awareness of a set of project success factors (many of which are presented in the literature), it does not follow that such knowledge can directly improve project performance. Practitioners need to understand, for example, which of the factors are most important to project success (Altuwaijei and Khorsheed, 2012, p. 38; Fortune and White, 2006, p. 54; Söderlund, 2004a, p. 186); how these factors can be used to alleviate problems faced in practice (Clarke, 1999, p. 139; Nakatsu and Iacovou, 2009, p. 64); and the potential consequences of actions taken (King and Burgess, 2006, p. 59; King and Burgess, 2008, p. 421). In short, literature that gives evidence of the existence of project success factors in the form of lists, but does not give advice on how they can be used to help eliminate problems in practice, is inadequate. There is a need for an additional narrative that describes what these real-world factors represent and also for advice and guidance on the practical application of these factors so that practitioners can predict the consequences of their actions. Stronger theoretical models would be beneficial to those who wish to understand the primary causes of success and failure (King and Burgess, 2008, p. 421). However, with very few exceptions (for example, Rodriguez-Repiso, Setchib and Salmeron, 2007), the literature is almost completely silent on the subject of project success factor management.

There are several possible explanations for the criticisms levelled above. First, the lack of strong research tradition in project management can be cited. The main impact of this deficiency is that this domain lacks research direction, both in terms of the identification of priority areas for research and in promoting appropriate approaches for the execution of empirical work. Equally a literature that is scattered makes it difficult for researchers to access, critically evaluate and draw upon established work in the course of their own research. It is also the case that the question of context - for example in terms of the timing, culture, governance, geographic location, strategic intent - as related to the organizational information systems development projects under scrutiny has a strong influence on project success factor behaviour. The complexity of organizational information systems development projects is also acknowledged as an inhibiting factor here. Thus it may be unreasonable to expect the findings from any particular study to necessarily match with others.

That said, this review makes clear that there are several areas related to project success factors in organizational information systems development projects that are ripe for research and extending knowledge in the context of the project management of organizational information systems development projects would also be welcomed by the broader information systems community. Some of the opportunity lies in addressing straightforward gaps in knowledge. For example, it is not possible to say with certainty which are the key project success factors in organizational information systems development projects. Others have made comments such as this in the context of information systems projects in general (for example, Fortune and White, 2006, p. 54). However, none have gone as far as to say that the notion of the existence of critical success factors may be illusory and the search for them is not worth continuing. Our own view is that that the word complex may be a better epithet than critical to apply to the phrase success factors. Further gaps in knowledge may be filled by exploring artefacts and actors as success factors for organizational information systems development projects and by examining the use of simulation models to enhance understanding of dynamic relationships between success factors in organizational information systems development projects.

Further research could address the contradictory nature of earlier studies in the domain. For example, there is little agreement as to the relative importance of particular project success factors. Even if agreement cannot be reached, the reasons why it is lacking would be a starting point of interest to the wider research community. Future research contributions that explore these issues and address others highlighted in this review will help build an evidence base that would engage both the academic and practitioner communities. This, it is hoped, will lead to the generation of theoretical insight to enhance the understanding of real-world organizational information systems development projects, as well as improve the general theoretical underpinning of project management in the domain of information systems as a valuable, and valued, domain of research.

Some of these questions may be addressed by engaging a broader community of researchers in studies that have traditionally been dominated by project management and/or general information systems research. Such individuals may be found working in particular areas of information systems research, such as socio-technical studies, social informatics and other related disciplines. Much more may be achieved by drawing on such research domains that (1) have a tradition of in-depth qualitative analyses of systems implementations and (2) use a range of approaches that go beyond a focus on processes to include consideration of actors and artefacts in the context of the wider macrostructure of systems implementation. Thus there is the potential for the discussion in this domain to move on from lists of factors to the presentation of frameworks, detailed simulation models and the possible development of theory.

Acknowledgements

The authors gratefully acknowledge the assistance of Dr Bruce Ryan in preparing the final version of this paper for publication.

About the authors

Robert Irvine is an Information Systems and Software professional. He has over 20 years' experience of managing and leading projects in Europe, the US and China for a range of companies that includes SAP, Motorola and Rolls Royce. He holds a Bachelors degree in Energy Engineering, a Masters degree in Business Administration, a Masters degree in Software Engineering, and a PhD in Computing.
Hazel Hall is Professor of Social Informatics in the Institute for Informatics and Digital Innovation within the School of Computing at Edinburgh Napier University, UK. She holds a Bachelors degree in French, a Masters degree in Library and Information Studies, and a PhD in Computing. For further information please see the profile at http://hazelhall.org/about. Hazel Hall can be contacted at h.hall@napier.ac.uk