Embracing open innovation in agile software development
13 February 2014
Authors: Dr Kieran Conboy (BComms MBS PhD), dean of the College of Business, Public Policy and Law, NUI Galway and Dr Lorraine Morgan (Dip MSc PhD), senior research fellow with Lero, the Irish Software Engineering Research Centre, at NUI Galway
The increasingly complex nature of contemporary software requires contemporary development and project management approaches. Amongst these are agile software development approaches such as Scrum and more ‘open innovation’ approaches such as open and inner source software development. Both are individually quite prevalent in the community (approximately 78% of software developer teams use some form of agile and open innovation).
While both have shown to be very effective, neither exist in isolation and software organisations striving for increased productivity may try to combine both. This article explains the basics of each and then discusses some of the challenges we have seen where organisations try to embrace both agile and open innovation in a single context.
Traditional software development methods were largely plan-driven and ‘waterfall’ in nature, and have often been blamed for the frequency with which projects run over time, budget or fail to deliver value to the customer.
Today, agile methods such as Scrum are highly prevalent in software development practice and are often lauded for being flexible and light-weight and thus enabling creativity and innovation, given their emphasis on close communication and collaboration and valuing people over process. These are achieved through processes such as short sprints, minimal documentation, stand-up meetings, co-located teams and minimising documentation.
A particular strength of agile approaches is that they move away from ‘introverted’ development where the team building the system is detached from the customer. Instead, agile approaches continually involve the customer in the development process, supposedly leading to the development of a more innovative and hence more valuable software product.
Traditionally, new business development and innovation processes took place within the firm boundaries and exclusively with internal resources. In this type of model, typically referred to as closed innovation, firms invest in and manage their own research and development, and rely on their own staff to outperform their competitors in new product and service development.
In addition, after producing a stream of new ideas and inventions, firms must defend their intellectual property thoroughly against the competition (Chesbrough, 2004). Changes in society and industry, however, have led to an increased availability and mobility of knowledge workers and the development of new financial structures. Shorter innovation cycles, industrial research, the rising costs of development and a lack of resources are motives that are driving companies to embrace more ‘open’ innovation strategies (Gassmann and Enkel, 2004). Open innovation has emerged as a model where firms commercialise both external and internal ideas that can be taken to market through external channels, outside a firm’s current business, to generate value.
The term ‘open innovation’ has been defined by West and Gallagher (2006) as “systematically encouraging and exploring a wide range of internal and external sources for innovation opportunities, consciously integrating that exploration with firms capabilities and resources and broadly exploiting those opportunities through multiple channels”.
In addition, Laursen and Salter (2006) focused on external search breadth and external search depth for different types of innovation in a large-scale sample of UK manufacturing firms. These authors defined openness as “the number of different sources of external knowledge that each firm draws upon in its innovative activities” (2006).
The most common interpretation of open innovation is that of Gassmann and Enkel (2004), where they deconstruct the concept into outside-in innovation (where an organisation pulls ideas from outside the organization in a value adding manner, the inside-out innovation (where the organisation brings ideas to market in an integrated and value adding manner) and the coupled process, which combines the outside-in and inside-out (see Figure 1).
However, the concept of open innovation challenges existing theories of innovation adoption due to changes in organisational control. Indeed, it has been argued that open innovation is far more complicated than it seems because having a high degree of openness is not always that clear-cut for organisations and in addition, it can also be costly to implement.
Thus, it is necessary to identify the challenges of an open innovation approach in order to provide credible insights for practitioners. In this regard, exploring the applicability and implications of open innovation principles in an agile environment is a step in the right direction.
CHALLENGES IN COMBINING APPROACHES
There are a number of challenges that organisations need to consider when combining both agile and open development. Firstly, agile methods open development to the customer only. The customer plays an essential part in the agile process, involved in the retrospectives, planning meetings and often continuously on-site to play an integrated role in development.
However, this practice does not extend to include multiple stakeholders and other organisations. We propose that it is useful to consider how the agile innovation process can benefit from becoming more ‘open’, e.g., by opening up the boundaries of a systems development entity to include other stakeholders besides that of the customer. As far as we are aware, no research has focused on the role of other stakeholders in agile development besides the customer.
Secondly, agile methods and open innovation require significant time and resource investment in time from other parties and stakeholders, as well as from the development team who need to commit time to them. There is a tendency for organisations to commit more junior or ‘disposable’ staff to the open networks and retain key staff for closed activities.
Thirdly, due to the commitment required, staff may miss or not prioritise their commitment when their time is constrained or spread across multiple initiatives. This can reduce the impact of the on-site customer. Furthermore, when multiple projects are in progress and each require a commitment, an inequity is created if some resources under-deliver.
To make this issue worse, in one company we studied, engagement was high amongst under-delivering projects where representatives ‘had enough free time to be there’. According to another representative, this paradoxically resulted in the requirements of the least utilised business units being satisfied.
Fourthly, open innovation can be difficult amongst people, teams and organisations when there is a level of competition or diverse interests. This is usually managed to good effect in traditional environments, where plans can be put in place to only engage and share in the non-competitive elements of the work.
However, in an agile contexts where all are co-located, and there is a philosophy of communication and transparency, such a split can be difficult to achieve. Some participants we studied talked of open networks where some were ‘very happy to discuss the ideas of others while being very protective of their own’.
Finally, agile can damage ‘visibility’ in open environments. Agile methods attempt to optimise the use of documentation and eliminate waste paperwork. According to a senior developer we interviewed, team, ideas, experimentation and creativity increased dramatically within the team. The transition ‘allowed the team to invent and develop’ with ‘minimal distraction with documentation, useless team meetings and plans that never get followed’.
However, the new approach caused significant problems in terms of transferring the ideas outside the team and across projects in the research and development group. ‘Visibility as to what the new [agile] team were doing dropped very quickly’ following the transition. This is a significant concern if trying to create an open network of teams or organisations.
In conclusion, there is great potential for agile and open practices to be combined to increase effectiveness of software development. However, a clear understanding of the inter- and intra-organisational applicability and implications of open innovation in agile systems development is required to address key challenges for research and practice.
Dr Kieran Conboy is the dean of the College of Business, Public Policy and Law at NUI Galway, Ireland. His research focuses on contemporary software project management methods such as lean and agile and leads a research team of 15. His other interests include systems analysis and management accounting in systems development projects. His research has been published in the Information Systems Journal, European Journal of Information Systems, Journal of the AIS, IEEE Software and the International Conference in Information Systems. Prior to joining NUI Galway, Conboy was a management consultant with Accenture, where he worked on a variety of projects across Europe and the US.
Dr Lorraine Morgan is a senior research fellow with Lero, the Irish Software Engineering Research Centre, at NUI Galway. Her research interests include open source software, open innovation, cloud computing and inner source. She is currently involved in two Science Foundation Ireland-funded projects around open innovation, open source software, agile and lean systems development, cloud computing and project management. Her research has been published in the European Journal of Information Systems, Database for Advances in Information Systems and Information and Software Technology) and premier conference proceedings (ICIS, ECIS, the International Conference on Open Source Systems) and the IFIP Working Group 8.2 and 8.6 Conferences.
Chesbrough, H. Managing open innovation.’ Research and Technology Management, 47(1), 2004, pp. 23-26.
Gassmann, O. and Enkel, E. ‘Towards a Theory of Open Innovation: Three Core Process Archetypes.’ Proceedings of the R&D Management Conference, Lisbon, Portugal, 2004, July 6-9.
Laursen, K., and Salter, A. ‘Open for Innovation: The Role of Openness in Explaining Innovation Performance among UK Manufacturing Firms.’ Strategic Management Journal (27), 2006, pp. 131-150.
West, J. and Gallagher, S. ‘Challenges of open innovation: the paradox of firm investment in open-source software.’ R&D Management (36:3), 2006, pp. 319-331.http://www.engineersjournal.ie/2014/02/13/embracing-open-innovation-in-agile-software-development/http://www.engineersjournal.ie/wp-content/uploads/2014/02/Head-1024x768.jpghttp://www.engineersjournal.ie/wp-content/uploads/2014/02/Head-300x300.jpgTechNUI Galway,research,software