Introducing Inner Source? What to consider

Introducing inner source? Here's what to think about

Inner source concerns the use of open source development practices inside of the organizational walls of a company. Or using the terms of Eric Raymond, building a bazaar inside of the Cathedral (Wesselius, 2008).

These practices (among many) include:

  • Universal access to code, information, and knowledge
  • Peer-reviews and community scrutiny
  • Informal communication, coordination, and requirements engineering
  • Self-selection of tasks and merit-based governance
  • Frequent and fast releases

Companies have started gaining interest in adopting such practices and craftsmanship for building software internally. They see a need to open up inside and break down silos around teams both inside and between business units.

Inner source helps breaking down silos

This way of working is often implicit for many startups and small-sized companies. However, when you have a large company with 100+ developers, inner source can help to create bridges and improve cross-functional collaboration while working towards a common goal.

This way of working has been given many names in research such as Hybrid Open Source, Firm-internal Open Source, Corporate Open Source and Progressive Open Source, but more commonly as inner source.

The first reported case study of inner source was at HP in 2001. Since then, further case studies have been reported from companies such as SAP, Nokia, Alcatel-Lucent, Philips, and IBM.

In later years, it’s seen an increased adoption rate among more “open source mature” companies, e.g., Pay-Pal and other TODO group members. For some, it’s been the standard way of working without knowing the phenomenon had a name, as exemplified by Will Norris at Google’s open source programs office.

What are the benefits?

Reuse of software through inner source and shared assets

Reported and perceived benefits align with many of those connected to the use of open source, and includes:

  • Increased reuse through shared assets
  • Improved quality through peer-reviews and more eyeballs on the code
  • Increased innovation as creativity and requirements engineering is opened up to other parts of the company
  • Increased speed of development
  • Improved mobility of developers as they have open access to all the code and information, as well as access to necessary communication channels.
  • Increased awareness of the software that is being developed inside the company and its goals and vision.

Another feature compared to open source is that the company can open up sensitive and differential projects internally that would not be able to be published as open source.

So, how do you set it up?

That all sounds great, but how do you go about to introduce inner source to your company? And what projects should you inner source?

First, we can note there is no correct way of doing things. If you look and compare any handful of open source communities you will notice that practices and processes vary a lot. Hence, what practices to adopt and how needs to be tailored to each organization, its context, and culture.

Second, we can differentiate between two types of inner source setups, infrastructure-based and project-based inner source. Do note that the two types, of course, can be mixed.

Infrastructure-based vs. Project-based inner source

In the infrastructure-based, the company sets up an infrastructure on which developers can put up their projects, much like GitHub. They offer functionality for reporting and discussing features and bugs as well as source code repositories enabling collaborative development. These platforms may also be referred to as Forges (if you think back to for example SourceForge and derivatives such as FusionForge). Developers can browse through the projects and contribute or engage where they wish. The reuse can be considered opportunistic and ad-hoc.

In the project-based, the company has chosen a specific and often critical software project where a core-team is started to serve and maintain the project for the rest of the users (or customer-groups) inside the company. This is a common approach for platforms and frameworks used in software product lines for example.

In both setups, the goal is to build internal communities with developers similar to those of open source communities, where they are allowed to scratch their own itches rather than putting a ticket on the other team, and also encouraged to talk to each other.

As I’ve seen in my research, it can also be used as a springboard to release new projects as open source. Once it’s inner sourced it’s more about attaching an open source license to it and contribute it to a foundation or put it on GitHub and start building an external community (which in itself deserves a series of blog posts).

What factors should you consider?

Below I will iterate nine key factors to adopting inner source from a framework developed by Klaas-Jan Stol and his colleagues at Lero and the University of Limerick (Stol, Avgeriou, Babar, Lucas, & Fitzgerald, 2014). They created the framework based on literature and the case studies I mentioned earlier, and validated it with new case studies, at Philips, Rolls Royce, and Neopost.

I have myself applied it to a smaller development organization where I investigated the possibilities of introducing inner source and how a so-called forge and internal infrastructure should be built (Linåker, Krantz, & Höst, 2014).

Factors to consider when introducing inner source

The nine factors can be divided into the three categories Product, Practices & Tools, and People & Management.

Product

1. Seed product

The first step is to select an initial seed product to boot-strap and start the inner source process. This product should have an initial implementation but not be in a maintenance state so that it can start being worked on and a target for contributions. There should be a mission statement and road-map for the product stating the direction so others know if it is aligned with what they want, especially as they will be creating a dependency towards it.

The product should also be something that is of value to a lot of people inside the company, such as a common component or platform used in a product line or family, in order to attract as many users and developers as possible. One idea may be to do a mapping of the potential use cases and users to get an overview of how a potential shared asset may be valued by the different parts of the company.

2. Stakeholders

A second factor that goes hand in hand with the former is that the seed product should be of value to a wide set of stakeholders so that it can attract diverse views, competencies, contributions, and requirements from different parts of the company. Also if many stakeholders have an interest in the seed project it will motivate managers to dedicate and pool more resources.

To get more stakeholders on board and help with the internal community building, certain actions may need to be taken to spread the word and market the shared asset. One such action could be to create a focused user group, arrange town hall meetings, or lunch & learns where potential users and developers are helped, e.g., with setting up the development environment.

Another approach to help potential users get their hands dirty is to invite them to do a rotation for a week or two at the team or group that may have the main responsibility for the shared asset.

3. Modularity

Modularity is a generally accepted practice, also in open source development and for any projects that should be inner sourced. It allows developers to focus on subsets of functionalities and enables parallel development. This is a balancing act as the project needs to provide utility enough to be of value, while not being too heavyweight and complex, in order to encourage reuse of the asset. In their paper, Stol et al. highlight that the goal is to create an “architecture for participation” to allow developers to better understand the code and enable them to engage and contribute.

Practices & Tools

Tools & Practices for introducing inner source

4. Development practices

The development style in open source is usually informal and unstructured compared those found inside companies. This is especially noticeable in the requirements engineering process as requirements are implemented alongside their elicitation and specification, and usually found spread out in multiple channels such as issue trackers, mailing lists, and IRCs. Issues are often self-assigned based on the individuals’ personal interests.

There is, however, a limit to how informal things can get, given that it is a professional business and the development must align with the organizational environment and surrounding processes. For example, the release planning and requirements engineering may have to be supervised in order to align with the company’s general product planning and strategies for successful products to be shipped on time. Hence, e.g., self-assignment may have to be somewhat in-line with the scope of the respective teams.

In cases of a project-based inner source, it is important to consider the different processes of the core-team and the customer-groups that uses the shared asset in their development. This is especially important for customer groups to consider when contributing something upstream to the core-team. Close collaboration is also needed so relevant domain knowledge is acquired and market requirements communicated. If the core-team does not have the capacity to implement requested features, the customer groups can still implement the necessary changes and later have the option to contribute upstream to the shared asset.

5. Quality assurance

Linus law - “given enough eyeballs all bugs are shallow” - is often used when talking about quality assurance in open source. Peer-reviews and community scrutiny are central practices. Again, these practices are not unique for open source development, but the open way in which it is done is. This pushes developers to improve their code as it will be put up for public display with their name attached to it.

Regular and frequent releases is another common practice in open source, but also in many company environments that use CI/CD. Open source communities have generally moved to use time-based releases as these avoid rushing in features and allows more structured bug-squashing.

6. Tools

A common infrastructure and tool-set are pivotal to make shared assets accessible and enable collaboration and contributions. Infrastructure-platforms (cf. Forges) are used to host the different collaboration and communication tools. A popular alternative is GitHub Enterprise. The public version of GitHub is becoming a defacto standard for open source projects why the step for many developers may not be that big.

The pull-requests serve as a stage-gate and coordination-point for contributions and allow for peer reviews and code-centric discussions, while more general discussions can be performed via other mediums such as mailing-lists or Instant Messaging applications (Kalliamvakou, Damian, Blincoe, Singer, & German, 2015). An issue with GitHub, however, may be that it can be troublesome and confusing for non-technical users which creates a barrier for new users, hence needing consideration.

Another tip I got when giving an industry presentation on inner source was Skalli. It can be a good alternative when you have all the infrastructure already in terms of source code repositories and issue trackers, but you need a searchable interface with a splash screen for each project. Currently, it’s only maintained by some people at SAP, but may still be a good to check out.

People & Management

Factors concerning People and Management for introducing inner source

7. Coordination & Leadership

Leadership and influence in open source communities are often based on principles of meritocracy. Those who contribute and show merit gain more influence and coordination responsibilities. This requires flexibility and for some a new mindset in order for new “leaders” to emerge, outside of the hierarchical and organizational structure of the company. Initially, these roles are constituted by the creators of a project, often referred to as the maintainers (or the “benevolent dictator” for some).

For more critical assets it may be needed to have designated roles in the core-team that oversees release planning and requirements engineering, as self-assignment of a task may not be feasible. This especially concerns shared assets with many customer groups. In this case, the core-team need to consider the different release schedules but also coordinate contributions to minimize the risk of duplicate work. The different wills and wishes may pose another area needing supervision, as different customer groups may have conflicting feature requests, maybe even conflicting with the road-map of the core-team.

8. Transparency

Key characteristics of open source are that the development and community are transparent and open to anyone. Discussions, decisions, and code are available for anyone to read and review. This may impose discomfort for some developers as their code will be open for scrutiny for anyone to see or that they feel threatened by giving up knowledge unique to them.

However, it also offers possibilities such as helping newcomers in their on-ramp, enables mobility of developers between the shared assets, and a possibility for converting lurkers to users. For managers, fears regarding transparency can be more oriented to losing control and resources, and exposing project planning and not meeting performance goals.

To grow the internal communities it is extremely important that there is a welcoming attitude to anyone interested, and that maintainers have time to support and help with any inquiries. First-time contributors should be shepherded (if requested).

Basic documentation (or open source hygiene as I like to call it) needs to be up-to-date, and openly available, with as few barriers as possible, including contribution and support guidelines.

Transparency is also an important aspect in order to create awareness among the internal community of what’s going on, in order to lower barriers and ease confusion which can easily arise in terms of communication and coordination.

9. Management support & motivation

Gaining top-level management support is important as the dedication of resources to shared assets may not be in the scope for certain teams, hence not a top priority or even possible due to budget constraints.

The shared assets need to be valued from a higher level in regards to the greater good for the company, and not just seen as cost centers, but rather as a (potential) source of profit (Riehle, Capraro, Kips, & Horn, 2016). Without this perspective, teams and business units can easily fall back and prioritize their own work and ignore the collaborative benefits of inner source, which can lead to trust issues and power-play where everyone is focused on pushing their own agenda.

Creating incentives for developers is equally important. This could be done using both intrinsic and extrinsic rewards, such as public recognition by management or by being considered in performance reviews. In a recent study, I did with a company it was highlighted that having ones’ project open sourced successfully should be equally rated as having a patent approved.

Advocating and evangelizing may further prove helpful in educating developers and managers about the potential benefits of inner source, and how to overcome perceived barriers. Appointing special ambassadors for the program throughout the organization may be one approach. Others could include social events such as town hall meetings and meet-ups combined with internal online marketing.

Summing up

What then are the biggest challenges with introducing inner source? For many, I would hypothesize it would a cultural shift that may be needed for the company on all levels. Encouraging and educating both developers and managers about the benefits, and how to overcome the many psychological and organizational barriers that sit in the company’s walls and in the minds of the employees. I believe this is mostly true for companies that are less used to working with open source and releasing their own projects.

As I noticed in a presentation I did recently, for those that have this mindset already, the challenge can be to set up the infrastructure needed to get a collective overview of all of the projects and shared assets. The solution should consider both accessibility and usability.

Rather than replacing existing tools, it should focus on bringing the different data sources together, such as source code repositories, issue trackers, and other communication channels. It should be easy enough so that lurkers can get a quick and truthful overview of the shared assets and decide if they want do dig deeper or move on.

Further reading

References

  1. Wesselius, J. (2008). The bazaar inside the cathedral: business models for internal markets. Software, IEEE, 25(3), 60–66.
  2. Stol, K.-J., Avgeriou, P., Babar, M. A., Lucas, Y., & Fitzgerald, B. (2014). Key factors for adopting inner source. ACM Transactions on Software Engineering and Methodology (TOSEM), 23(2), 18.
  3. Linåker, J., Krantz, M., & Höst, M. (2014). On Infrastructure for Facilitation of Inner Source in Small Development Teams. In Product-Focused Software Process Improvement (pp. 149–163). Springer.
  4. Kalliamvakou, E., Damian, D., Blincoe, K., Singer, L., & German, D. M. (2015). Open source-style collaborative development practices in commercial projects using github. In Proceedings of the 37th International Conference on Software Engineering-Volume 1 (pp. 574–585). IEEE Press.
  5. Riehle, D., Capraro, M., Kips, D., & Horn, L. (2016). Inner source in platform-based product engineering. IEEE Transactions on Software Engineering, 42(12), 1162–1177.