Skip to content

Commit a3313e1

Browse files
dicortazardicortazar
committed
Add the inner source metrics chapter
1 parent 354119f commit a3313e1

File tree

9 files changed

+846
-0
lines changed

9 files changed

+846
-0
lines changed

‎SUMMARY.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# Summary
2+
3+
* [Summary](README.md)
4+
* [Introduction](introduction/introduction.md)
5+
* [What, When and How to Measure](measuring/introduction.md)
6+
* [Goals using Metrics](measuring/goals.md)
7+
* [Areas of Analysis](measuring/areas.md)
8+
* [Goal-Question-Metric Approach](measuring/gqm.md)
9+
* [Strategy](measuring/strategy.md)
10+
* [Examples of Interest](measuring/metrics.md)
11+
* [Mentorship and helping Newcomers](measuring/metrics.md#mentorships)
12+
* [Development Cycle](measuring/metrics.md#development)
13+
* [Contributors Funnel](measuring/metrics.md#funnel)
14+
* [Engagement](measuring/metrics.md#engagement)
15+
* [Break Silos](measuring/metrics.md#silos)
16+
* [Spreading the Knowledge](measuring/metrics.md#knowledge)
17+
* [Middle Management Cultural Change](measuring/metrics.md#cultural_change)
18+
* [Scalability](measuring/metrics.md#scalability)
19+
* [Keep great Developers in House](measuring/metrics.md#happy_devs)
20+
* [Allow Innovation in Detail](measuring/metrics.md#innovation)
21+
* [References](measuring/references.md)
22+
* [Authors and Reviewers](measuring/AUTHORS.md)

‎measuring/AUTHORS.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
Author(s) (Chronological order):
2+
3+
     Daniel Izquierdo Cortázar. Bitergia
4+
5+
Reviewer(s) (Chronological order).
6+
7+
    Gregorio Robles. Universidad Rey Juan Carlos.
8+
9+
    José Manrique López. Bitergia.
10+
11+
    Kate Stewart. The Linux Foundation.
12+

‎measuring/areas.md

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
Areas of Analysis
2+
=================
3+
4+
When analyzing software development projects, there are several areas
5+
that can be analyzed. Traditional analysis are focused on code metrics,
6+
while the activity, community and process are key components to take
7+
into account. This handbook brings into context those areas not that
8+
well studied, but key when inner sourcing: community and process.
9+
10+
Inner source is basically all about direct interactions between
11+
developers, removing all of the hierarchical aspects remaining in the
12+
middle. It is all about creating communities and building social
13+
networks that allow the knowledge to fly around. Inner source is about
14+
leaving developers to work with other developers (D2D) jumping over the
15+
traditional hierarchies. This helps to increase the development
16+
activity, break silos of information and allow innovation within the
17+
organization.
18+
19+
- **Activity**: this component focuses on the basics. Any developer
20+
when interacting with the infrastructure of the organization leaves
21+
traces of activity. From commits to emails, from tickets activity to
22+
code review actions. Any of these traces is potentially measurable
23+
and many of these traces from a lot of developers display an
24+
organization chart with trends information. Measuring activity means
25+
quantifying the basic traces left by the developers. And when this
26+
information is aggregated and calculated over time, it is possible
27+
to obtain trends about this type of information. Usual activity
28+
measurements can be the number of commits up to some date, but also
29+
the increase or decrease of such number of commits when comparing to
30+
timeframes. This can be extended to any potential *event* that may
31+
take place in a community of developers: emails, reviews, opening a
32+
ticket, closing a ticket, commenting on a code review or sending an
33+
email, but there are others more elaborated such as actions as
34+
mentor, newcomers activity, cross-referenced information like
35+
tickets referencing pull requests and those referencing commits,
36+
etc.
37+
38+
- **Community**: people are key when talking about inner source. If
39+
one of the goals consists of flattering the hierarchical structure
40+
of an organization, we need to measure how people interact with
41+
other developers. Although this is again measured through the traces
42+
left by the developers, the analysis proposed are pretty focused on
43+
the people and how they build their own social networks. As each
44+
developer has her own interests, it is necessary to understand how
45+
developers are attracted and retained in a project. What policies
46+
are more successful than others when attracting those and who are
47+
leaders in the software development process. Usual measurements in
48+
this case are related to the analysis of the contributors funnel
49+
coming to an internal project, effectiveness of specific policies to
50+
attract developers (e.g., hackathons) and others more specialized
51+
such as territoriality (how territorial a developer is) or
52+
betweenness if we play with some indicators when building networks.
53+
54+
- **Process**: thus we have a community of developers working together
55+
and leaving traces of their activity. But inner source is something
56+
more than spreading the knowledge across the organization. It is
57+
also a matter of reducing the time to market, increasing the
58+
velocity of development and building better products. That is the
59+
reason why process is also a needed area of analysis. Process can be
60+
defined as the bureaucracy followed by the developers to write a
61+
piece of code. In a more hierarchical structure, there might be some
62+
daily meetings where some decisions are taken and requirements
63+
implemented. That is process and if we want to improve that, we need
64+
to measure in first place key parameters of such process. In inner
65+
source communities, process is quite similar to open source
66+
communities where code style, documentation, code review process,
67+
continuous integration, design meetings are part of this process.
68+
The main difference here is how developers directly interact with
69+
other developers, even being from different business units, even
70+
been far away geographically. That interaction is the one we need to
71+
polish and improve if we want to increase the quality and velocity
72+
of the development process. Indeed the ideal goal when improving
73+
process is that the total process takes exactly 0 seconds. Once a
74+
piece of code comes out of the developers minds, this works, and can
75+
be automatically deployed. How can we reach that situation? I do not
76+
know, but hey!, at least we can measure that and check how far we
77+
are from heaven.
78+
79+
- **Code**: this aspect of the analysis is probably a well known one.
80+
There are usual metrics that determine the complexity of the code,
81+
modularity, test coverage, documentation coverage, and others. What
82+
we have when measuring this type of metrics is that we can control
83+
where our inner source method is leading those metrics. Increases in
84+
the code complexity or less test coverage may indicate unexpected
85+
behaviours that should be fixed and controlled.

‎measuring/goals.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
Goals using Metrics
2+
===================
3+
4+
This section aims at providing a strategy for your inner source metrics
5+
that help to understand the path from your initial process till a full
6+
inner source organization.
7+
8+
It is important to remark that metrics useful for some organizations are
9+
not that useful in other contexts. This is similar to the open source
10+
projects where a project is not that similar to other in terms of
11+
governance, licenses, infrastructure or detailed process, but they are
12+
producing open source software and working as a community. This handbook
13+
has a similar goal, to detail how an ideal inner source project would
14+
be, but there are not two organizations using the same inner source
15+
approach.
16+
17+
Thus, metrics useful for some contexts, for example technological
18+
organizations, might not apply to other context such as banks due to
19+
external factors such as legal regulations that are even different from
20+
country to country.
21+
22+
In addition to all of this, when measuring inner source, there are three
23+
main purposes to use metrics: check on going work, lead process
24+
improvement[^5] and motivational aspects.
25+
26+
- **Check** on going work: this helps to understand where the
27+
development is right now. To be aware of the status helps to
28+
understand how fast things are changing when a new process is in the
29+
pipeline. This also helps to go from A to B and even trying several
30+
approaches to the same problem and have tests for this.
31+
32+
- **Lead** process improvement: inner source means a change in how
33+
process works in the following. From a hierarchical way to a flatter
34+
way of working, inner sources needs indicators to help to determine
35+
if that process improvement is properly working. And if this is not
36+
working, then using another approach or apply other policies.
37+
38+
- **Motivational** aspects: inner source also means cultural change,
39+
and this is not usually taken into account in other methodologies.
40+
Indeed, this cultural change is identified in inner source as key.
41+
This should be the type of actions that will help to migrate from a
42+
traditional way of working to a more transparent and community-
43+
oriented way to work. And metrics can help to lead this process.
44+
First, to let developers know where they are and how their process
45+
is working, but also to have some fun within the work and
46+
competitions through challenges, hackathons and other measurable
47+
activities that lead to a more community-oriented organizations.

‎measuring/gqm.md

Lines changed: 97 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,97 @@
1+
Goal-Question-Metric Approach
2+
=============================
3+
4+
The Goal Question Metric approach defines a mechanism to provide an
5+
umbrella that helps to define and detailed software metrics. This can be
6+
easily extended to other backgrounds, although inner source method
7+
basically fits in this type of evaluation.
8+
9+
As stated in the original paper: “Any engineering process requires
10+
feedback and evaluation. Software development is an engineering
11+
discipline and measurement is an ideal mechanism for feedback and
12+
evaluation. The measurement and information... helps in the
13+
understanding and control of the software processes and products. It
14+
helps in making intelligent decision and improving over time. But
15+
measurement must be focused, based upon goals and models. We need to
16+
establish goals for the various software processes and products and
17+
these goals should be measurable, driven by the appropriate
18+
models.”[^6]. The GQM has been extensively used in academia and the
19+
industry as a method to detail metrics useful for a set of pre-defined
20+
goals.
21+
22+
In the specific case of inner source, and summarizing, the following can
23+
be seen as some of the most important goals for inner source:
24+
25+
- Goal 1. Improve code quality through CI and peer review
26+
27+
- Goal 2. Decrease time to market
28+
29+
- Goal 3. Allow innovation within developers
30+
31+
- Goal 4. Reduce development and maintenance costs
32+
33+
- Goal 5. Improve engagement within the organization
34+
35+
Then, each of the goals may have one or more questions that will answer
36+
those specific goals. And those are particularly related to the stage
37+
the process is. Although we may have the same goal when starting to
38+
apply inner source within an organization and when that method is fully
39+
established, the related questions are potentially different.
40+
41+
Let’s focus on Goal 5: improve engagement within the organization. We
42+
may start this process from a hierarchical organization where silos of
43+
developers exist and agile is used as their main methodology for
44+
software development. The specific question for a starting situation
45+
would be to understand if specific policies, such as opening the
46+
infrastructure, allowing public discussion, having design meetings and a
47+
long tail of activities fostering interactions among developers are
48+
actually working. On the other hand, in an inner-sourced organization,
49+
the question would be to keep improving the engagement or at least have
50+
a stable software development process with this respect.
51+
52+
**Starting from scratch**: Question 1: How many new attracted developers
53+
are participating in the inner sourced projects?
54+
55+
**Working on an inner-sourced community**: Question 1: Is the attraction
56+
of new developers improving over time? And this question leads to
57+
Question 2: Are we retaining the new developers? And this finally leads
58+
to Question 3: How long does it take for a developer to leave the
59+
community?
60+
61+
It is clear that starting from scratch needs to focus on the initial
62+
stages of the process improvement, while in a mature process, there are
63+
other questions of importance.
64+
65+
And finally metrics. These are linked to each of the questions and may
66+
answer some of them at the same time. If we go for the four questions,
67+
we may define the following metrics:
68+
69+
- Q1. How many new attracted developers are participating in the inner
70+
sourced projects?
71+
72+
- M1. Number of newcomers.
73+
74+
- M2. Relative number of newcomers out of the total number of
75+
developers.
76+
77+
- Q2. Is the attraction of new developers improving over time?
78+
79+
- M1. Number of newcomers.
80+
81+
- M2. Relative number of newcomers out of the total number of
82+
developers
83+
84+
- M3. Slope of the tendency of newcomers coming to the project
85+
86+
- Q3. Are we retaining the new developers?
87+
88+
- M4. Number of developers still developing that entered during
89+
the last year.
90+
91+
- M5. Relative number of retained developers out of the newcomers
92+
during the last year.
93+
94+
- Q4. How long does it take for a developer to leave the community?
95+
96+
- Median of the total time of all of the developers since their
97+
first commit till their last commit.

‎measuring/introduction.md

Lines changed: 107 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,107 @@
1+
**
2+
3+
Author(s) (Chronological order):
4+
5+
     Daniel Izquierdo Cortázar. Bitergia
6+
7+
Reviewer(s) (Chronological order).
8+
9+
    Gregorio Robles. Universidad Rey Juan Carlos.
10+
11+
    José Manrique López. Bitergia.
12+
13+
    Kate Stewart. The Linux Foundation.
14+
15+
16+
Introduction
17+
============
18+
19+
Choosing the right software metrics is not an easy task. If you have
20+
access to data, and in software development a lot of data can be
21+
gathered, you can easily think of many possible metrics. Sometimes, too
22+
many... up to the point of reaching saturation!
23+
24+
This report aims to formalize a strategy and a method to identify,
25+
acquire and understand metrics, and how to apply those to the inner
26+
source world. Part of the information found in this document is built on
27+
top of previous literature and talks about inner source. However, none
28+
of the former seems to focus especifically on the metrics needed to
29+
understand if a process is working as expected, or if substantial
30+
changes are required.
31+
32+
As inner source is a medium or long term activity, as any new
33+
methodology to be applied within a company, feedback about the process
34+
and performance is key to assure its quality. This report has as goal to
35+
be the *glue* between developers, managers and C-level. It aims to help
36+
them build a proper mining software structure based on key indicators
37+
that will help to lead the improvement of the process. And specifically
38+
in the case of inner sourcing, this document is expected to help when
39+
selecting initial metrics and studies to lead that process.
40+
41+
It is worth mentioning that the whole organizational structure should be
42+
aligned with the metrics used for the analysis. Qualitative feedback
43+
from the several layers of the organization is also a key part of this
44+
process, involving from developers to C-level and going through middle
45+
management. They all should understand that these tracking actions are
46+
following a specific goal and not tracking their own individual
47+
activities.
48+
49+
Rewarding systems on top of the metrics are also recommended, but always
50+
with a specific focus on fostering some actions such as pushing
51+
developers to commit their first pull request[^1]. The point about
52+
having metrics is that people can cheat on them, so when fostering
53+
specific behaviours, those should be use in short periods of time to
54+
help developers to get used to some way of developing. The most
55+
recommended use of metrics is to track the performance of the whole
56+
community and how to avoid bottlenecks and actions that may delay their
57+
activity.
58+
59+
The approach presented in this report follows the GQM approach
60+
(Goal-Question-Metric)[^2]. This consists of declaring the goal(s) of a
61+
specific decision, then state a set of questions that fit in that goal,
62+
and finally come back with a list of metrics that answer each of the
63+
proposed questions.
64+
65+
As Inner Source has a different meaning depending on the organization
66+
where it is deployed, inner source may have different goals. However,
67+
there are some goals that are usually required across organizations and
68+
that can be divided into these main groups[^3]:
69+
70+
- Improve **code quality** through continuous integration (CI). Open
71+
source projects are used to CI and peer review process. And
72+
specifically having many eyes to any piece of code going to master
73+
help a lot to detect potential issues in advance. It has been
74+
measured that having code review in your process helps up to the
75+
point of saving half of the potential spending when maintaining the
76+
software[^4]. On the other hand, CI allows to automate checks that
77+
were usually done by human beings, such as unit testing, regression
78+
tests, style checkers and others.
79+
80+
- Decrease **time to market**. As silos are broken within the
81+
organization, there are developers and business units across the
82+
organization that are willing to work on the same topic. And this
83+
helps to increase the velocity of the development process.
84+
85+
- Allow **innovation** within developers. Developers are now allowed
86+
to collaborate with others and create their own social networks with
87+
other business units. This collaboration helps to bring new points
88+
of view to the same problem what allows to bring innovation to the
89+
same problem. Developers can easily now create new repositories
90+
under some rules, having those projects as incubators and get
91+
traction from other developers that may participate in such project
92+
at some point. Do-ocracy could be also part of the process.
93+
94+
- Reduce development and maintenance **costs**. Maintenance is reduced
95+
to the decrease of business units working on the same topic and this
96+
cost is now shared among all of them. And this applies to the
97+
development and maintenance point of view. Any business unit
98+
interested in participating in a project as their goals are aligned
99+
to such project may provide resources.
100+
101+
- And last but not least, allowing developers to work on the topics
102+
they feel are important for the organization or on their own
103+
interest help them to be more comfortable. Inner source helps to
104+
improve the retention of the good developers for the organizations,
105+
but also helps in the recruitment process as the organization is
106+
seen as innovative and listen to developers needs. This helps to the
107+
general **engagement** in the organization.

0 commit comments

Comments
 (0)