Skip to content

Commit 69a621d

Browse files
spierspier
authored
Merge pull request #50 from InnerSourceCommons/split-infrastructure.md
Split up infrastructure.md into 3 files
2 parents f2ecb16 + 584be30 commit 69a621d

File tree

5 files changed

+287
-301
lines changed

5 files changed

+287
-301
lines changed

‎SUMMARY.md

Lines changed: 2 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,8 @@
66
* [Framework](introduction/framework.md)
77
* [Authors and Reviewers](introduction/authors.md)
88
* [Infrastructure](infrastructure/infrastructure.md)
9-
* [Basic Infrastructure](infrastructure/infrastructure.md#basic-infrastructure)
10-
* [Development Process](infrastructure/infrastructure.md#development-process-infrastructure)
11-
* [Communication Channels](infrastructure/infrastructure.md#communication-channels-infrastructure)
12-
* [Monitoring](infrastructure/infrastructure.md#monitoring-infrastructure)
13-
* [Comparing How Inner-Sourced Your Infra Is](infrastructure/infrastructure.md#comparing-how-inner-sourced-your-infrastructure-is)
9+
* [Basic Infrastructure](infrastructure/basics.md.md)
10+
* [Comparing How Inner-Sourced Your Infra Is](infrastructure/assessment.md)
1411
* [Authors and Reviewers](infrastructure/authors.md)
1512
* [What, When and How to Measure](measuring/introduction.md)
1613
* [Goals using Metrics](measuring/goals.md)

‎infrastructure/assessment.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Comparing How InnerSourced your Infrastructure is
2+
3+
Just detailing the infrastructure needed within an organization to effectively
4+
apply InnerSource would be simplistic. This section aims at listing the
5+
questions you need to ask to your infrastructure team to check if
6+
that internal and well known infrastructure is able to be part of the
7+
InnerSource process.
8+
9+
The goal of this section is to compare the internal infrastructure used within
10+
an organization and check how close this is to an ideal InnerSource toolchain.
11+
As detail, in the software development process, it is necessary the use of
12+
specific tools such as the versioning system, code review process, ticketing
13+
system, continuous integration, documentation storage and collaborative
14+
platform to share technical decisions.
15+
16+
For each of those, we need to check if they are following the key aspects
17+
provided such as openness.
18+
19+
## Development Process Infrastructure
20+
21+
| | Openness | Transparency | Archivable | Searchable | Monitoring | Access Rights |
22+
|---------------------|----------|--------------|------------|------------|------------|---------------|
23+
| Versioning | | | | | | |
24+
| Ticketing system | | | | | | |
25+
| Code Review | | | | | | |
26+
| CI | | | | | | |
27+
| Wiki/Documentation | | | | | | |
28+
| TODO List | | | | | | |
29+
| Collaborative notes | | | | | | |
30+
31+
## Communication Channels Infrastructure
32+
33+
| | Openness | Transparency | Archivable | Searchable | Monitoring | Access Rights |
34+
|---------------------|----------|--------------|------------|------------|------------|---------------|
35+
| Mailing lists/forums| | | | | | |
36+
| Instant channels | | | | | | |
37+
| Questions/Answers | | | | | | |
38+
39+
## Monitoring Infrastructure
40+
41+
| | Openness | Transparency | Archivable | Searchable | Monitoring | Access Rights |
42+
|-------------------------|----------|--------------|------------|------------|------------|---------------|
43+
| Retrieval platform | | | | | | |
44+
| Enrichment platform | | | | | | |
45+
| Visualization platform | | | | | | |
46+
47+
# Some Examples of Infrastructure
48+
49+
GitHub enterprise.
50+
GitLab.
51+
In house repositories.
52+
Atlasian stack.

‎infrastructure/authors.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
1+
# Authors and Reviewers
2+
13
## Authors
24

35
(Chronological order)
46

5-
- Daniel Izquierdo ([Bitergia](http://bitergia.com))
7+
- Daniel Izquierdo ([Bitergia](http://bitergia.com))
68

79
## Reviewers
810

‎infrastructure/basics.md

Lines changed: 230 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,230 @@
1+
# Basic Infrastructure
2+
3+
As InnerSource is mainly about cultural change, we need to have an easy-access
4+
and low barrier tools. The easier to use, the more developers that will try
5+
in first place to work with other business units and inner-sourced projects.
6+
7+
Although there is a code review process and this takes time to learn, there are
8+
other areas where developers can start to contribute. From documentation and
9+
mere typos in the collaborative wiki to design meetings and even review
10+
activities in projects of your interest or asking for feature requests. There
11+
is a myriad of potential actions that anyone within the organization can help
12+
with. And the goal of InnerSource is to foster those actions as much as possible
13+
letting developers know that those actions are really much appreciated.
14+
15+
The infrastructure is thus divided into three main areas:
16+
17+
* In first place the development process infrastructure that contains
18+
the basic tooling for developers.
19+
Selecting the right tools will help to have a clear process and that process
20+
will bring trustiness to the community. Any developer must follow that process
21+
and work-arounds should not exist. As an example, any developer, even core
22+
or trusted committers should face a review process when submitting a piece
23+
of code. It is clear that trusted committers have a reputation in the community
24+
and this will help in the review process, but they still need to go through
25+
the process.
26+
A clear workflow brings trust across the business units. That certainty in the
27+
definition of requirements, software development process, use of versioning or
28+
ticketing systems, the code review process and the continuous integration helps
29+
with this.
30+
31+
* In second place a solid use of the communication channels infrastructure.
32+
These tools should be as transparent as possible and any technical meeting must
33+
be followed by a summary of results and decisions in the mailing list. This helps
34+
to open technical discussions, but also to reference to previous decisions
35+
made.
36+
As we are considering large organizations, it is also necessary the use of
37+
asynchronous communication channels such as the usual IRC in open source
38+
communities. More advanced options could be the use of [Slack](https://slack.com) but also
39+
[Mattermost](https://mattermost.com) if the organization prefers to use
40+
open source and in house SaaS deployments.
41+
42+
* In third place the monitoring infrastructure is key when applying InnerSource and
43+
in general when bringing a new methodology to organizations. This is one of the
44+
main differences with open source communities. They are open by default and basically
45+
follow the detailed key aspects. However, infrastructure to measure process advances
46+
have not been one of the main goals in the case of open source communities.
47+
Basically they are using a successful development methodology, each of them
48+
with their own peculiarities, but open by default.
49+
InnerSource needs of this type of infrastructure as managers and developers
50+
need feedback about their performance. A change in the software development
51+
process of large organizations, a cultural change and the community building
52+
process needs a large set of actions and those actions should have the
53+
confirmation that they are working. For this the organization and its
54+
business units need of a monitoring infrastructure.
55+
56+
## Development Process Infrastructure
57+
58+
When developing there are three main tools to take into account: the versioning
59+
, code review and continuous integration systems. Those should follow a process
60+
similar to the one depicted in the following picture. If this process
61+
is familiar to you is because this is based on the [OpenStack software development
62+
process](https://docs.openstack.org/infra/manual/developers.html) as detailed in their wiki site.
63+
I have copied the workflow as this contains the basic pieces also needed for
64+
InnerSource. Other communities use a similar approach, although I did not find
65+
a nice picture! Sorry folks!. In addition to this, this is a new figure as
66+
I wanted to have it independent of the infrastructure. OpenStack uses
67+
Git, Gerrit and other tooling for this process, but others are also possible.
68+
As an example the Linux Kernel uses mailing lists for the code review process
69+
or the Mozilla community that uses another toolchain. Think of the figure
70+
as a generic way to introduce code review and continuous integration aspects
71+
in the software development process.
72+
73+
In short, the process in the figure is as follows: the developer should clone the repository(1)
74+
make changes to it (2), run some local tests (3), pull request those changes (4),
75+
tests will be run (5) with a specific result (6). If that result is negative,
76+
then we need to go through a new version of the code and come back to the
77+
local environment (7: Review process -> Updated Copy of Upstream). If the
78+
CI works and there is a positive answer from the Review Process, then this go
79+
again through CI before being committed to master (8). If the review process
80+
provides a negative evaluation then the piece of code goes back again
81+
to the submitter (7: Review process -> Updated Copy of Upstream).
82+
83+
![Usual software development process](development_workflow.jpg)
84+
85+
* **Versioning system**: this tool is used by developers to store the
86+
several iterations of a given piece of software. As developers are basically
87+
geographically distributed, the versioning system should allow this type of
88+
interactions, where any developer at any time may submit a piece of code
89+
to be reviewed. Systems that allow off-line development are highly
90+
recommended as developers will be able to locally work and later submit the code.
91+
92+
* **Code review system**: once the piece of code is ready to be submitted,
93+
this should be previously reviewed by another developer. This forces developers
94+
to submit that piece of code through a specific process. As an example,
95+
there are several ways where open source communities code review others, using
96+
specific tools, sending the piece of code to a mailing list or integrated
97+
in the versioning system tool. As one of the main goals when inner sourcing
98+
is to keep the process as simple as possible, the main recommendation is to
99+
avoid too noise channels (as mailing lists) or too hard to use tools.
100+
It is also recommended to use tools that help others to start developing a new
101+
piece of source code without needing to submit that to review. Early discussions
102+
in the code review process helps to produce better code and having mentors
103+
involved in the process.
104+
105+
* **Continuous integration (CI) system**: this is one of the key tooling when
106+
developing. There are already several eyes having a look at the source code
107+
in the code review process. With the addition of a continuous integration
108+
platform, any type of test should be covered: regression, unit testing, end
109+
user tests, etc. Ideally this platform should be integrated with the code
110+
review process. In this way, developers can wait for the answer for the CI
111+
system before proceeding with the code review process. They would be sure
112+
that this works prior any effort from them.
113+
114+
* **Ticketing system**: tickets are useful to attract community to an
115+
InnerSource project. This helps in two specific ways: transparency of the
116+
development process, raising issues and having a roadmap of the issues
117+
to be closed. And in second place, to provide a platform for newcomers and
118+
users to detail their needs. Tickets are helpful to bring community in inner
119+
source projects as users of the platform will open bug reports, but also
120+
feature requests. And even those can socialized as the community can vote
121+
those reports and declare what are the most important for them. This information
122+
is key to let developers know about the community and business units needs.
123+
Then all of this can be discussed during the design summits defining further
124+
roadmaps based on users, developers and organizations requirements.
125+
126+
* **Documentation system**: documentation is now available to any member
127+
of the organization. And documentation has extra goals when producing it. Not
128+
only to developers, but to users. Indeed the documentation should be focused
129+
on several roles. From developers to users, the documentation should cover
130+
their needs. And as such, documentation should be transparent and open to
131+
any potential change from members. This will help to adequate the documentation
132+
to the users needs, but also to other members within the organization.
133+
The tool used should allow to have all of these pieces of information. From the
134+
usual developers APIs to high level users interested in understanding what
135+
that piece of code offers to the organization. It is worth mentioning that
136+
the documentation also covers information as general as the mission and
137+
the type of things that the piece of code does and the things that this does
138+
not do.
139+
140+
* **Collaborative design platform**: InnerSource in large organizations
141+
is a synonym of geographically distributed teams. Face to face meetings are
142+
hard to have in this type of organizations, but there should exist infrastructure
143+
to bridge those difficulties. Requirements specifications, technical decisions,
144+
TODOs lists and others should be stored in this type of collaborative environments.
145+
This will provide transparency to the process, but also informal documentation
146+
and communication. Even when the developers are in face to face meetings,
147+
those tools should be used as they will leave traces of activity readable
148+
by others within the organization.
149+
150+
![Extended usual software development process](development_workflow_all.jpg)
151+
152+
## Communication Channels Infrastructure
153+
154+
InnerSource is about cultural change. And that cultural change is based on
155+
transparency and meritocracy. Communication channels should be open within
156+
the organization, and anyone is allowed to post to them.
157+
158+
Any decision out of the public channels should be later written down in these
159+
as any decision should be traceable and referenceable.
160+
161+
* **Mailing Lists / Forums**: this asynchronous way of communicating across the
162+
developer teams is highly effective. Being geographically distributed force
163+
the members of the organization to avoid direct communication channels
164+
when possible as people lives in different time zones.
165+
166+
* **Instant Messaging**: this is another asynchronous communication channel.
167+
From the usual IRC channels used in open source software, to other open
168+
source options such as Mattermost, this helps to lead technical discussions,
169+
store the log information and have all of the developers in a virtual room
170+
where they can discuss, but also users can enter looking for advice.
171+
172+
* **Questions / Answers**: this type of platforms help to raise questions and
173+
share those with the rest of the community. Users and developers can vote
174+
the most interesting ones and this helps to bring attention to issues of interest
175+
for the internal inner-sourced community.
176+
177+
* **Video conference**: face to face meeting definitively helps. And even more
178+
when discussing about technical issues. This type of synchronous communication
179+
channels are useful for discussions but force people to be at the same time
180+
in the same virtual room. As there could be members from several time zones,
181+
those are more difficult to set than conversations in the instant messaging
182+
or mailing lists.
183+
184+
## Monitoring Infrastructure
185+
186+
This infrastructure is needed to understand the current situation of the
187+
software development process and should help in the decision making process.
188+
One of the key aspects when choosing a specific toolchain is that this is
189+
_data retrieval friendly_. This means that any tool is now a data source
190+
and such data source should provide a way to mine this. This will provide
191+
raw data information that should be later parsed and treated to be useful.
192+
193+
There is extra information in the metrics chapter, but in brief, any organization
194+
applying InnerSource, or any other new methodology, should have a way to
195+
check how that new process is performing when compared to the old way.
196+
197+
This monitoring infrastructure should also have as outcomes several ways of
198+
accessing such information. And this depends on the role accessing the information.
199+
Although everyone in the organization may have access to the information
200+
released by the development and communication channels toolchain, not all of them
201+
will be interested in accessing the raw data or the high level and quarterly reports
202+
focused on C-level.
203+
204+
For this the monitoring platform should provide access to the raw data retrieved
205+
from all of the data sources, to the enriched data after removing inconsistencies,
206+
and the final outcomes as dashboards, KPIs or quarterly reports. Indeed, having
207+
actionable data will help the members of the community to align those outcomes
208+
to their specific needs.
209+
210+
The following is a potential architecture that could help when accessing
211+
the several data layers, from raw information to detailed visualizations.
212+
213+
![Monitoring Infrastructure](monitoring_infrastructure.jpg)
214+
215+
* **Retrieval Platform**: this first part uses as input any of the data
216+
sources already mentioned. Version systems, mailing lists, tickets,
217+
collaborative documents and others should have some way of retrieving
218+
the information those contain. There may be some tools that produce events
219+
and those events are not stored anymore. This platform should take care of
220+
these cases. As an example of temporary logs, if there is not a policy to take
221+
care of the Apache logs or the jobs run by a Jenkins instance, there is not a
222+
way to retrieve old datasets.
223+
224+
* **Enrichment Platform**: this part of the data analysis focuses on cleaning
225+
and preparing the data for visualization.
226+
227+
* **Visualization Platform**: this third part of the monitoring platform should
228+
help to produce the outcomes needed to track the status of the methodology
229+
process. By visualization, this platform produces actionable charts, but
230+
also KPIs or static documents to be shared with third parties.

0 commit comments

Comments
 (0)