-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathdetaching-resources-architecture-foundation.html
More file actions
267 lines (218 loc) · 12.1 KB
/
detaching-resources-architecture-foundation.html
File metadata and controls
267 lines (218 loc) · 12.1 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js lt-ie9 lt-ie8 lt-ie7"> <![endif]-->
<!--[if IE 7]> <html class="no-js lt-ie9 lt-ie8"> <![endif]-->
<!--[if IE 8]> <html class="no-js lt-ie9"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js"> <!--<![endif]-->
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title>Detaching Resources: Architecture foundation ← Vinicius Horewicz</title>
<meta name="viewport" content="width=device-width">
<meta name="author" content="Vinicius Horewicz (wicz)">
<meta name="description" content="wicz' personal notes">
<link rel="alternate" type="application/atom+xml" href="/atom.xml" title="RSS feed" />
<link rel="stylesheet" href="/assets/stylesheets/normalize.css">
<link rel="stylesheet" href="/assets/stylesheets/monokai.css">
<link rel="stylesheet" href="/assets/stylesheets/main.css">
<link rel="stylesheet" href="/assets/stylesheets/screen.css">
<link href="//netdna.bootstrapcdn.com/font-awesome/4.0.0/css/font-awesome.css" rel="stylesheet">
<link href="//fonts.googleapis.com/css?family=Roboto:400,300,300italic|Roboto+Slab:300|Ubuntu+Mono" rel="stylesheet" type="text/css">
<script src="/assets/javascripts/modernizr-2.6.2.min.js"></script>
</head>
<body>
<div class="container">
<p><a href="/">← index</a></p>
<section class="post-content">
<header>
<h1>Detaching Resources: Architecture foundation</h1>
<time datetime="2013-10-21T00:00:00+02:00" pubdate>21 October 2013</time>
</header>
<article>
<p><em>Detaching Resources is as series about my personal findings and
experiments on building web applications in a service-oriented fashion.
The title and content are based on “Backing Services” from the
<a href="http://12factor.net/">Twelve-Factor App</a> methodology.</em></p>
<h2 id="a-quick-introduction-on-virtualization">A Quick Introduction on Virtualization</h2>
<p><a href="https://en.wikipedia.org/wiki/Virtualization">Virtualization</a>, as the
name suggests, is the act of creating a virtual version of something. It
can be used to create virtual hardware, operating system, memory (swap),
network (VPN), storage (RAID), etc. We will focus on the first two.</p>
<p>Hardware virtualization is probably the most known type. You certainly
have heard of virtual machines (VM) and softwares like Parallels, VMware
and VirtualBox. It allows you run, for instance, Linux inside Mac OS X.
Since it creates an entire virtual computer, it can become very
heavyweight, CPU and memory hungry.</p>
<p><a href="https://en.wikipedia.org/wiki/Operating_system-level_virtualization">Operating system (OS)
virtualization</a>
acts in the OS kernel layer. It creates lightweight virtual environments
(VE or containers), because they use the host OS’ system call. There is
no need of an intermediate VM; and still being completely isolated. On
the other hand, it is not as flexible as the hardware virtualization,
since the guest and host OS must be the same.
<a href="http://linux-vserver.org">Linux-VServer</a>, <a href="http://openvz.org/">OpenVZ</a>
and <a href="http://linuxcontainers.org/">Linux Containers (LXC)</a> are examples
of implementations of this technology.</p>
<p>LXC is the newest implementation of OS virtualization in Linux. It is
based on the <a href="https://en.wikipedia.org/wiki/Cgroups">Linux kernel
cgroups</a> (control groups) to
limit, account and isolate hardware resources of process groups. LXC is
considered as something between a “chroot on steroids” and a full VM.
Its goal is to create an environment as close as possible to a standard
Linux installation, but without the need of a separate kernel.</p>
<p><a href="http://www.docker.io/">Docker</a> provides a high level API on top of LXC.
It automates the creation of containers based on a Dockerfile, making it
repeatable and consistent. It allows container versioning with git-like
capabilities. Allows reuse. Any container can be used as a base image to
create new ones. It encourages sharing. Anyone can upload a container to
their public registry. The
<a href="https://github.com/dotcloud/docker-registry">registry</a> can also be used
in your private servers.</p>
<h2 id="hands-on-docker">Hands on Docker</h2>
<p>To install Docker, check its
<a href="http://docs.docker.io/en/latest/">documentation</a> for your OS. It is
comprehensive and easy to follow.</p>
<p>If you are using a Mac OS X, you need Linux running in a VM. Docker
already makes our lives easier by supporting Vagrant. All you need is to
install <a href="https://www.virtualbox.org/">VirtualBox</a> and
<a href="http://www.vagrantup.com/">Vagrant</a> and then</p>
<div class="highlight"><pre><code class="language-console"><span class="gp">$</span> git clone https://github.com/dotcloud/docker.git
<span class="gp">$</span> <span class="nb">cd </span>docker
<span class="gp">$</span> vagrant up
<span class="gp">$</span> vagrant ssh</code></pre></div>
<p>The docker daemon should be already running as root. Since the user
<code>vagrant</code> is in the group <code>docker</code>, it is not necessary to run the
<code>docker</code> CLI with sudo.</p>
<p>To get the basics on Docker, I suggest you follow their <a href="http://www.docker.io/gettingstarted/#h_tutorial">interactive
tutorial</a>. You will
learn how to <code>pull</code> container images from the <a href="https://index.docker.io/">index
repository</a>, <code>run</code> the container, <code>commit</code>
your changes and <code>push</code> it back to the repository. I will skip straight
to the use of a <code>Dockerfile</code>, where things get more interesting.</p>
<h2 id="the-dockerfile">The Dockerfile</h2>
<p>The <code>Dockerfile</code> contains the set of
<a href="http://docs.docker.io/en/latest/use/builder/">steps</a> docker will
use to reproduce the system images.</p>
<div class="highlight"><pre><code># Dockerfile
FROM ubuntu:latest
MAINTAINER Vinicius Horewicz <vinicius@horewi.cz>
RUN sed -i.bak "s/main$/main universe/" /etc/apt/sources.list
RUN apt-get update
RUN apt-get upgrade -y
RUN locale-gen en_US en_US.UTF-8
RUN echo "root:root" | chpasswd
# initctl workaround
RUN dpkg-divert --local --rename --add /sbin/initctl
RUN ln -s /bin/true /sbin/initctl
# prevent services from starting automatically
RUN echo exit 101 > /usr/sbin/policy-rc.d
RUN chmod +x /usr/sbin/policy-rc.d
RUN apt-get install -y curl monit openssh-server rabbitmq-server
# disable upstart for SSH
RUN mv /etc/init/ssh.conf /etc/init/ssh.conf.dist
ADD ./etc/monit/sshd.conf /etc/monit/conf.d/sshd.conf
ADD ./etc/monit/rabbitmq.conf /etc/monit/conf.d/rabbitmq.conf
# run monit in foreground
CMD ["/usr/bin/monit", "-I"]
EXPOSE 22 :5672
</code></pre></div>
<p>Dockerfiles follow the format <code>INSTRUCTION arguments</code>. <strong>The first
instruction must be</strong> <code>FROM</code>. It specifies the base image from which we
will build.</p>
<p>Images names are in the format <code>user/image:tag</code>. Where <code>user</code> is the
username of its creator [in the repository]. When it is not specified,
means you are using official images from the Docker team. <code>image</code> is the
image name. Images can be tagged, just like git. That is the reason of
the <code>tag</code> argument. When omitted, the value <code>latest</code> is assumed.</p>
<p><code>RUN</code> instruction runs a command and commits the result at the build
time. Every <code>RUN</code> means a new image is committed, which serves as a base
for the next instruction. Thanks to the this
<a href="http://docs.docker.io/en/latest/terms/layer/">layered</a> approach and to
<a href="http://aufs.sourceforge.net/">AuFS</a>, Docker can cache build steps,
making them blazing fast.</p>
<p><code>ADD</code> commands will copy files from <code><src></code> to <code><dest></code>. Source can be a
file or directory relative to the source directory being built or a
remote file URL. Destination is the path in the container. <code>ADD</code> is not
cached (<a href="https://github.com/dotcloud/docker/issues/880">yet?</a>).</p>
<p><code>CMD</code> specifies the command to be executed when the container is started.
<strong>There can exist only one in a Dockerfile. If there are more than one,
only the last will take effect.</strong></p>
<p><code>EXPOSE</code> sets port to be publicly exposed when running the container. It
is specified as <code>[PUBLIC:]PRIVATE</code>. When PUBLIC is omitted, a random port
is assigned. If you want to use the same port for public and private,
you can specify only <code>:PRIVATE</code>.</p>
<h2 id="running-the-container">Running the Container</h2>
<p>You start the container with the <code>docker run</code> command. It will run the
specified <code>CMD</code>. <strong>When this process completes, the container is fully
stopped.</strong> Think about containers as “process in a box”. This is why we
must run the <code>CMD</code> in foreground instead of using a daemon or running
<code>/etc/init.d/monit start</code> in <code>CMD</code>.</p>
<p>One might ask: Why monit? Remember a container run only one process and
stops right after this process ends. Even though a container can
<a href="https://github.com/dotcloud/docker/issues/223">start</a>
<a href="https://github.com/dotcloud/docker/pull/898">with</a>
<a href="https://github.com/dotcloud/docker/pull/1267">/sbin/init</a>, docker still
has some <a href="https://github.com/dotcloud/docker/issues/2276">issues with
upstart</a>, so we will use
monit to start and keep our process running.</p>
<p>We still have to do
<a href="https://github.com/dotcloud/docker/issues/1024#issuecomment-20018600">some</a>
<a href="http://jpetazzo.github.io/2013/10/06/policy-rc-d-do-not-start-services-automatically/">workarounds</a>
and disable upstart for some services to be able to use init.d scripts.</p>
<h2 id="the-final-result">The Final Result</h2>
<p>The final result is available in this
<a href="https://github.com/wicz/detaching-resources/tree/master/v1">repository</a>.
I have created basic monitrc files for SSH and RabbitMQ, so monit can
start these services. If you want to give it a try:</p>
<div class="highlight"><pre><code class="language-console"><span class="gp">$</span> git clone https://github.com/wicz/detaching-resources.git
<span class="gp">$</span> <span class="nb">cd </span>detaching-resources/v1/docker-rabbitmq
<span class="gp">$</span> docker build -t <yourusername>/rabbitmq .
<span class="gp">$</span> docker run -t <yourusername>/rabbitmq</code></pre></div>
<p>Finally, we have set up the architecture foundation. In the next post of
the series we will start implementing services which will exchange
messages through this message queue container.</p>
</article>
</section>
<!-- Enable Disqus comments -->
<div id="disqus_thread"></div>
<script type="text/javascript">
var disqus_shortname = 'horewicz';
var disqus_identifier = '/detaching-resources-architecture-foundation';
var disqus_url = 'http://horewi.cz/detaching-resources-architecture-foundation.html';
(function() {
var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
})();
</script>
<footer>
<a href="about.html">
<i class="fa fa-info-circle fa-fw fa-lg"></i>
</a>
<a href="https://www.linkedin.com/in/viniciushorewicz">
<i class="fa fa-l1nkedin fa-fw fa-lg"></i>
</a>
<a href="https://github.com/wicz">
<i class="fa fa-github fa-fw fa-lg"></i>
</a>
<a href="https://twitter.com/wicz">
<i class="fa fa-tw1tter fa-fw fa-lg"></i>
</a>
<a href="https://stackoverflow.com/users/3243455/wicz">
<i class="fa fa-stack-overflow fa-fw fa-lg"></i>
</a>
<a href="/assets/wicz.pub.asc">
<i class="fa fa-lock fa-fw fa-lg"></i>
</a>
<a href="/atom.xml">
<i class="fa fa-rss fa-fw fa-lg"></i>
</a>
</footer>
</div>
<script type="text/javascript">
var _gaq=[['_setAccount','UA-8480243-3'],['_trackPageview']];
(function(d,t){var g=d.createElement(t),s=d.getElementsByTagName(t)[0];
g.src=('https:'==location.protocol?'//ssl':'//www')+'.google-analytics.com/ga.js';
s.parentNode.insertBefore(g,s)}(document,'script'));
</script>
</body>
</html>