|
| 1 | +--- |
| 2 | +title: CernVM's fruitful Summer - GSoC 2011 |
| 3 | +layout: plain |
| 4 | +year: 2011 |
| 5 | +--- |
| 6 | +{:refdef: style="text-align: center;"} |
| 7 | + |
| 8 | +{: refdef} |
| 9 | + |
| 10 | +This was the first year [CERN](http://public.web.cern.ch/public/) |
| 11 | +participated in *[Google Summer of Code](http://code.google.com/soc/)*, |
| 12 | +and it turned out to be an amazing experience for us! We were given four |
| 13 | +students to mentor, all of whom proved to be very skilled developers. |
| 14 | +The students quickly familiarized themselves with our code base and |
| 15 | +managed to make valuable contributions within the three month time frame |
| 16 | +of *Google Summer of Code*. Our students were very open and willing to |
| 17 | +learn and spent a considerable amount of their time researching tools, |
| 18 | +libraries, and the latest technological developments. As a result, all |
| 19 | +four students were able to solve their problems and come up with |
| 20 | +interesting ideas for future development. The code and the documentation |
| 21 | +they produced is available |
| 22 | +[here](http://code.google.com/p/google-summer-of-code-2011-cernvm/downloads/list). |
| 23 | +The specific problems (projects) that we suggested to our students |
| 24 | +spanned several domains, ranging from consistent replication of |
| 25 | +terabytes of data across several remote sites to automated testing of |
| 26 | +virtual machine releases. |
| 27 | + |
| 28 | +**Josip Lisec** was working on the development of the monitoring system |
| 29 | +for the [CernVM](http://cernvm.cern.ch/portal/) Co-Pilot framework, |
| 30 | +which is mainly used as a distributed computing platform within the |
| 31 | +LHC\@home 2.0 volunteer computing project. The [LHC\@home |
| 32 | +2.0](http://lhcathome2.cern.ch/) project currently has more than 9,000 |
| 33 | +registered users who contribute their spare CPU cycles for the |
| 34 | +simulation of the particle collision events in CERN\'s [Large Hadron |
| 35 | +Collider](https://home.cern/science/accelerators/large-hadron-collider) (LHC). |
| 36 | +After some research, Josip decided to integrate existing tools with the |
| 37 | +Co-Pilot as opposed to trying to reinvent the wheel by rewriting |
| 38 | +everything from scratch. This resulted in a nicely engineered monitoring |
| 39 | +framework, parts of which were put into production while the *Google |
| 40 | +Summer of Code* was still going on (Josip\'s developments have now been |
| 41 | +fully integrated after completion of the program). Since this was |
| 42 | +Josip\'s first encounter with Perl, he has been seen adding support for |
| 43 | +\'[my](http://perldoc.perl.org/functions/my.html)\' keyword to every |
| 44 | +other major programming language since the *Google Summer of Code* |
| 45 | +concluded.\ |
| 46 | +\ |
| 47 | +The goal of **Yin Qiu**\'s project was to devise a mechanism for a |
| 48 | +consistent replication of changes made to the central repository of |
| 49 | +[CernVM File System](http://cernvm.cern.ch) |
| 50 | +(CernVM-FS) to a globally distributed set of mirror servers. CernVM-FS |
| 51 | +is used to host and distribute the application software of CERN LHC |
| 52 | +experiments to hundreds of Grid sites, as well as the laptops and |
| 53 | +workstations of users worldwide. As such, it is currently one of the |
| 54 | +central components of the distributed computing infrastructures on which |
| 55 | +CERN [ATLAS](http://www.atlas.ch/) and |
| 56 | +[LHCb](http://user.web.cern.ch/public/en/LHC/LHCb-en.html) experiments |
| 57 | +rely. Yin\'s approach was to organize CernVM-FS mirrors into a |
| 58 | +Paxos-managed replication network and to enforce state machine version |
| 59 | +transitions on them. Following the suggestion of Jakob, his mentor, Yin |
| 60 | +implemented a messaging framework which is used to orchestrate the |
| 61 | +replication process and facilitates the implementation of new features. |
| 62 | +He also managed to implement a couple of Python plugins which ensure the |
| 63 | +consistency of data across replicas. The project is currently in the |
| 64 | +state of a working prototype.\ |
| 65 | +\ |
| 66 | +**Jesse Williamson** took up the challenge of designing a new library |
| 67 | +for CernVM-FS to consolidate support for various cryptographic hashing |
| 68 | +algorithms. The first task was to survey the implementation of CernVM-FS |
| 69 | +and establish a list of requirements. Next, quite a bit of effort was |
| 70 | +spent on designing the library specifically so that it would be easy to |
| 71 | +use, comparatively simple to extend, and robust enough to support |
| 72 | +extensions like a streaming interface and compression. Since CernVM-FS |
| 73 | +is heavily used in production, it has been very important to make sure |
| 74 | +that the new developments do not break anything. Jesse has developed a |
| 75 | +set of unit tests which ensure that all the existing features and |
| 76 | +properties were maintained.\ |
| 77 | +\ |
| 78 | +The design of new C++ libraries was certainly an improvement, but it |
| 79 | +also became clear late in the cycle that a further abstraction to fully |
| 80 | +separate digests and hash functions will be necessary to avoid memory |
| 81 | +fragmentation issues and ensure stronger const-correctness\ |
| 82 | +\ |
| 83 | +**Jonathan Gillet** worked on implementing a solution for automating the |
| 84 | +testing of CernVM virtual machine images on multiple hypervisors and |
| 85 | +operating systems. The solution, which is a ready to use testing |
| 86 | +infrastructure for CernVM, was developed in collaboration with other |
| 87 | +open source projects such as AMD |
| 88 | +Tapper (used for the reports and web interface), [libvirt](http://libvirt.org/) |
| 89 | +(interaction with hypervisors), and |
| 90 | +Homebrew (OS X support). The main |
| 91 | +goals of the project were accomplished with support for all major |
| 92 | +hypervisors running on Linux and OS X platforms. The framework automates |
| 93 | +the task of downloading and configuring the CernVM images on the fly, |
| 94 | +and executing a series of thorough tests which check various features of |
| 95 | +CernVM images before release. Documentation was also an important goal |
| 96 | +of the project; in total there are now over two hundred pages of |
| 97 | +documentation which cover everything from setting up the testing |
| 98 | +infrastructure and virtual machines to a complete API reference.\ |
| 99 | +\ |
| 100 | +We certainly enjoyed *Google Summer of Code* 2011, and we sincerely |
| 101 | +congratulate all of our students and mentors for successfully completing |
| 102 | +the program!\ |
| 103 | +\ |
| 104 | +*By Artem Harutyunyan, Senior Fellow, CernVM Project (CERN) and Google |
| 105 | +Summer of Code Mentor*\ |
| 106 | +[(original article on Google Open Source |
| 107 | +Blog)](http://google-opensource.blogspot.fr/2011/12/cernvms-fruitful-summer.html) |
0 commit comments