11# BiBiGrid
2+
23BiBiGrid is a framework for creating and managing cloud clusters, currently supporting OpenStack.
34Future versions will support additional cloud providers.
45
5- BiBiGrid uses Ansible to configure standard Ubuntu 20.04/22.04 LTS as
6- well as Debian 11 cloud images. Depending on your configuration BiBiGrid
7- can set up an HCP cluster for grid computing (Slurm Workload Manager,
8- a shared filesystem (on local discs and attached volumes),
6+ BiBiGrid uses Ansible to configure standard Ubuntu 22.04/24.04 LTS cloud images. Depending on your configuration BiBiGrid
7+ can set up an HCP cluster for grid computing (Slurm Workload Manager,
8+ a shared filesystem (on local discs and attached volumes),
99a cloud IDE for writing, running and debugging (Theia Web IDE) and many more.
1010
11-
1211> ** Note**
1312> The latest version is currently work in progress. Future changes are likely.
1413> Not all features of the previous version are available, but they will come soon.
1514> The [ previous version] ( https://github.com/BiBiServ/bibigrid/tree/bibigrid-2.3.1 ) is still available,
1615> but not maintained anymore.
1716
1817## Getting Started
18+
1919For most users the [ Hands-On BiBiGrid Tutorial] ( https://github.com/deNBI/bibigrid_clum2022 )
2020is the best entry point.
2121
@@ -26,7 +26,9 @@ might be just what you need.
2626<summary > Brief, technical BiBiGrid overview </summary >
2727
2828### How to configure a cluster?
29+
2930#### Configuration File: bibigrid.yaml
31+
3032A [ template] ( bibigrid.yaml ) file is included in the repository ([ bibigrid.yaml] ( bibigrid.yaml ) ).
3133
3234
@@ -39,6 +41,7 @@ The configuration template [bibigrid.yaml](bibigrid.yaml) contains many helpful
3941[ You need more details?] ( documentation/markdown/features/configuration.md )
4042
4143#### Cloud Specification Data: clouds.yaml
44+
4245To access the cloud, authentication information is required.
4346You can download your ` clouds.yaml ` from OpenStack.
4447
@@ -47,10 +50,12 @@ Place the `clouds.yaml` file in the `~/.config/bibigrid/` directory. BiBiGrid wi
4750[ You need more details?] ( documentation/markdown/features/cloud_specification_data.md )
4851
4952### Quick First Time Usage
53+
5054If you haven't used BiBiGrid1 in the past or are unfamiliar with OpenStack, we heavily recommend following the
5155[ tutorial] ( https://github.com/deNBI/bibigrid_clum2022 ) instead.
5256
5357#### Preparation
58+
54591 . Download (or create) your ` clouds.yaml ` file (and optionally ` clouds-public.yaml ` ) as described [ above] ( #cloud-specification-data-cloudsyaml ) .
55602 . Place the ` clouds.yaml ` into ` ~/.config/bibigrid `
56613 . Fill in the ` bibigrid.yaml ` configuration file with your specifics. At a minimum you need to specify: a master instance with valid type and image,
@@ -63,13 +68,15 @@ See [here](https://www.akamai.com/blog/developers/how-building-virtual-python-en
63686 . Take a look at [ First execution] ( #first-execution )
6469
6570#### First execution
71+
6672Before proceeding, ensure you have completed the steps described in the [ Preparation section] ( #preparation ) .
6773
6874After cloning the repository, navigate to the bibigrid directory.
6975Source the virtual environment created during [ preparation] ( #preparation ) to execute BiBiGrid.
7076Refer to BiBiGrid's [ Command Line Interface documentation] ( documentation/markdown/features/CLI.md ) if you want to explore additional options.
7177
7278A first execution run through could be:
79+
73801 . ` ./bibigrid.sh -i [path-to-bibigrid.yaml] -ch ` : checks the configuration
74812 . ` ./bibigrid.sh -i 'bibigrid.yaml -i [path-to-bibigrid.yaml] -c' ` : creates the cluster (execute only if check was successful)
75823 . Use ** BiBiGrid's create output** to investigate the created cluster further. Especially connecting to the ide might be helpful.
@@ -85,6 +92,7 @@ Great! You've just started and terminated your first cluster using BiBiGrid!
8592</details >
8693
8794### Troubleshooting
95+
8896If your cluster doesn't start up, first ensure your configuration file is valid using the ` -ch ` option.
8997If the configuration is invalid, modify the file as needed.
9098Use the ` -v ` or ` -vv ` options for more verbose output to help identify the issue faster.
@@ -95,29 +103,34 @@ Additionally, manually check if your quotas are exceeded, as some quotas cannot
95103** Whenever you contact a developer, please send your logfile along.**
96104
97105# Documentation
106+
98107For more information about BiBiGrid, please visit the following links:
108+
99109- [ BiBiGrid Features] ( documentation/markdown/bibigrid_feature_list.md )
100110- [ Software used by BiBiGrid] ( documentation/markdown/bibigrid_software_list.md )
101111
102112<details >
103113<summary > Differences to old Java BiBiGrid</summary >
104114
105- * BiBiGrid no longer uses RC- but cloud.yaml-files for cloud-specification data. Environment variables are no longer used (or supported).
115+ - BiBiGrid no longer uses RC- but cloud.yaml-files for cloud-specification data. Environment variables are no longer used (or supported).
106116See [ Cloud Specification Data] ( documentation/markdown/features/cloud_specification_data.md ) .
107- * BiBiGrid has a largely reworked configurations file, because BiBiGrid core supports multiple providers this step was necessary.
117+ - BiBiGrid has a largely reworked configurations file, because BiBiGrid core supports multiple providers this step was necessary.
108118See [ Configuration] ( documentation/markdown/features/configuration.md )
109- * BiBiGrid currently only implements the provider OpenStack.
110- * BiBiGrid only starts the master and will dynamically start workers using slurm when they are needed.
119+ - BiBiGrid currently only implements the provider OpenStack.
120+ - BiBiGrid only starts the master and will dynamically start workers using slurm when they are needed.
111121Workers are powered down once they are not used for a longer period.
112- * BiBiGrid lays the foundation for clusters that are spread over multiple providers, but Hybrid Clouds aren't fully implemented yet.
122+ - BiBiGrid lays the foundation for clusters that are spread over multiple providers, but Hybrid Clouds aren't fully implemented yet.
123+
113124</details >
114125
115126# Development
127+
116128## Development-Guidelines
117129
118130[ https://github.com/BiBiServ/Development-Guidelines ] ( https://github.com/BiBiServ/Development-Guidelines )
119131
120132## On implementing concrete providers
133+
121134Implementing new cloud providers is straightforward.
122135Copy the ` provider.py ` file and implement all necessary methods for your cloud provider.
123136Inherit from the ` provider ` class.
0 commit comments