Merge pull request lithops-cloud#1357 from JosepSampe/lithops-dev

Update docs

Author: JosepSampe

.gitignore

@@ -9,6 +9,7 @@ lithops_*.zip
 *.log
 *.txt
 *.csv
+*.coverage*
 
 # Virtual environments
 .env

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 ### Added
 - [CLI] Allow to pass a name in the "lithops runtime list" command
+- [Ceph] Added extra region parameter to Ceph backend
 
 ### Changed
 - [Setup] Removed unused 'lxml', 'docker' and 'python-dateutil' packages from the setup.py
@@ -16,6 +17,7 @@
 - [AWS Lambda] Fixed wrong AWS Lambda delete runtime_name match semantics 
 - [Worker] Fixed potential issue that can appear during 'func_obj' loading from cache
 - [Monitor] Fixed potential 'keyerror' exceptions
+- [Swift] Fixed OpenStack Swift parameters and authentication by adding domain information
 
 
 ## [v3.3.0]

README.md

@@ -195,4 +195,4 @@ If you are interested in contributing, see [CONTRIBUTING.md](./CONTRIBUTING.md).
 
 
 # Acknowledgements
-This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825184.
+This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 825184 (CloudButton).

config/README.md

@@ -117,11 +117,20 @@ Example of providing configuration keys for IBM Code Engine and IBM Cloud Object
 ```python
 import lithops
 
-config = {'lithops': {'backend': 'code_engine', 'storage': 'ibm_cos'},
-          'ibm': {'region': 'REGION',
-                  'iam_api_key': 'IAM_API_KEY',
-                  'resource_group_id': 'RESOURCE_GROUP_ID'},
-          'ibm_cos': {'storage_bucket': 'STORAGE_BUCKET'}}
+config = {
+    'lithops': {
+        'backend': 'code_engine',
+        'storage': 'ibm_cos'
+    },
+    'ibm': {
+        'region': 'REGION',
+        'iam_api_key': 'IAM_API_KEY',
+        'resource_group_id': 'RESOURCE_GROUP_ID'
+    },
+    'ibm_cos': {
+        'storage_bucket': 'STORAGE_BUCKET'
+    }
+}
 
 def hello_world(name):
     return f'Hello {name}!'
@@ -136,8 +145,8 @@ if __name__ == '__main__':
 
 |Group|Key|Default|Mandatory|Additional info|
 |---|---|---|---|---|
-|lithops | backend | aws_lambda | no | Compute backend implementation. IBM Cloud Functions is the default |
-|lithops | storage | aws_s3 | no | Storage backend implementation. IBM Cloud Object Storage is the default |
+|lithops | backend | aws_lambda | no | Compute backend implementation. `localhost` is the default if no config or config file is provided|
+|lithops | storage | aws_s3 | no | Storage backend implementation. `localhost` is the default if no config or config file is provided|
 |lithops | data_cleaner | True | no |If set to True, then the cleaner will automatically delete all the temporary data that was written into `storage_bucket/lithops.jobs`|
 |lithops | monitoring | storage | no | Monitoring system implementation. One of: **storage** or **rabbitmq** |
 |lithops | monitoring_interval | 2 | no | Monitoring check interval in seconds in case of **storage** monitoring |

docs/source/compute_config/aws_batch.md

@@ -14,13 +14,13 @@ python3 -m pip install lithops[aws]
 
 1. [Login](https://console.aws.amazon.com/?nc2=h_m_mc) to Amazon Web Services Console (or signup if you don't have an account)
 
-2. Navigate to **IAM > Roles** to create the ECS Task Execution Role. AWS provides a defualt role named `ecsTaskExecutionRole`, which can be used instead. If you want to create another role or it is missing, create a new role attached to `Elastic Container Service Task`, and add the following policies:
+2. Navigate to **IAM > Roles** to create the ECS Task Execution Role. AWS provides a default role named `ecsTaskExecutionRole`, which can be used instead. If you want to create another role or it is missing, create a new role attached to `Elastic Container Service Task`, and add the following policies:
     - `SecretsManagerReadWrite`
     - `AmazonEC2ContainerRegistryFullAccess`
     - `CloudWatchFullAccess`
     - `AmazonECSTaskExecutionRolePolicy`
 
-3. Navigate to **IAM > Roles** to create the ECS Instance Role. AWS provides a defualt role named `ecsInstanceRole`, which can be used instead. If you want to create another role or it is missing, create a new role attached to `EC2`, and add the following policy:
+3. Navigate to **IAM > Roles** to create the ECS Instance Role. AWS provides a default role named `ecsInstanceRole`, which can be used instead. If you want to create another role or it is missing, create a new role attached to `EC2`, and add the following policy:
     - `AmazonEC2ContainerServiceforEC2Role`
 
 ## AWS Credential setup

docs/source/compute_config/aws_ec2.md

@@ -77,7 +77,7 @@ In summary, you can use one of the following settings:
 |aws_ec2 | region | |yes | Region name of the VPC. For example `us-east-1`. Lithops will use the region set under the `aws` section if it is not set here |
 |aws_ec2 | ssh_username | ubuntu |no | Username to access the VM |
 |aws_ec2 | ssh_key_filename | ~/.ssh/id_rsa | no | Path to the ssh key file provided to create the VM. It will use the default path if not provided |
-|aws_ec2 | worker_processes | AUTO | no | Number of Lithops processes within a given worker. This is used to parallelize function activations within the worker. By default it detects the amount of CPUs in the VM|
+|aws_ec2 | worker_processes | AUTO | no | Number of parallel Lithops processes in a worker. This is used to parallelize function activations within the worker. By default it detects the amount of CPUs in the VM|
 |aws_ec2 | runtime | python3 | no | Runtime name to run the functions. Can be a container image name. If not set Lithops will use the defeuv python3 interpreter of the VM |
 |aws_ec2 | auto_dismantle | True |no | If False then the VM is not stopped automatically.|
 |aws_ec2 | soft_dismantle_timeout | 300 |no| Time in seconds to stop the VM instance after a job **completed** its execution |
@@ -151,7 +151,7 @@ In summary, you can use one of the following settings:
 |aws_ec2 | worker_instance_type | t2.medium | no | Profile name for the worker VMs |
 |aws_ec2 | delete_on_dismantle | True | no | Delete the worker VMs when they are stopped. Master VM is never deleted when stopped |
 |aws_ec2 | max_workers | 100 | no | Max number of workers per `FunctionExecutor()`|
-|aws_ec2 | worker_processes | 2 | no | Number of Lithops processes within a given worker. This can be used to parallelize function activations within a worker. It is recommendable to set this value to the same number of CPUs of a worker VM. |
+|aws_ec2 | worker_processes | AUTO | no | Number of parallel Lithops processes in a worker. This is used to parallelize function activations within the worker. By default it detects the amount of CPUs in the `worker_instance_type` VM|
 |aws_ec2 | runtime | python3 | no | Runtime name to run the functions. Can be a container image name. If not set Lithops will use the default python3 interpreter of the VM |
 |aws_ec2 | auto_dismantle | True |no | If False then the VM is not stopped automatically.|
 |aws_ec2 | soft_dismantle_timeout | 300 |no| Time in seconds to stop the VM instance after a job **completed** its execution |
@@ -176,7 +176,7 @@ lithops logs poll
 
 ## VM Management
 
-Lithops for AWS EC2 follows a Mater-Worker architecrue (1:N).
+Lithops for AWS EC2 follows a Mater-Worker architecture (1:N).
 
 All the VMs, including the master VM, are automatically stopped after a configurable timeout (see hard/soft dismantle timeouts).
 

docs/source/compute_config/aws_lambda.md

@@ -39,7 +39,7 @@ python3 -m pip install lithops[aws]
 }
 ```
 
-4. Click **Next: Tags** and **Next: Review**. Fill the policy name field (you can name it `lithops-policy` or simmilar) and create the policy.
+4. Click **Next: Tags** and **Next: Review**. Fill the policy name field (you can name it `lithops-policy` or similar) and create the policy.
 
 5. Go back to **IAM** and navigate to **Roles** tab. Click **Create role**.
 
@@ -105,7 +105,7 @@ In summary, you can use one of the following settings:
 | aws_lambda | architecture | x86_64 | no | Runtime architecture. One of **x86_64** or **arm64** |
 | aws_lambda | ephemeral_storage | 512 | no | Ephemeral storage (`/tmp`) size in MB (must be between 512 MB and 10240 MB) |
 | aws_lambda | env_vars | {} | no | List of {name: ..., value: ...} pairs for Lambda instance environment variables |
-| aws_lambda | namespace |  | no | Virtual namespace. This can be usefull to virtually group Lithops function workers. The functions deployed by lithops will be prefixed by this namespace. For example you can set it to differentiate between `prod`, `dev` and `stage` environments.  |
+| aws_lambda | namespace |  | no | Virtual namespace. This can be useful to virtually group Lithops function workers. The functions deployed by lithops will be prefixed by this namespace. For example you can set it to differentiate between `prod`, `dev` and `stage` environments.  |
 | aws_lambda | runtime_include_function | False | no | If set to true, Lithops will automatically build a new runtime, including the function's code, instead of transferring it through the storage backend at invocation time. This is useful when the function's code size is large (in the order of 10s of MB) and the code does not change frequently |
 
 

docs/source/compute_config/azure_containers.md

@@ -95,7 +95,7 @@ az containerapp env create --name lithops --resource-group LithopsResourceGroup
 |---|---|---|---|---|
 |azure_containers| resource_group | |no | Name of a resource group, for example: `LithopsResourceGroup`. Lithops will use the `resource_group` set under the `azure` section if it is not set here |
 |azure_containers| region |  |no | The location where you created the `lithops` Container APP environment. For example: `westeurope`, `westus2`, etc. Lithops will use the `region` set under the `azure` section if it is not set here|
-|azure_containers| environment | lithops |no | The environemnt name you created in the step 5 of the installation |
+|azure_containers| environment | lithops |no | The environment name you created in the step 5 of the installation |
 |azure_containers | docker_server | index.docker.io |no | Container registry URL |
 |azure_containers | docker_user | |no | Container registry user name |
 |azure_containers | docker_password | |no | Container registry password/token. In case of Docker hub, login to your docker hub account and generate a new access token [here](https://hub.docker.com/settings/security)|

docs/source/compute_config/azure_functions.md

@@ -89,7 +89,7 @@ az login
 |---|---|---|---|---|
 |azure_functions| resource_group | |no | Name of a resource group, for example: `LithopsResourceGroup`. Lithops will use the `resource_group` set under the `azure` section if it is not set here |
 |azure_functions| region |  |no | The location of the consumption plan for the runtime. Use `az functionapp list-consumption-locations` to view the available locations. For example: `westeurope`, `westus2`, etc. Lithops will use the `region` set under the `azure` section if it is not set here|
-|azure_functions | max_workers | 1000 | no | Max number of parallel workers. Altough Azure limits the number of workrs to 200, it is convenient to keep this value high|
+|azure_functions | max_workers | 1000 | no | Max number of parallel workers. Although Azure limits the number of workers to 200, it is convenient to keep this value high|
 |azure_functions | worker_processes | 1 | no | Number of Lithops processes within a given worker. This can be used to parallelize function activations within a worker |
 |azure_functions| runtime |  |no | Runtime name already deployed in the service|
 |azure_functions | runtime_timeout | 300 |no | Runtime timeout in seconds. Default 5 minutes |

docs/source/compute_config/azure_vms.md

@@ -89,7 +89,7 @@ Edit your lithops config and add the relevant keys:
 |azure_vms | ssh_username | ubuntu | yes | Username to access the VM. It will use `ubuntu` if not provided |
 |azure_vms | ssh_key_filename | ~/.ssh/id_rsa | yes | Path to the ssh key file provided to create the VM. It will use the default path if not provided |
 |azure_vms | region | |no | Location of the resource group, for example: `westeurope`, `westus2`, etc. Lithops will use the region set under the `azure` section if it is not set here |
-|azure_vms | worker_processes | AUTO | no | Number of Lithops processes within a given worker. This is used to parallelize function activations within the worker. By default it detects the amount of CPUs in the VM|
+|azure_vms | worker_processes | AUTO | no | Number of parallel Lithops processes in a worker. This is used to parallelize function activations within the worker. By default it detects the amount of CPUs in the VM|
 |azure_vms | runtime | python3 | no | Runtime name to run the functions. Can be a container image name. If not set Lithops will use the defeuv python3 interpreter of the VM |
 |azure_vms | auto_dismantle | True |no | If False then the VM is not stopped automatically.|
 |azure_vms | soft_dismantle_timeout | 300 |no| Time in seconds to stop the VM instance after a job **completed** its execution |
@@ -139,12 +139,12 @@ Edit your lithops config and add the relevant keys:
 |azure_vms | worker_instance_type | Standard_B2s | no | Profile name for the worker VMs |
 |azure_vms | delete_on_dismantle | False | no | Delete the worker VMs when they are stopped. Master VM is never deleted when stopped. `True` is NOT YET SUPPORTED |
 |azure_vms | max_workers | 100 | no | Max number of workers per `FunctionExecutor()`|
-|azure_vms | worker_processes | 2 | no | Number of Lithops processes within a given worker. This can be used to parallelize function activations within a worker. It is recommendable to set this value to the same number of CPUs of a worker VM. |
+|azure_vms | worker_processes | AUTO | no | Number of parallel Lithops processes in a worker. This is used to parallelize function activations within the worker. By default it detects the amount of CPUs in the `worker_instance_type` VM|
 |azure_vms | runtime | python3 | no | Runtime name to run the functions. Can be a container image name. If not set Lithops will use the default python3 interpreter of the VM |
 |azure_vms | auto_dismantle | True |no | If False then the VM is not stopped automatically.|
 |azure_vms | soft_dismantle_timeout | 300 |no| Time in seconds to stop the VM instance after a job **completed** its execution |
 |azure_vms | hard_dismantle_timeout | 3600 | no | Time in seconds to stop the VM instance after a job **started** its execution |
-|azure_vms | exec_mode | reuse | no | One of: **consume**, **create** or **reuse**. If set to  **create**, Lithops will automatically create new VMs for each map() call based on the number of elements in iterdata. If set to **reuse** will try to reuse running workers if exist |
+|azure_vms | exec_mode | reuse | no | One of: **consume**, **create** or **reuse**. If set to  **create**, Lithops will automatically create new VMs for each map() call based on the number of elements in `iterdata`. If set to **reuse** will try to reuse running workers if exist |
 
 
 ## Test Lithops
@@ -164,7 +164,7 @@ lithops logs poll
 
 ## VM Management
 
-Lithops for Azure VMs follows a Mater-Worker architecrue (1:N).
+Lithops for Azure VMs follows a Mater-Worker architecture (1:N).
 
 All the VMs, including the master VM, are automatically stopped after a configurable timeout (see hard/soft dismantle timeouts).