Fix documentaion version libraries. Add how_to_use section

Author: pockerman

.readthedocs.yaml

@@ -18,9 +18,9 @@ build:
       - doxygen
       - pip install pydot
       - pip install sphinx==5.0.2
-      - pip install breathe
-      - pip install -U exhale
-      - pip install sphinx_rtd_theme
+      - pip install breathe==4.35.0
+      - pip install -U exhale==0.3.7
+      - pip install sphinx_rtd_theme==3.0.2
 
 
 # Build documentation in the "docs/" directory with Sphinx

docs/conf.py

@@ -99,7 +99,7 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'alabaster'
+html_theme = 'default'
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,

docs/how_to_use.rst

@@ -0,0 +1,141 @@
+How to use 
+=============
+
+The following is an example how to use the 
+``FrozenLake``   environment from <a href="https://github.com/Farama-Foundation/Gymnasium/tree/main">Gymnasium</a>.
+
+.. code-block::
+
+	#include "rlenvs/rlenvs_types_v2.h"
+	#include "rlenvs/envs/gymnasium/toy_text/frozen_lake_env.h"
+	#include "rlenvs/envs/api_server/apiserver.h"
+
+	#include <iostream>
+	#include <string>
+	#include <unordered_map>
+	#include <any>
+
+	namespace example_1{
+
+	const std::string SERVER_URL = "http://0.0.0.0:8001/api";
+
+	using rlenvscpp::envs::gymnasium::FrozenLake;
+	using rlenvscpp::envs::RESTApiServerWrapper;
+
+
+	void test_frozen_lake(const RESTApiServerWrapper& server){
+
+		FrozenLake<4> env(server);
+
+		std::cout<<"Environame URL: "<<env.get_url()<<std::endl;
+
+		// make the environment
+		std::unordered_map<std::string, std::any> options;
+		options.insert({"is_slippery", false});
+		env.make("v1", options);
+
+		std::cout<<"Is environment created? "<<env.is_created()<<std::endl;
+		std::cout<<"Is environment alive? "<<env.is_alive()<<std::endl;
+		std::cout<<"Number of valid actions? "<<env.n_actions()<<std::endl;
+		std::cout<<"Number of states? "<<env.n_states()<<std::endl;
+
+		// reset the environment
+		auto time_step = env.reset(42, std::unordered_map<std::string, std::any>());
+
+		std::cout<<"Reward on reset: "<<time_step.reward()<<std::endl;
+		std::cout<<"Observation on reset: "<<time_step.observation()<<std::endl;
+		std::cout<<"Is terminal state: "<<time_step.done()<<std::endl;
+
+		//...print the time_step
+		std::cout<<time_step<<std::endl;
+
+		// take an action in the environment
+		// 2 = RIGHT
+		auto new_time_step = env.step(2);
+
+		std::cout<<new_time_step<<std::endl;
+
+		// get the dynamics of the environment for the given state and action
+		auto state = 0;
+		auto action = 1;
+		auto dynamics = env.p(state, action);
+
+		std::cout<<"Dynamics for state="<<state<<" and action="<<action<<std::endl;
+
+		for(auto item:dynamics){
+
+			std::cout<<std::get<0>(item)<<std::endl;
+			std::cout<<std::get<1>(item)<<std::endl;
+			std::cout<<std::get<2>(item)<<std::endl;
+			std::cout<<std::get<3>(item)<<std::endl;
+		}
+		
+		action = env.sample_action();
+		new_time_step = env.step(action);
+
+		std::cout<<new_time_step<<std::endl;
+		
+		// synchronize the environment
+		env.sync(std::unordered_map<std::string, std::any>());
+		
+		auto copy_env = env.make_copy(1);
+		copy_env.reset();
+		
+		std::cout<<"Org env cidx: "<<env.cidx()<<std::endl;
+		std::cout<<"Copy env cidx: "<<copy_env.cidx()<<std::endl;
+		
+		copy_env.close();
+
+		// close the environment
+		env.close();
+
+	}
+
+	}
+
+
+	int main(){
+
+		using namespace example_1;
+		
+		RESTApiServerWrapper server(SERVER_URL, true);
+
+		std::cout<<"Testing FrozenLake..."<<std::endl;
+		example_1::test_frozen_lake(server);
+		std::cout<<"===================="<<std::endl;
+		return 0;
+	}
+
+
+
+
+In general, the environments exposed by the library  follow the semantics in <a href="https://github.com/deepmind/dm_env/blob/master/docs/index.md">Environment API and Semantics</a> specification.
+For more details see the <a href="doc/env_spec.md">```rlenvscpp``` environment specification</a> document.
+
+The general use case is to build the library and link it with your driver code to access its functionality.
+The environments specified as using REST in the tables above, that is all ```Gymnasium```, ```gym_pybullet_drones``` and ```GymWalk``` 
+environments are accessed via a client/server pattern. Namely, they are exposed via an API developed using 
+<a href="https://fastapi.tiangolo.com/">FastAPI</a>.
+You need to fire up the FastAPI server, see dependencies, before using the environments in your code. 
+To do so
+
+``
+./start_uvicorn.sh
+``
+
+By default the ```uvicorn``` server listents on port 8001. Change this if needed. You can access the OpenAPI specification at
+
+``
+http://0.0.0.0:8001/docs
+``
+
+Note that currently the implementation is not thread/process safe i.e. if multiple threads/processes access the environment
+a global instance of the environment is manipulated. Thus no session based environment exists.
+However, you can create copies of the same environment and access this via its dedicate index.
+If just one thread/process touches this specific environment you should be ok.
+Notice that the FastAPI server only uses a single process to manage all the environments.
+In addition, if  you need multiple instances of the same environment you can also  use one 
+of the exissting vectorised environments (see table above).
+
+Finally,   you can choose to launch several instances of ```uvirocrn``` (listening on different ports). 
+However in this case you need to implement all the interactions logic yourself as currently no implementation exists to handle such a scenario.

docs/index.rst

@@ -3,14 +3,15 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-```rlenvscpp``` documentation
+``rlenvscpp`` documentation
 =====================================
 
 .. toctree::
    :maxdepth: 2
    :caption: Contents:
 
    overview
+   how_to_use
    installation
    api/library_root
 

docs/installation.rst

@@ -1,2 +1,88 @@
 Installation
-------------
+============
+
+This section describes how to install ``rlenvscpp``.
+
+Dependencies
+------------
+
+The library has the following general dependencies
+
+- A compiler that supports C++20 e.g. g++-11
+- `Boost C++ <https://www.boost.org/>`_ 
+- `CMake >= 3.10 <https://cmake.org/>`_
+- `Gtest  <https://github.com/google/googletest>`_  (if configured with tests)
+- `Eigen3 <https://eigen.tuxfamily.org/index.php?title=Main_Page>`_
+
+Using the Gymnasium environments requires `Gymnasium <https://github.com/Farama-Foundation/Gymnasium/tree/main>`_ 
+installed on your machine.
+In addition, you need to install
+
+- `FastAPI <https://fastapi.tiangolo.com/>`_
+- `Uvicorn <https://www.uvicorn.org/>`_
+- `Pydantic <a href="https://docs.pydantic.dev/latest/>`_
+
+By installing the requirement under ``requirements.txt`` should set your Python environment  up correctly.
+
+``rlenvscpp`` also incorporates, see ``(src/extern)``, the following libraries
+
+- `HTTPRequest <https://github.com/elnormous/HTTPRequest>`_
+- `nlohmann/json <https://github.com/nlohmann/json>`_
+
+
+There are extra dependencies if you want to generate the documentation. Namely,
+
+- Doxygen
+- Sphinx
+- sphinx_rtd_theme
+- breathe
+- m2r2
+
+The usual CMake based installation process is used. Namely
+
+.. code-block::
+
+	mkdir build && cd build && cmake ..
+	make install
+
+
+You can toggle the following variables
+
+- CMAKE_BUILD_TYPE (default is RELEASE)
+- ENABLE_TESTS_FLAG (default is OFF)
+- ENABLE_EXAMPLES_FLAG (default is OFF)
+- ENABLE_DOC_FLAG (default is OFF)
+
+For example enbling the examples 
+
+.. code-block::
+
+	cmake -DENABLE_EXAMPLES_FLAG=ON ..
+	make install
+
+
+
+Run the tests
+-------------
+
+You can execute all the tests by running the helper script ``execute_tests.sh``.
+
+
+Issues
+-------
+
+**Could not find ``boost_system``**
+
+It is likely that you are missing the boost_system library with your local Boost installation. This may be the case
+is you installed boost via a package manager. On a Ubuntu machine the following should resolve the issue
+
+.. code-block::
+
+	sudo apt-get update -y
+	sudo apt-get install -y libboost-system-dev
+
+
+**FastAPI throws 422 Unpocessable entity**
+
+Typically, this is a problem with how the client (400-range error) specified the data
+to be sent to the server.