Mainly, there are two types of methods to create a bathymetric prediction using satellite imagery. Two of which are analytical methods and empirical methods. The former predicts depth using water body properties and then calculates depth using a formula and those properties as variable inputs. The latter predicts depth using depth training samples and fits the sample into a model and then predicts the depth using the model based on the depth sample training.
There are not many GUI (Graphical User Interface) based software to predict bathymetric depth using satellite imagery. Most of the software available are either commercial software or requires programming skills to use. In the age of automation, programming knowledge is a valuable skill to possess. However, not everyone has the time or the will to learn programming. This project aims to provide a GUI-based software that is easy to use and does not require programming skills to predict bathymetric depth using satellite imagery and depth samples.
Starting from version 4, the data processing and depth prediction processes are now moved from sdb_gui.py to a separate sdb package. This decision was made for two main reasons. The first is to simplify the code and make it easier to maintain. The second is to allow users to fully utilise the sdb package however they want, without the need to use the GUI.
This SDB project is using python and would not work without packages listed below:
| Packages | Description |
|---|---|
| numpy | The fundamental package for scientific computing with Python. It provides functions for tasks such as array operations, matrix operations, and random number generation. It is the most widely used Python package for numerical computing. |
| scipy | The SciPy library is one of the core packages that make up the SciPy stack. It provides functions for scientific and engineering applications. The SciPy library depends on NumPy, which provides support for large, multi-dimensional arrays and matrices, including a large collection of high-level mathematical functions to operate on these arrays. The SciPy library is built on top of the NumPy extension of the Python programming language. It adds functionality in several areas, including numerical integration, special functions, statistics, and optimization. |
| pandas | Used for data manipulation and analysis. It offers data structures and functions to efficiently handle structured data, including tabular data such as spreadsheets and SQL tables. |
| xarray | The project integrates the array-orientated features of NumPy with the labeling features of Pandas. It provides a powerful and flexible way of working with labeled, multidimensional arrays. |
| rioxarray | Rioxarray is a Python package that enables the use of rasterio for xarray's raster-based operations. It provides an optional dependency for xarray, allowing it to read and write raster formats supported by rasterio. |
| geopandas | Geopandas extends the datatypes used by pandas to allow spatial operations on geometric types. It provides tools to read, write, and process geospatial data, making it easier to work with geographic datasets in Python. |
| scikit-learn | A machine learning library for Python that features various classification, regression and clustering algorithms including support vector machines, random forests, gradient boosting, k-means and DBSCAN, and is designed to interoperate with the Python numerical and scientific libraries NumPy and SciPy. |
| joblib | A set of tools to provide lightweight pipelining in Python. It is particularly useful for running tasks in parallel and for caching the results of function calls. In this project, joblib is used to speed up the processing time by parallelizing the prediction process. |
| matplotlib | A plotting library for creating static, animated, and interactive visualizations in Python. |
| pyqt5 | Used to create the GUI of this software. It is a set of Python bindings for Nokia's Qt application framework and runs on all platforms supported by Qt including Windows, OS X, Linux, iOS and Android. |
| pyyaml | A YAML parser and emitter for Python. It is used to read and write YAML files, which are commonly used for configuration files. In this project, it is used to read the configuration file for the processing options. |
- Satellite Derived Bathymetry (SDB) GUI
Image below is the workflow of predicting bathymetric depth using SDB GUI if you're running the latest release and the latest source code or release version 3 and up.
You can download the latest release or clone the latest source code from sdb_gui by following the next instructions. If you are using the executable version, you can skip the python and packages installation steps and head to Data Preparation.
If you're downloading or cloning the source code, you need to have the packages from the table listed above installed in your preferred python environment.
Installing the packages can be a bit tricky and might cause some issues if not done properly. Therefore, it is highly recommended to use conda or pixi to set an environment and install the packages. If you want to use conda, you can install Miniconda or Miniforge. If you want to use pixi, you can follow the installation guide in pixi installation guide.
Clone sdb_gui repository and move to the root folder of the repository.
git clone https://github.com/rifqiharrys/sdb_gui.git
cd sdb_guiEnvironment setup and packages installation using conda is easy enough by using the environment.yaml file provided in this repository and running the command below.
conda env create -f environment.yamlEnvironment setup and packages installation using pixi is relying on the pixi.toml file provided in this repository. The pixi.toml file has been set up to provide 4 different environments for different purposes. The environments are:
defaultfor core functionality of SDB processing without GUIguifor SDB processing with GUI or SDB GUInotebookfor SDB processing using Jupyter Lab/Notebookexe-buildfor building the executable file using auto-py-to-exe
To setup the default environment and install the packages, you can run the command below.
pixi installIf you don't state the environment name just like the command above, pixi will install the packages of the default environment. To install the packages of your desired environment, you can run the command below by replacing <ENV_NAME> with the name of the environment you want to install.
pixi install -e <ENV_NAME>So, if you want to use SDB GUI, you can install the gui environment by running the command below.
pixi install -e guiIf you're using conda, you can run SDB GUI by activating the environment first and then running sdb_gui.py using python in the terminal and deactivate the environment after you're done using the command below.
conda activate sdb_gui
python sdb_gui.py
conda deactivateOr you could run sdb_gui.py without activating the environment by using conda run command as shown below.
conda run -n sdb_gui python sdb_gui.pyIf you're using pixi, you can run SDB GUI by activating the gui environment first, and then running sdb_gui.py using python in the terminal and deactivate the environment after you're done using the command below.
pixi shell -e gui
python sdb_gui.py
exitOr you could run sdb_gui.py without activating the environment by using pixi run command as shown below.
pixi run -e gui python sdb_gui.pyFor more information about environment management using conda and pixi, you can refer to their documentations at conda documentation and pixi documentation.
There are two types of data needed to use SDB GUI. They are:
- Georeferenced and corrected imagery in GeoTIFF format
- Depth samples in ESRI Shapefile format
The imagery required should be a multi-band imagery (e.g. RGB, RGBN, or others) in one stacked file. The depth samples doesn't have special requirements except if you want to process by attribute selection. In that case, the depth samples should have an additional field to differentiate the attributes which you want to select as training data, while the rest will marked as test data. Also, you need to understand the vertical reference and depth units of your depth samples because the results of prediction will be in the same units.
Open sdb_gui_x.x.x_one_file.exe if you're using the executable version or run sdb_gui.py by following these instructions if you're using the source code.
If you're using the executable version, you can open by double click on the file (sdb_gui_x.x.x_one_file.exe) or if you're using terminal, you can type sdb_gui_x.x.x_one_file.exe in the terminal in the same directory. If you're using the source code, run sdb_gui.py by using your preferred method as shown in the instructions above. Wait until SDB GUI opens and shows a window like the picture shown below. The preview of SDB GUI interface is available at SDB GUI Preview document.
Load your data into SDB GUI. When your data is successfully loaded, the GUI will show the file name beside the load buttons.
After loading depth sample data, you will notice a table loaded with said data and show the first 100 rows (or all depend on your chosen setting while loading data). Above the table are two selection about the loaded sample data. The first one is the header of the depth data, while the second one is the direction of the depth data.
Correctly selecting depth header and depth direction is important because it will allow SDB GUI to process the data correctly. The first selection is the column name or header name of the column of the depth data. The selection will show all of the header/column names of the sample data. The second selection will show two options, which are Positive Up and Positive Down. If your depth data decreases in values as it goes deeper, choose Positive Up because the positive values are going up, otherwise choose Positive Down.
The next parameters are depth limitation window for sample data input. There are two values for depth limit window, upper limit (default value is 2) and lower limit (default value is -15). Both values are in the Positive Up direction manner. You could disable depth limitation by checking the Disable Depth Limitation checkbox.
Next, select your desired regression method. There are three options to select, which are K-Nearest Neighbors, Multiple Linear Regression, and Random Forest. For every regression method, you could change its hyperparameters by clicking the Method Options button. The explanation of every hyperparameter is in scikit-learn user guide.
On the right of the Method Options button is the Processing Options button which contain options related to the overall process that are unrelated to the regression method. Leaving it as is would set the processing parameters using default values and settings.
Generate depth prediction by pressing Generate Prediction button. While processing occurs, some information will be displayed under Result Information section. After the process completed, there will be a pop up alert showing the process is done. Any information regarding the processing will be displayed under Result Information section too.
After depth prediction was generated, you can save it into a Geotiff or XYZ file. In the save file window, there are other options to use median filter to remove noise (default is on), save report, save train and test data, and create scatter plot using test data.
To have a better understanding about the new SDB processing workflow in SDB GUI project, you could read the Jupyter Notebook in this repository. There are two notebooks in this repository, which are sdb-how-to-xarray-workflow.ipynb and sdb-module-how-to.ipynb. Both notebooks contain a simple SDB processing workflow without GUI using Random Forest Regression. The workflow of the first notebook might be a bit different from the recent SDB GUI, the basic idea is the same and this notebook is a prototype to the recent update (v4) of SDB GUI. The second notebook contain SDB processing using SDB module.
Some issues that might occur based on user reports and personal testing are:
- incorrect depth data header selection
- incorrect selection of depth direction
SDB GUI might close unexpectedly due to issues above or other issues. The error messages might logged in sdb_gui_error.log or only shown in the terminal because exceptions are not handled properly. If you're running it using the excecutable version, it might help to run it using terminal to prevent the console from closing immediately after an error occurs.
If you have any questions, issues, or suggestions, please open an issue or a discussion in this repository or contact me via email at @rifqi.m.h, @rifqiharrys or what is displayed in my GitHub profile.
Please cite this project if you find this useful for your work or research using the metadata in the CITATION file.
See RELEASES
See LICENSE