Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docs-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ jobs:
pip install -r requirements.txt -r requirements_dev.txt
pip install .
- name: Discover typos with codespell
run: codespell --skip="*.csv,*.geojson,*.json,*.js,*.html,*cff,*.pdf,./.git" --ignore-words-list="aci,acount,hist"
run: codespell --skip="*.csv,*.geojson,*.json,*.js,*.html,*cff,*.pdf,*.bib,./.git" --ignore-words-list="aci,acount,hist"
- name: PKG-TEST
run: |
python -m unittest discover tests/
Expand Down
23 changes: 23 additions & 0 deletions .github/workflows/draft-pdf.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Draft PDF
on: [push]

jobs:
paper:
runs-on: ubuntu-latest
name: Paper Draft
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Build draft PDF
uses: openjournals/openjournals-draft-action@master
with:
journal: joss
# path to paper file
paper-path: paper/paper.md

- name: Upload PDF
uses: actions/upload-artifact@v4
with:
name: paper
path: paper/paper.pdf
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ repos:
args:
[
"--ignore-words-list=aci,acount,acounts,fallow,ges,hart,hist,nd,ned,ois,wqs,watermask,tre,mape",
"--skip=*.csv,*.geojson,*.json,*.yml*.js,*.html,*cff,*.pdf",
"--skip=*.csv,*.geojson,*.json,*.yml*.js,*.html,*cff,*.pdf*,*.bib",
]

- repo: https://github.com/kynan/nbstripout
Expand Down
122 changes: 118 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,123 @@ For a complete list of examples and use cases, visit the [notebooks](https://git

## Key Features

- Extracting long-term time series data from GEE for both point and polygon geometries.
- Extract image patches from satellite imagery in GEE to support local-scale computer vision model training.
- Quickly transform complex multivariate datasets into a few principal components while preserving critical information.
- Easy implementation of Harmonic Regression on vegetation or climate indices.
* **Long-term Time Series Extraction** — Retrieve satellite or climate time series from Google Earth Engine (GEE) for both point and polygon geometries.
* **Image Patch Generation** — Extract image tiles or patches from satellite imagery in GEE to support local-scale computer vision or deep learning model training.
* **Multivariate Dimensionality Reduction** — Transform complex multivariate datasets into a few principal components while preserving essential information using PCA.
* **Harmonic Regression Analysis** — Easily perform harmonic regression on vegetation or climate indices to study seasonal and periodic trends.
* **Cloud-free Time Series Creation** — Generate regular, gap-filled, and cloud-free time series from irregular satellite observations.
* **Phenology and Smoothing Tools** — Apply smoothing algorithms and extract phenological metrics (Start of Season, Peak of Season, End of Season) from high-resolution satellite data.

---

## Installation
```bash
conda create -n geeagri python=3.10
conda activate geeagri
pip install geeagri
# (Optional) Upgrade to the latest version if already installed
pip install --upgrade geeagri
```

---

## Example Usage

#### Example 1: Extract timeseries to point
```python
import ee
import geeagri
from geeagri.extract import extract_timeseries_to_point

# Authenticate and initialize the Earth Engine API
ee.Authenticate()
ee.Initialize()

# Define point location (longitude, latitude)
lon, lat = -98.15, 30.50
point = ee.Geometry.Point([lon, lat])

# Load ERA5-Land daily aggregated climate dataset
era5_land = ee.ImageCollection("ECMWF/ERA5_LAND/DAILY_AGGR")

# Extract daily temperature, precipitation, and solar radiation time series
era5_land_point_ts = extract_timeseries_to_point(
lat=lat,
lon=lon,
image_collection=era5_land,
start_date="2020-01-01",
end_date="2021-01-01",
band_names=[
"temperature_2m_min",
"temperature_2m_max",
"total_precipitation_sum",
"surface_solar_radiation_downwards_sum",
],
scale=11132, # spatial resolution in meters (~11 km)
)
```
This example demonstrates how to use `geeagri` to extract daily climate variable time series (temperature, precipitation, and solar radiation) from the **ERA5-Land** dataset at a specific geographic point using the Google Earth Engine (GEE) Python API.

**Output Plot:**
![ERA5-Land Temperature Time Series](paper/figure.png)

#### Example 2: Create regular satellite timeseries
```python
import ee
from geeagri.preprocessing import Sentinel2CloudMask, RegularTimeseries

# Authenticate and initialize the Earth Engine API
ee.Authenticate()
ee.Initialize()

# Define the bounding box
bbox = [-98.451233, 38.430732, -98.274765, 38.523996]
region = ee.Geometry.BBox(*bbox)

# Get cloud masked Sentinel-2 image collection
s2_cloud_masker = Sentinel2CloudMask(
region=region,
start_date="2020-01-01",
end_date="2021-01-01",
cloud_filter=60,
cloud_prob_threshold=50,
nir_dark_threshold=0.15,
shadow_proj_dist=1,
buffer=50
)

s2_cloud_masked = s2_cloud_masker.get_cloudfree_collection()

# Calculate NDVI
def calculateNDVI(image):
ndvi = image.expression(
"(NIR - Red) / (NIR + Red)",
{"NIR": image.select("B8"), "Red": image.select("B4")},
).copyProperties(image, ["system:time_start"])

ndvi = ee.Image(ndvi).rename("ndvi").clip(region)

return ndvi


ndvi_col = s2_cloud_masked.map(calculateNDVI)

# Instantiate a 'RegularTimeseries' object
reg_timeseries = RegularTimeseries(
image_collection=ndvi_col,
interval=5, # Interval (in days) between consecutive target dates
window=45, # Temporal window in days
)

# Get the regular timeseries
ndvi_regular = reg_timeseries.get_regular_timeseries()
```
This example demonstrates how to extract a regular, gap-filled NDVI time series from Sentinel-2 imagery using `geeagri`. First, a `Sentinel2CloudMask` object is created to mask clouds and shadows over a defined bounding box using thresholds for cloud probability, dark NIR pixels, and shadow projection. The cloud-masked images are then processed with a custom function to calculate NDVI for each image. Finally, a `RegularTimeseries` object generates a temporally consistent NDVI time series at a specified interval and temporal window. This workflow allows users to efficiently obtain high-quality, regular NDVI time series from raw Sentinel-2 imagery while handling cloud and shadow contamination.

**Raw NDVI Time Series with Cloud Gaps:**
![Raw Satellite Time Series with Cloud Gaps](docs/assets/ndvi_raw.gif)

**Regular Gap-filled NDVI Time Series:**
![Regular Gap-filled NDVI Time Series](docs/assets/ndvi_regular.gif)

---
Binary file added docs/assets/ndvi_raw.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/assets/ndvi_regular.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
81 changes: 8 additions & 73 deletions docs/examples/land_surface_phenology.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"https://githubtocolab.com/geonextgis/geeagri/blob/main/docs/examples/image_scaling.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"/></a>\n",
"<a href=\"https://githubtocolab.com/geonextgis/geeagri/blob/main/docs/examples/land_surface_phenology.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open in Colab\"/></a>\n",
"\n",
"Uncomment the following line to install the latest version of [geeagri](https://geonextgis.github.io/geeagri) if needed."
]
Expand Down Expand Up @@ -74,7 +74,7 @@
"metadata": {},
"outputs": [],
"source": [
"bbox = [-93.79303, 43.009668, -93.753891, 43.038031]\n",
"bbox = [-98.451233, 38.430732, -98.274765, 38.523996]\n",
"region = ee.Geometry.BBox(*bbox)\n",
"region_style = {\"color\": \"red\", \"width\": 1}\n",
"Map.addLayer(region, region_style, \"Region\")\n",
Expand Down Expand Up @@ -177,21 +177,17 @@
"reg_timeseries = RegularTimeseries(gcvi_col, interval=5, window=45)\n",
"\n",
"# Get the regular timeseries\n",
"gcvi_col_regular = reg_timeseries.get_regular_timeseries()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"gcvi_col_regular = reg_timeseries.get_regular_timeseries()\n",
"\n",
"# Apply Savitzky-Golay filter\n",
"sg = SavitzkyGolayEE(\n",
" image_collection=gcvi_col, window_length=11, polyorder=2, band_name=\"gcvi\"\n",
")\n",
"\n",
"# Get the coefficients\n",
"coeff = sg.coefficients()\n",
"\n",
"# Extract phenometrics\n",
"phenometrics = Phenometrics(coeff)\n",
"metrics = phenometrics.metrics()"
]
Expand All @@ -209,16 +205,7 @@
"metadata": {},
"outputs": [],
"source": [
"cmap = cm.get_cmap(\"gist_rainbow\", 20)\n",
"cmap"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Plot the phenometrics (SOS, POS, and EOS)\n",
"cmap = cm.get_cmap(\"gist_rainbow\", 20)\n",
"cmap = [to_hex(cmap(i)) for i in range(cmap.N)]\n",
"\n",
Expand All @@ -228,58 +215,6 @@
"Map.addLayer(metrics.select(\"POS\"), vis, \"POS\")\n",
"Map.addLayer(metrics.select(\"EOS\"), vis, \"EOS\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Apply on MODIS Data"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"modis = (\n",
" ee.ImageCollection(\"MODIS/061/MOD13Q1\")\n",
" .filterDate(\"2020-01-01\", \"2021-01-01\")\n",
" .map(lambda img: img.clip(region))\n",
")\n",
"\n",
"sg = SavitzkyGolayEE(\n",
" image_collection=modis, window_length=15, polyorder=2, band_name=\"NDVI\"\n",
")\n",
"\n",
"coeff = sg.coefficients()\n",
"\n",
"phenometrics = Phenometrics(coeff)\n",
"metrics = phenometrics.metrics()\n",
"\n",
"cmap = cm.get_cmap(\"gist_rainbow\", 20)\n",
"cmap = [to_hex(cmap(i)) for i in range(cmap.N)]\n",
"\n",
"vis = {\"min\": 100, \"max\": 330, \"palette\": cmap}\n",
"\n",
"Map.addLayer(metrics.select(\"SOS\"), vis, \"SOS (MODIS)\")\n",
"Map.addLayer(metrics.select(\"POS\"), vis, \"POS (MODIS)\")\n",
"Map.addLayer(metrics.select(\"EOS\"), vis, \"EOS (MODIS)\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
1 change: 1 addition & 0 deletions docs/examples/timeseries_gap_filling.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,7 @@
"\n",
" return ndvi\n",
"\n",
"\n",
"ndvi_col = s2_cloud_masked.map(calculateNDVI)"
]
},
Expand Down
1 change: 1 addition & 0 deletions geeagri/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,4 @@
from . import analysis
from . import extract
from . import preprocessing
from . import phenology
18 changes: 4 additions & 14 deletions geeagri/analysis.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,26 +117,16 @@ class HarmonicRegression:
Perform harmonic regression on an Earth Engine ImageCollection.

Attributes:
image_collection (ee.ImageCollection): Input time series of selected band.
ref_date (ee.Date): Reference date to calculate time.
band (str): Name of dependent variable band.
order (int): Number of harmonics.
image_collection (ee.ImageCollection): Input image collection.
ref_date (str or ee.Date): Reference date to compute relative time.
band_name (str): Name of dependent variable band.
order (int): Number of harmonics (default 1).
omega (float): Base frequency multiplier.
independents (List[str]): Names of independent variable bands.
composite (ee.Image): Median composite of the selected band.
"""

def __init__(self, image_collection, ref_date, band_name, order=1, omega=1):
"""
Initialize the HarmonicRegression object.

Args:
image_collection (ee.ImageCollection): Input image collection.
ref_date (str or ee.Date): Reference date to compute relative time.
band_name (str): Name of dependent variable band.
order (int): Number of harmonics (default 1).
omega (float): Base frequency multiplier (default 1).
"""
self.image_collection = image_collection.select(band_name)
self.ref_date = ee.Date(ref_date) if isinstance(ref_date, str) else ref_date
self.band = band_name
Expand Down
6 changes: 0 additions & 6 deletions geeagri/preprocessing.py
Original file line number Diff line number Diff line change
Expand Up @@ -789,12 +789,6 @@ class RegularTimeseries:
def __init__(
self, image_collection: ee.ImageCollection, interval: int, window: int
):
"""
Args:
image_collection (ee.ImageCollection): Input collection to regularize.
interval (int): Interval (in days) between consecutive target dates.
window (int): Window size (in days) for interpolation.
"""
self._ic = image_collection
self.interval = interval
self.window = window
Expand Down
4 changes: 3 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,11 @@ nav:
- examples/moving_window_smoothing.ipynb
- examples/timeseries_gap_filling.ipynb
- examples/regular_timeseries.ipynb
- examples/land_surface_phenology.ipynb
- API Reference:
# - geeagri module: geeagri.md
# - common module: common.md
- extract module: extract.md
- preprocessing module: preprocessing.md
- analysis module: analysis.md
- analysis module: analysis.md
- phenology module: phenology.md
Binary file added paper/figure.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading