Google Summer of Code 2025: Cloudcasting ML Discussion Thread

[dfulu]

Cloudcasting ML Discussion Thread

This space is for you to ask any questions you have about this project. We're here to provide clarifications and help you understand the project's goals, scope, and requirements. Feel free to ask about anything that interests you!

Please note that this discussion is for questions and clarifications, not for formal applications.

Project Description

Traditionally, forecasting tools are trained using historical satellite data. As part of an innovative new project, we have been training a model to predict satellite images up to 3 hours ahead over the UK. This work in early stages, but we have already proved that a satellite forecast using a very simple ML model can improve our solar energy forecast. There is lots of opportunity to improve on this new and unique satellite forecast, from trying different video prediction and AI weather model architectures to training a diffusion based model to stop the satellite forecast being blurry.

Expected Outcome

An improved satellite forecast model

Other Key Information

• Expected Size: 350hrs
• Skills: ML, pytorch, python
• Difficulty level: Hard
• Related Reading:
  • https://www.openclimatefix.org/projects/cloud-forecasting
  • https://github.com/alan-turing-institute/cloudcasting
  • https://github.com/openclimatefix/sat_pred
• Potential mentors: @dfulu

[sumana-2705]

Hello @dfulu

I have opened an issue regarding training models on CPU-only machines. I wanted to ask whether GPU acceleration is strictly necessary for training, or if the models can be trained efficiently on a CPU as well. Are there any recommended optimizations or modifications to make training feasible on CPU-only systems?

[dfulu]

Hi @sumana-2705, I've responded to your issue on that repo. I haven't tested that code without using a GPU though, so you may encounter more issues.

More generally, even if the code can be run on CPU I think it will turn out to be too slow to train it this way. The current best performing model that we've trained (the simVP model) was trained on a GPU (though only an single GPU and not a great one) for 12 days. The model performance was improving for pretty much all of those 12 days of training. I'd expect CPU training to take significantly longer

[aayushyatiwari]

model) was trained on a GPU (though only an single GPU and not a great one) for 12 days. The model performance was improving for pretty much all of those 12 days of training

hey! uh dam, this might sound like a stupid question but I don't know. when training for 12 days, what machines did you use? were they normal pcs with NVIDIA GPUs or where they modified machines? Because I have a 4050 RTX and I was thiking to work on this project myself.

[dfulu]

I was a fairly modest GPU - an NVIDIA TESLA T4

[Syedmahmood777]

Hey @dfulu,

My name is Syed Mahmood, and I am a CSE-DS professional with a strong background in machine learning. The Cloudcasting ML project really interests me, and I would love to contribute to it as part of GSoC '25. I wanted to ask if there are any subject-specific resources or key areas I should focus on to better understand this project and align my contributions effectively.

I have experience working with PyTorch and ML model development, so I’d love to know if there are any particular frameworks, datasets, or methodologies I should get familiar with beforehand. Looking forward to your guidance!

[dfulu]

Hi @Syedmahmood777, thanks for your interest.

This project is still in its early stages, and we're primarily working with a custom dataset of satellite imagery which we maintain as a google public dataset. The focus is on developing machine learning models that predict future satellite images based on past observations—essentially a video-to-video prediction task.

Relevant areas to explore would be spatiotemporal deep learning models, such as ConvLSTMs, Transformers for video prediction, and/or diffusion models for reducing blur in predicted frames.

A good starting point might be to review our the github repo here, which includes the code for preparing and downloading datasets, and also links to a few of our ML experiment repos for this project. You might also find it helpful to look into research on deep learning for weather forecasting and video prediction models.

[abdksyed]

Hi James,

First, came to know about Open Climate Fix, and wanted to give a huge round of applause for building the need of hour tech for climate change. I have previously worked with image and video data, but mainly with medical data. Before I commit, wanted to know is there a need for subject expertise like GIS, Climate Science etc.?

You also mentioned that this project is in it's early stage, but is there any similar published word out there? The repo (and other linked repos) you shared doesn't have any detailed information of the models, results or anything as such.

Thanks!

[dfulu]

Hi @abdksyed, there is no expectation of knowledge of GIS or climate science. Our tech stack is all python and pytorch and we are mostly treating this as a video-to-video prediction task. Knowledge of weather/climate is not key but could be a bit of an advantage to help design better networks.

We have not published anything about this work yet, but a couple of the papers which have been of interest to us are [1] and [2]

• [1] https://arxiv.org/abs/2206.05099
• [2] Earthformer: Exploring Space-Time Transformers for Earth System Forecasting

[abdksyed]

Oh!, I realised that this project is part of something called GSoC, which I am not 100% sure is for students. I am a full-time AI software engineer, who wanted to contribute to a good OSS project, and came across this project.

I don't know if you are the right person to ask this, but will I have to apply for GSoC to work on this project? (as far as I have seen the eligibility doesn't mention that applicants have to be students, it just mentioned that one have to at least 18 years of age and a resident in participating county, and I satisfy both of them.)
But if GSoC is only for students, is there any way for me to be part of this project? I wanted to work on some nice CV OSS projects in a community setting

[dfulu]

I believe you can apply to GSoC if you are new to contributing to open source code, you do not need to be a student. At the moment we are only thinking of this project as part of GSoC.

[vinay752]

Hi James,

I'm Vinay Palakurthy. I hope you are doing great! I've gone through Open Climate Fix's website and read the story of OCF. I felt happy reading the origin of Open Climate Fix, developing AI driven solutions to improve efficiency in the energy sector and reducing green house gas emissions.

I was particularly impressed by the 5% accuracy improvement in the UK in solar generation forecast with AI cloud Forecasting that out-performed both UK and Europe Met Office Weather Services' when measuring the impact on short-term solar generation forecasting. It's amazing what Open Climate Fix has achieved in 6 years!

I'm excited for Cloudcasting to go live in the summer 2025. I'm eager to contribute to the Cloudcasting project, especially given my background in data science and with the expertise in time series forecasting. I believe my skills and knowledge in these areas could be valuable to your team.

I have couple of question for you if you don't mind to answer:

1. May I know how much of data did your team use to train the models to beat the agencies like Met Office Weather Services' who are in the market for a really long time?
2. Do you see Cloudcasting going beyond solar,generation forecast in the future, like maybe wind for those big windmills?

Best,
Vinay

[dfulu]

Hi @vinay752, thanks for your question

1. For training the cloudcasting model we use 15-minutely satellite data covering 2008-2022. Since we slice it down to only cover the UK it is about 1TB.

   This is just for the cloudcasting model which predicts future satellite data. We have a second model which ingests satellite data, the output of the cloudcasting model, and multiple weather forecasts from the Met Office and ECMWF. These add up to about maybe 10TB, but won't be used as part of this project.

2. Not at the moment. It isn't clear to us whether satellite data is useful for wind forecasting - we haven't tested it

[siddharth7113]

Hey @dfulu

First off, thanks for making this discussion space available! I’ve been going through the repo and had a few questions:

1. Last week, I set up the repo and downloaded the data from the public bucket. It took me nearly 18 hours to download just the 2019 data—probably due to a slow internet connection. Is there any plan to integrate this with ocf-data-sampler so that we can train on a larger dataset directly from cloud storage? It would be great if we could create chunks of Torch datasets on the fly and train models without needing to download everything locally. Apologies if I’m missing something here!

2. I also went through the Earthformer paper. With this project, is the main goal to develop new architectures that perform better on the current dataset, or is there also scope for increasing the dataset size to train models on a larger scale?

3. I noticed you mentioned AI diffusion models (video to video) in a previous message. Are there any recent papers in this area that you’d recommend checking out, Also are GANs a potentiall candidate for this , or do they have some inherent flaws with this?

4. Given the high computational cost of diffusion models, would most of the training be done on the cloud? Or are there plans to optimize it for local GPUs as well?

[dfulu]

Hi @siddharth7113, thanks for your qustions

1. The reason we download in the first place is because having the data stored locally stops the network speed from slowing down training. Streaming samples directly from cloud storage requires even faster network speed than storing them locally. So no, we don't plan to integrate with ocf-data-sampler. When downloading, I recommend using the utilities in https://github.com/alan-turing-institute/cloudcasting which crops the images down to the UK and subsets them to 1 image every 15 mins. The original images cover a much larger area and are 5-minutely

2. The main aim of the project is to make the model perform better on the current dataset. We believe there is a lot of room for improvement for our UK solar forecast we only really want to cover the UK currently. We will likely scale up geographically, but not until we've spent time making the model moire accurate at a smaller scale.

3. I don't have an particular papers to mention. Some if my colleagues found this helpful: https://arxiv.org/abs/2206.00364
   We don't plan to use GANs. As I understand it, now that diffusion networks have been developed more, they have made GANs outdated

4. As it is, it should be possible to train on a reasonably powered desktop. We found that about 1TB of disk space, one NVIDIA T4 GPU (with 16GB VRAM), and a CPU with 8 cores and 30GB RAM was enough to train a model. However, last year for our computationally demanding GSoC projects we set up candidates to use OCF's internal computer or on cloud VMs

[ShubhamChauhan22222]

Hi @dfulu, I am a 3rd-year student, pursuing an Integrated M.Tech in Mathematics and Computing. I have a strong background in AI/ML and GPU computing, and I am very interested in contributing to this project.
I have researched the available data in Zarr format and explored previous work related to satellite forecasting and the Cloudcasting repo. Now, I would like to understand the specific models and techniques you plan to implement for improving the satellite forecast model. If you have any particular approaches in mind, I’d be happy to explore them, or I can experiment with different methods independently.
Due to resource and time constraints, I will initially work with 30 days of data and share my findings here. Additionally, is there a way to connect privately for further discussion?
Looking forward to your guidance and feedback!

[dfulu]

Hi @ShubhamChauhan22222, thanks for your question

This is intended to be quite an open ended project. We don't have specific models and techniques in mind yet. So far the best model we have tried is SimVP, which is a very simple video prediction model. There are papers which follow on from SimVP to improve it. We have not fully explored those, but that would be my first intended direction. There are plenty of shortcomings in the SimVP architecture which we could improve on. We also tried using training the Earthformer model, but with the default parameter settings we could not beat SimVP for this task. The project will likely be to improve on the performance of SimVP, but a student with a strong interest in another model could try it out.

We will not be meeting privately with students until the project begins

[arjungithu53]

Hello @dfulu,

I would love to be part of the Cloudcasting ML project. Because of my skills in Python, PyTorch, and ML, as well as my working knowledge of Generative Adversarial Networks, I am certain I can contribute positively to the satellite forecast model.

I've been digging through the project description and related discussions, and I'm intrigued by the potential to explore different video prediction models like ConvLSTM or Temporal Convolutional Networks (TCNs), and AI weather models. Plus, integrating diffusion models could be a game-changer for reducing blurriness in the forecasts.

To get a better sense of the project's technical scope and challenges, I had a few questions:

Model Architecture: Have you explored using U-Net or ResNet architectures as the generator in a GAN framework for satellite image prediction? How do these models perform compared to simpler architectures?

Evaluation Metrics: Are you using metrics like Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), or Mean Squared Error (MSE) to evaluate the model's performance? Are there any specific benchmarks or baselines you're comparing against?

Scalability and Deployment: Do you have plans to deploy the model using tools like Docker and Kubernetes for scalability and cost efficiency? How are you handling hyperparameter tuning and model serving in a production environment?

I'm really looking forward to contributing to this project and learning more about your approach. Thanks for your time!

[dfulu]

Hi @arjungithu53, thanks for your questions

Model Architecture

We are aiming to predict satellite data at 15 minutely frequency over 3 hours. i.e. the target is to produce 12 image frames of video prediction. The original UNet or ResNet architectures were only designed to predict a single frame. However an XNet (quite similar architecture to UNet) is used as part of the best performing model we have tried so far (simVP) - but we think we can beat this model. We haven't tried using GANs yet.

Evaluation Metrics

Well really thing we are about most is how the satellite prediction improves the accuracy of our PV forecast. However, going from a trained network to a measure of how much it increased our PV forecast accuracy is a process that take a week and lot of computing power. In absence of this, we have been using MAE as our main metric of how good the satellite predictions are. There are no useful benchmarks in the literature because we are using a custom dataset, but initially we were comparing out results to optical flow and persistence - but we comfortably beat these baselines with simVP.

Scalability and Deployment

We have already deployed the simVP model via a docker image run and would expect to deploy a model that beats simVP in the same way. We wouldn't expect to do any hyperparameter tuning in the production setting. Model updates would be rolled out to our live service when they've been proved to be more accurate in a historic backtest.

[CaiRuinhan]

Hi, my name is Catherine, I am from China, and I am currently a second-year master's student in the Cognitive Science Lab. My research focuses on various deep learning models based on EEG signals, and most of my work is based on Pytorch. I am familiar with Python, Transformers, TCN, and LSTM (the above models are replicated because of the need for baseline algorithm comparison). I am very interested in the Cloudcasting ML project, and I would like to contribute to it as part of GSoC '25. I would like to know if there are any specific frameworks, datasets, or methods that I should be familiar with in advance. Since I am currently doing related research, I think there is the ability to operate on a local GPU unless the amount of data reaches the level of LLM. In the sat_pred repository, I saw that train.py calls the pre-trained model. This project is mainly based on video prediction, and I think such practical work can help me accumulate more experience before graduation. In GSoC 2025, will this task focus on improving the model? Will more data be involved? Looking forward to your reply~ I am very eager to join you to contribute to this great work, thank you.

[dfulu]

Hi @CaiRuinhan, happy to hear you are interested in the project.

You are right that this project will focus on trying to beat our current best performing model (simVP). That could be by changing the model architecture or by finding a more promising model from the literature. In terms of frameworks, good knowledge of pytorch is required and knowing pytorch-lightning would be useful, but lightning would be fairly easy to pick up. We have our own custom dataset we are working with so there is no need for knowledge of that. And we will be creating and training spatiotemporal models so some knowledge of those would be useful.

The full dataset is 1TB of data, so not much compared to state-of-the-art LLMs. But since this is a compute intensive project, we will likely set up the student on OCFs internal compute server.

The train.py script doesn't always use a pretrained model, that is only an option. We have been training models from scratch most of the time and that option to start from pretrained was added so we could fine-tune our own models we had already trained

[safalsingh1]

Hi @dfulu and team,

This project really caught my attention! The idea of improving short-term satellite image prediction to enhance solar forecasting sounds both challenging and impactful. I have a solid background in machine learning, PyTorch, and Python, and I'm especially interested in applying deep learning to real-world environmental challenges.

I'm curious—are you leaning towards any specific architectures for the next iteration? For example, ConvLSTMs, transformers, or diffusion models, as you mentioned? Also, will the evaluation focus more on visual quality (reducing blur) or on downstream tasks like solar forecast accuracy?

Excited to learn more about the project and how I can contribute!

[dfulu]

Hi @safalsingh1, thanks for your questions.

It is an open ended project and there is room for the student to try out some different models to beat our current one. We previously tried ConvLSTMs but we didn't get good results out of them - though I must admit that our implementation was as good as it could have been. We also tried experimenting with transformers (particularly the earthformer architecture) but didn't beat simVP. There is scope for a student to take these models up and try again, to find a model from the literature and try it, or simply to make modifications to simVP to try to make up for some of its shortcomings.

Ultimately we are looking for downstream accuracy in our solar forecast. We don't know whether reducing the bluriness using diffusion models will make our solar forecast more accurate but that is an interest direction.

[safalsingh1]

Hi James, Thank you for the clarification! I am very interested in the *Cloudcasting ML* project and confident that my AI/ML and PyTorch experience will allow me to contribute effectively. I’m excited to work on improving simVP and exploring alternative spatiotemporal models. Would it be possible to share a draft of my GSoC proposal with you early on for feedback? Thanks again, and I look forward to collaborating with the OCF team! Best regards, Safal Singh

…

On Sat, Mar 22, 2025 at 1:19 AM James Fulton ***@***.***> wrote: Hi @CaiRuinhan <https://github.com/CaiRuinhan>, happy to hear you are interested in the project. You are right that this project will focus on trying to beat our current best performing model (simVP). That could be by changing the model architecture or by finding a more promising model from the literature. In terms of frameworks, good knowledge of pytorch is required and knowing pytorch-lightning would be useful, but lightning would be fairly easy to pick up. We have our own custom dataset we are working with so there is no need for knowledge of that. And we will be creating and training spatiotemporal models so some knowledge of those would be useful. The full dataset is 1TB of data, so not much compared to state-of-the-art LLMs. But since this is a compute intensive project, we will likely set up the student on OCFs internal compute server. The train.py script doesn't always use a pretrained model, that is only an option. We have been training models from scratch most of the time and that option to start from pretrained was added so we could fine-tune our own models we had already trained — Reply to this email directly, view it on GitHub <#25 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BHCPSEC65NUIEZVCUNCBILT2VRULJAVCNFSM6AAAAABX7JDZC2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENJYGE2DOOA> . You are receiving this because you commented.Message ID: ***@***.*** com>

[dfulu]

Hi @safalsingh1, thanks for your message

Unfortunately, to ensure fairness to all applicants, we aren't able to review proposals before the official submission.

However, I'm happy to answer any specific questions you might have about the project ideas or the application process here in the discussion thread. Please feel free to ask away!

[Goosey-bruv]

Hello @dfulu

I've been exploring various architectures for this task, and I believe there's potential to enhance the current model by implementing a Hybrid Diffusion-Transformer Model. The idea is to:
Use a Diffusion Model for improving image clarity and sharpness, particularly to address the issue of blurry predictions.
Integrate a Transformer-based architecture (like Swin Transformer) to capture both short-term and long-term dependencies in satellite image sequences.
Combine Perceptual Loss functions with conventional MSE/MAE to improve visual quality.

I'm enthusiastic about contributing to this project, and I would love to discuss my ideas further with you. Additionally, I would appreciate your guidance on how I can effectively contribute to the project. Are there specific areas or issues you would recommend I start with? Also, would you be open to me implementing the Hybrid Diffusion-Transformer architecture to compare its performance with the existing models?

[dfulu]

Hi @Goosey-bruv

Thanks for laying out your diffusion-transformer idea – combining those techniques is definitely worth exploring.

As mentioned in other replies, our current best model is SimVP. Improving its blurry predictions is something we're interested in, potentially via diffusion models, although we haven't tested them extensively yet. We've also experimented with transformers (like Earthformer) but haven't surpassed simVP for this task so far, but they certainly could. So we'd be open to your proposal.

I don't think there are currently any open issues across the cloudcasting repos that are worth exploring right now. The next stage of this work will be to dive in and just try to beat simVP which will require a lot of time, effort, and compute. So I'd not recommend any student start on that until the project begins.

[Ashishkumar0302]

Hello @dfulu and everyone,

I’m Ashish Kumar, a second-year undergraduate student at IIT Kharagpur with a strong interest in Machine Learning and Deep Learning. I’ve been exploring the Cloudcasting ML project and find the idea of improving satellite image forecasting with AI really exciting.

I’ve worked on ML projects involving PyTorch, computer vision, and time-series forecasting, and I’m particularly interested in exploring a Hybrid Diffusion-Transformer Model for this project. My idea is to:

Use Diffusion Models to improve image clarity and reduce blurriness in satellite predictions.

Integrate a Transformer-based approach (such as Swin Transformer) to better capture short- and long-term dependencies in satellite image sequences.

Utilize Perceptual Loss functions along with MSE/MAE to enhance visual quality.

I’d love to discuss my ideas further and get your insights on how I can contribute effectively. Are there specific areas you’d suggest I start with? Also, would it be valuable to experiment with the Hybrid Diffusion-Transformer architecture to compare its performance with existing models?

Looking forward to learning and contributing!

[dfulu]

Thanks for your interest and outlining your ideas on the Hybrid Diffusion-Transformer model – it's an interesting direction.

As I've mentioned recently in this thread, simVP is our current best model, and the main goal is finding ways to improve upon it. We're open to exploring different architectures, including approaches like combining diffusion (for potential sharpness improvements) and transformers.

Regarding contributions right now: There aren't really any introductory issues available for this project currently. The core task is attempting to beat simVP, which will require substantial time and compute resources. Because of this, I'd recommend focusing on preparing your GSoC proposal for now, rather than starting implementation attempts before the program begins.

Happy to clarify anything else about the project goals or context here.

[YaxitaAmin]

Hello @dfulu ,

I'm a graduate student pursuing my MS in Applied Machine Learning at UMD. I'm very interested in the Cloudcasting ML project for GSoC 2025 and believe my background makes me a strong candidate.

My research experience includes extensive work with U-Net architectures for remote sensing applications, where I analyzed 60+ research papers on U-Net variants for satellite imagery. I also have experience with PyTorch, TensorFlow, and model optimization techniques - I recently compressed a BERT model by 80% while maintaining 91.5% of its performance.

After reviewing the repo and discussions, I have a few questions:

I see SimVP is currently your best performing model. Would you be interested in exploring modifications to address its shortcomings, particularly around the blurriness issue? I'm curious about potentially implementing a hybrid approach that maintains SimVP's strengths while incorporating diffusion techniques for sharper predictions.

You mentioned not having fully explored papers that follow and improve upon SimVP. Are there specific aspects of these follow-up architectures you'd like to prioritize investigating?

For evaluation, you mentioned MAE as the primary metric since direct PV forecast accuracy measurement is computationally intensive. Are there any other proxy metrics you've found correlate well with downstream PV forecast improvements?

I'm excited about the potential to contribute to this project and appreciate your time in answering these questions.
Looking forward to your response!

[sathvik-mn]

Hey @dfulu and everyone,

I am Sathvik, a Data Scientist. Enjoying reading through the thread. One thing that caught my attention was the curiosity around whether improving the sharpness of predictions (with diffusion models or otherwise) actually leads to better solar forecast performance.

That got me thinking, maybe we can test that connection on a smaller scale? Like comparing visual metrics like SSIM or perceptual loss with downstream PV accuracy. During my time working with satellite imagery in NASA’s Transform to Open Science workshop, I came across a similar challenge. Our models generated visually stunning results, but they didn’t always translate to better outcomes for the downstream task. That experience made this kind of trade-off really stick in my mind.

Open to any feedback in the thread. Let’s build something awesome together :)

Cheers,
Sathvik

[Oliver369X]

Hi @dfulu and the Open Climate Fix community,
My name is Diego Oliver, but I prefer to go by Oliver. I'm from Bolivia and very interested in applying for the Cloudcasting ML project for Google Summer of Code 2025.

My interest in applying technology to environmental challenges, specifically wildfires, grew significantly due to the severe events we face annually in Bolivia. This concern led me to get more involved starting in 2022. In 2023, I developed an early wildfire detection model which I managed to deploy on a small scale. While it was a helpful contribution, it also made me realize the effectiveness of such solutions is tied to complex temporal, social, and political factors.

Unfortunately, the 2024 wildfires in my region were even more devastating, burning over 10 million hectares and blanketing cities in smoke for weeks. This spurred a group of friends and me to seek more proactive ways to help. We decided to focus on mass reforestation using drones. We developed a platform and a small ML model that analyzes satellite imagery to identify soils with a higher probability of success for the germination of pelletized seeds. We had the chance to present this solution at a local smart city-themed hackathon, which we won, aiming to raise awareness of such alternatives among relevant authorities.

Early in 2025, we connected with the organization "Bosque Vidas," who shared valuable insights into the real-world challenges of bringing such projects into production and other issues surrounding fire management. Researching further into the connection between wildfires, climate change, and technology led me to discover Open Climate Fix and the GSoC program.
After spending the last few days intensively studying OCF's mission and the specifics of the Cloudcasting ML project (reviewing the cloudcasting and sat_pred repositories, and this discussion thread), I feel very motivated to apply. I see great potential in the video prediction and AI weather modeling techniques, not only for improving solar forecasting but also because these methodologies could potentially be adapted and applied to better understand atmospheric patterns related to wildfire risk or optimal reforestation conditions here in my region of Bolivia.

I admire OCF's focus on open-source solutions to combat climate change and I'm excited by the possibility of contributing my skills in ML (PyTorch, Computer Vision with satellite data) to this innovative project.
I have one specific question regarding the project's direction:

Considering that SimVP is the current best-performing model, and a key objective is tackling the blurriness issue (potentially with diffusion or transformers), while the ultimate measure of success is the downstream PV forecast improvement (which is computationally expensive to evaluate frequently), how does OCF/the mentor envision guiding the GSoC contributor? Specifically, how will the project balance exploring approaches that directly improve proxy metrics (like MAE, SSIM, visual sharpness) versus potentially different approaches that might initially score lower on these proxies but could better capture the underlying physical phenomena relevant to solar irradiance, thus potentially leading to a greater final impact on the PV forecast?

Thank you very much for creating this space and for your time. I look forward to the possibility of collaborating!

[RupeshMangalam21]

Hi, @dfulu

I’ve been exploring architecture ideas for improving the Cloudcasting model and wanted to share two promising directions grounded in recent research. Would love your thoughts on which aligns better with OCF’s goals!

Option 1: SimVP++-FNO Hybrid

• SimVP++ (SimVPv2)
  • Adds Gated Spatiotemporal Attention (GSTA) blocks to SimVP.
  • Reduces blur and improves MAE by 10.8% on weather benchmarks.
• FNO Layers (NeurIPS 2020)
  • Models global cloud motion in Fourier space (lightweight vs. transformers).
  • Proven for weather prediction tasks like WeatherBench.

Pros: Retains SimVP’s efficiency, adds physical consistency via FNO, T4-friendly.

Option 2: Optimized Diffusion

• Based on Karras et al. (NeurIPS 2022):
  • Achieves SOTA FID (1.36 on ImageNet-64) with 35-step sampling (vs. 1000+ in DDPM).
  • Simplified training and improved preconditioning.

Pros: Superior visual fidelity, sharper outputs.
Cons: Higher compute needs (A100 preferred), untested on satellite-scale data.

Some clarifying questions

1. Is preserving physical consistency (via FNO) more critical than visual sharpness for solar forecasts?
2. Has OCF tested diffusion models on satellite data at scale?

[vishwajitsarnobat]

Hi @dfulu,

I am Vishwajit Sarnobat, currently working as an AI-ML intern at ISRO (Indian Space Research Organisation) and pursuing my B.Tech. My partner and I are working on a very similar problem, utilizing precipitation satellite imagery available at 30-minute intervals to predict the next six frames here at ISRO. ISRO currently employs Pysteps (which uses the Optical Flow or Lucas-Kanade method) for precipitation prediction over the Indian Subcontinent. However, it offers just enough accuracy, and previous neural network models implemented by other interns produce blurry predictions (to reduce the metrics, models smooth out the predictions over pixels, which is not practically helpful).​
To address this, we have employed Conditional-GAN (from Skilful Nowcasting by DeepMind) and Latent Diffusion Model (we referred to the PreDiff implementation and paper). A notable aspect of PreDiff is its implementation of knowledge alignment, which uses Earthformer-UNet to improve predictions. We have started getting results and they look promising.​
After reviewing previous discussions on this page, I have the following questions:
1. You mentioned using a Diffusion model to remove blurriness. Are we planning to use it after the first model's prediction to obtain sharp predictions? If so, why not use the Diffusion model (with VAE for latent space reduction) entirely, as it offers promising accuracy?​
2. Regarding accuracy, forecasting models typically use CSI or FSS for fair evaluation. I also noticed that PreDiff included FVD as a metric for video quality evaluation. Are we planning to consider that as a metric, or stick to MAE and persistence?

[Rahul-JOON]

Hello @dfulu! I'm excited about the project focused on improving satellite predictions with machine learning. My background includes experience with ML through the Amazon ML Summer School 2024 and practical implementations like an LSTM for time-series data. I'm currently working on enhancing hourly temperature predictions using RNNs, which aligns well with the challenges of satellite data analysis. I'm passionate about leveraging ML for environmental applications and look forward to submitting a proposal.

One question I have, particularly regarding the satellite data, is: What are the primary sources and formats of the satellite data we'll be working with, and are there any known challenges or biases within this data that we should be aware of from the outset?

[yashaswip]

Hello @dfulu! I'm excited about the project focused on improving satellite predictions with machine learning—it's a fascinating and timely application. I'm currently pursuing an MS in Artificial Intelligence at Yeshiva University, where my coursework includes deep learning, reinforcement learning, NLP, and predictive modeling.

One question I have is: How is model performance currently evaluated in the project—are the focus areas metrics like SSIM or MSE on the predicted satellite frames, or is there a stronger emphasis on downstream performance, such as improvements in solar energy forecasting?

[AdityaRana112]

Hello @dfulu , I'm excited about the opportunity to contribute to the Cloudcasting ML project. With my prior experience in Python, PyTorch, machine learning, deep learning, Computer Vision along with a solid understanding of Generative Adversarial Networks and diffusion models. I’m confident in my ability to make a meaningful impact on the satellite forecasting model. I've been closely reviewing the project description and related discussions, and I'm particularly fascinated by the potential to explore various video prediction models.

[satsin06]

Hello @dfulu,

I am Satyam Sinha, I am actively exploring into sat_pred repository, I am currently facing issue finding .zarr files to execute train.py file. I'd really appreciate your help in resolving this.

FileNotFoundError: No such file or directory: '/mnt/disks/sat_data/sat_data_all/2008_training_nonhrv.zarr'

On the other hand, I went through the Project description and I have mostly figured out about the works to be done to improve model using SimVP. Simultaneously, I would also like to give it a try with other models like Earthformer or UNet variants model and evaluate models on MAE and visual consistency of predictions. Further I will be working into blurriness mitigation using GANs or perceptual losses and conduct comparisons between SimVP and newer models.

Please let me know if my approach is appropriate enough to get started with this project.

Thanks and regards,
Satyam Sinha

[satsin06]

@dfulu Please reply

[emlweb]

Google Summer of Code 2025 applications are now closed.

We are currently reviewing all applications. Contributors will be announced 8 May 2025. Thank you!

[satsin06]

None got selected for this project?

[satsin06]

@emlweb @dfulu Can you please help me with some details, I am still interested in contributing to this project.

[peterdudfield]

I'm closing this discussing now as GSOC 2025 is nearly over. Thank you for everyones input and help.

We hope to take part next year and we'll be posting info here