Master's Thesis

Short description

This thesis explores the development of multi-modal foundation models for the Earth system that are capable of efficiently processing multi-resolution data from diverse Earth observation sensors. The focus is on designing and evaluating novel deep learning architectures that support both Earth surface and atmospheric variables across land and maritime environments.

Keywords

Earth system · Remote sensing · Foundation models · Deep learning · Multimodal data · Computer vision

Thesis description

The rapid advancement of foundation models—large-scale AI systems pre-trained on vast amounts of unlabeled data—has opened new frontiers in Earth system science and remote sensing. These models, fine-tuned with smaller labeled datasets, offer a powerful framework for a wide range of downstream geospatial tasks.

Remote sensing is particularly well-suited for foundation model development due to the abundance of unlabeled data from a variety of satellite platforms. However, current models often rely on data from a single set of sensors or fixed resolution, limiting their generalizability and scope. There is a growing need for holistic models that can integrate multi-sensor, multi-resolution data, encompassing both surface and atmospheric observations.

This thesis aims to address this gap by designing deep learning architectures that can learn from heterogeneous geospatial data sources. The goal is to build scalable, efficient, and generalizable models that advance the state of the art in Earth system monitoring and analysis.

Objectives

• Develop embedding predictive architectures support multi-resolution and multimodal integration of Earth system data.
• Enable satellite-agnostic pretraining and finetuning workflows.
• Evaluate model performance across diverse Earth observation tasks, including both surface and atmospheric domains.

Technical Environment

• Languages & Tools: Python, PyTorch, DDP, OpenShift, LSF, GitHub/Lab

Skills required:

• Strong problem-solving mindset and ability to synthesize novel deep learning architectures from literature.
• Familiarity with deep learning, especially transformer models and their positional/temporal encoding mechanisms.

Preferred experience:

• Hands-on experience with high-performance computing (HPC) environments and job scheduling.
• OpenShift and LSF experience
• Deep understanding of multimodal learning
• Knowledge about large-scale geospatial datasets

Diversity

IBM is committed to diversity at the workplace. With us you will find an open, multicultural environment. Excellent flexible working arrangements enable all genders to strike the desired balance between their professional development and their personal lives.

How to apply

Please submit your application through the link below. We encourage candidates to also share a 3-minute video, in which they introduce themselves, as well as highlight their motivation and expertise. The video is not mandatory.

If you have any question related to this position, please contact Dr. Thomas Brunschwiler , E-Mail schreiben.

References
TerraMind: Large-Scale Generative Multimodality for Earth Observation. [TerraMind]
Foundation Models for Generalist Geospatial Artificial Intelligence. [Prithvi]
Lightweight, Pre-trained Transformers for Remote Sensing Timeseries. [Presto]
DiffusionSat: A Generative Foundation Model for Satellite Imagery. -[DiffusionSat]