ses 598 · school of earth and space exploration

SES 598: Space Robotics and AI

termSpring 2027
class #39591
meetsTu/Th 10:30-11:45am
whereISTB4 296, ASU Tempe campus
instructorDr. Jnaneshwar Das
contactjdas5@asu.edu

This course provides a comprehensive introduction to robotic exploration and AI-driven mapping and sampling techniques, tailored for space exploration and earth observation. Students will gain expertise in key areas such as computer vision, Simultaneous Localization and Mapping (SLAM), multi-robot coordination, and operations in extreme environments. The curriculum emphasizes a strong theoretical foundation leading up to real-world implementation, combining lectures with hands-on projects using mobility autonomy systems, including autonomous ground, aerial, and aquatic robots available as digital twins and physically in the DREAMS Laboratory. The course culminates in a group-based final project, where students design and demonstrate end-to-end robotic systems for space exploration, planetary science, and earth observation.

Ready for your space mission?

Complete the pre-enrollment quiz to assess your readiness.

pre-enrollment quiz →

Prerequisites

what to bring

Mathematics

Linear algebra (vectors, matrices, eigenvalues), calculus (derivatives, gradients), and probability theory (Bayes rule, distributions)
Programming

Strong Python programming skills with experience in scientific computing libraries (NumPy, SciPy, PyTorch/TensorFlow)
Computer Vision

Basic understanding of image processing, feature detection, and geometric transformations
Computing Systems

Experience with Linux/Unix systems, version control (Git), and command-line tools
Recommended

Prior exposure to ROS (Robot Operating System)
Required Software

Students must have a computer capable of running Linux and processing 3D graphics

Course Schedule

topics, lectures, assignments

Week	Topics	Lectures & assignments	Related resources
week 1-3 (Jan 12-Jan 29) State estimation and Controls	Least squares and maximum likelihood estimation (MLE) State space models and linear dynamical systems State-estimation with Kalman and particle filters PID control, linear quadratic regulator (LQR), and model predictive control (MPC) Entry descent and landing (EDL), guidance navigation and control (GNC), and attitude determination and control system (ADCS)	Assignment 1: First-Order Boustrophedon Navigator (Lawnmower pattern) using ROS2 (Due: Jan 26, 2027)	papers MPC for Quadrotor Flight Mars 2020 EDL Psyche Mission GNC tutorials Parameter Estimation Tutorial
week 4-5 (Feb 2-Feb 19) Computer Vision and 3D Reconstruction	Image formation and camera models Feature detection and matching Epipolar geometry and stereo vision Structure from Motion (SfM) Multi-View Stereo (MVS)	Assignment 2: Optimal Control of Cart-Pole System with LQR (Due: Feb 16, 2027) Assignment 3: ORBSLAM3 with ROS2 on PX4 SITL drone at Bishop Fault Scarp scene (Due: Feb 23, 2027)	papers DUSt3R SLAM Survey COLMAP: SfM Revisited ORB-SLAM tutorials Random Sample Consensus (RANSAC) Tutorial Multi-View Geometry Tutorial
week 6 (Feb 23-Mar 2) Scene Representation, View Synthesis, and Scene Analysis	Scene representation: Orthomaps, pointcloud, mesh models, voxel grids, implicit surface models, and surfels View synthesis: Neural Radiance Fields (NeRF), and Gaussian Splatting Scene analysis: Semantic segmentation of images and point clouds leveraging neural networks	Assignment 4: View synthesis and scene analysis on Apollo 17 and Lunar analog datasets.	papers Gaussian Splatting SLAM NeRF PIN-SLAM
week 7 (Mar 3-Mar 5) Sampling Strategies and Information Theory	Information theory fundamentals Active sampling and exploration Multi-armed bandits and Bayesian optimization Information gain in exploration	Assignment 5: Optimal sampling challenge on James Webb Space Telescope (JWST) datasets.	papers Marine Ecosystem Sampling JWST 3D Drizzle JWST Observations Optimization tutorials Gaussian Processes Tutorial Information-based Sampling Tutorial
week 8-9 (Mar 17-Mar 30) Digital and Cyber-Physical Twins	Decision support systems, geographic information systems (GIS), and digital twins Self-supervised learning of stochastic dynamical processes with physical twins Case study - earthquake geology: Virtual shake robot and particle dynamical studies Closing the loop on model improvement with cyber-physical twins	Exercise: Virtual Shake Robot and ShakeBot cyber-physical twin experiments (not graded)	papers Virtual Shake Robot
week 10-11 (Mar 31-Apr 13) SLAM and Active Perception	Information-theoretic SLAM Active view selection Uncertainty-aware mapping Exploration-exploitation trade-offs Resource-constrained planning	Midterm Project: Information-driven Robot Autonomy Challenge either in digital twins or physical robots.	papers Range-Visual-Inertial Odometry Next-Best-View Planning Active Semantic Mapping Ultimate SLAM Thermal-Inertial Odometry tutorials Path Planning Tutorial
week 12-13 (Apr 14-Apr 27) Multi-Robot Coordination and Distributed Learning	Distributed bandit algorithms Multi-agent exploration strategies Collaborative information gathering Decentralized decision making Communication-aware sampling		papers Distributed Multi-Armed Bandits Decentralized Cooperative Bandits
week 14-15 (Apr 28-Apr 30) Extreme Environment Operations	Risk-aware exploration Robust sampling strategies Adaptive resource allocation Environmental uncertainty modeling Safety-constrained learning	Final (group) Project: End-to-end autonomous robotic system themed around space exploration, planetary science, or earth observation.	papers Perseverance Progress Perseverance Pose Estimation Ingenuity Mars Helicopter

Grading

components and weights

Assignments 20%

Five graded assignments through the semester to reinforce learning concepts and practical skills.
Midterm Project 20%

A comprehensive project due mid-semester that integrates core concepts covered in the first half.
Final Project 50%

A major project that demonstrates mastery of course concepts, including implementation and documentation. This can be a continuation of the midterm project
Class Participation 10%

Active participation in class discussions, group activities, and engagement with course material.

Resources

slides, books, papers

2025 lecture slides archive

Google Drive

Preview lecture slides from the 2025 course offering to explore content and topics covered in Space Robotics and AI.

Recommended books

Probabilistic Robotics
Sebastian Thrun, Wolfram Burgard, Dieter Fox
A foundational text on probabilistic approaches to robotics, covering core algorithms for perception, estimation, and planning under uncertainty.
Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches
Dan Simon
A comprehensive guide to state estimation techniques with practical applications in navigation and control systems.
Pattern Recognition and Machine Learning
Christopher M. Bishop
The definitive text on modern pattern recognition methods with a focus on Bayesian techniques and machine learning algorithms.

read book →
Multiple View Geometry in Computer Vision
Richard Hartley and Andrew Zisserman
The foundational text for understanding geometric relationships between multiple views and 3D reconstruction techniques.
Optimal Control and Estimation
Robert F. Stengel
A classic text bridging theory and practice in optimal control, estimation, and stochastic systems analysis.

Interactive Tutorials

run in the browser

Computer Vision

Tutorial	Description	Difficulty
Multi-View Geometry	Epipolar geometry, fundamental matrix, and camera calibration	Intermediate	start →
Bundle Adjustment	Interactive demo of bundle adjustment with multiple cameras	Advanced	start →

Control and Planning

Tutorial	Description	Difficulty
Drone Control Primer	Interactive introduction to drone control and navigation	Advanced	start →
Path Planning	A*, RRT, and potential fields algorithms	Intermediate	start →
Cart Pole Control	LQR control for inverted pendulum	Advanced	start →

Estimation

Tutorial	Description	Difficulty
Parameter Estimation	Least-squares estimation for linear regression	Intermediate	start →
Random Sample Consensus (RANSAC)	Hands-on implementation of RANSAC for robust model fitting with outlier rejection	Beginner	start →
Gaussian Processes	Interactive visualization of GP regression for spatial prediction and uncertainty estimation	Advanced	start →
Information-based Sampling	Cross-Entropy Sampling	Intermediate	start →
Sensor Fusion	Interactive Kalman filter demo showing process vs measurement noise trade-offs in fast-moving systems	Advanced	start →

Optimization and Learning

Tutorial	Description	Difficulty
Multi-Armed Bandit Tutorial	Interactive demonstration of exploration vs. exploitation in Multi-Armed Bandit problems, comparing human performance against common algorithms like ε-Greedy, UCB1, and Thompson Sampling.	Intermediate	start →

Prerequisites

Mathematics

Programming

Computer Vision

Computing Systems

Recommended

Required Software

Course Schedule

Grading

Assignments 20%

Midterm Project 20%

Final Project 50%

Class Participation 10%

Resources

2025 lecture slides archive

Google Drive

Recommended books

Probabilistic Robotics

Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches

Pattern Recognition and Machine Learning

Multiple View Geometry in Computer Vision

Optimal Control and Estimation

Recommended papers

Scene Representation and View Synthesis

Gaussian Splatting SLAM

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Diffusion Models in Vision: A Survey

PIN-SLAM: LiDAR SLAM Using a Point-Based Implicit Neural Representation for Achieving Global Map Consistency

SLAM and Active Perception

Structure-Invariant Range-Visual-Inertial Odometry

Receding Horizon "Next-Best-View" Planner for 3D Exploration

Active Semantic Mapping and Pose Graph Spectral Analysis for Robot Exploration

Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High-Speed Scenarios

Data-Efficient Collaborative Decentralized Thermal-Inertial Odometry

Extreme Environment Operations

Autonomous robotics is driving Perseverance rover's progress on Mars

Precise pose estimation of the NASA Mars 2020 Perseverance rover through a stereo-vision-based approach

Ingenuity Mars Helicopter: From Technology Demonstration to Extraterrestrial Scout

Sampling Strategies and Information Theory

Data-driven robotic sampling for marine ecosystem monitoring

A 3D drizzle algorithm for JWST and practical application to the MIRI Medium Resolution Spectrometer

An information-theoretic approach to optimize JWST observations and retrievals of transiting exoplanet atmospheres

Multi-Robot Coordination

Distributed exploration in multi-armed bandits

Decentralized cooperative stochastic bandits

Digital and Cyber-Physical Twins

Virtual Shake Robot: Simulating Dynamics of Precariously Balanced Rocks for Overturning and Large-displacement Processes

Control and Planning

Model Predictive Contouring Control for Time-Optimal Quadrotor Flight

Assessment of the Mars 2020 Entry, Descent, and Landing Simulation

Psyche Mission System Level Guidance, Navigation, and Control Off-Nominal Testing

Computer Vision and 3D Reconstruction

DUSt3R: Geometric 3D Vision Made Easy

Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age

Structure-From-Motion Revisited

ORB-SLAM: A Versatile and Accurate Monocular SLAM System

Interactive Tutorials

Computer Vision

Control and Planning

Estimation

Optimization and Learning