RAS 598: Space Robotics and AI

Spring 2025
RAS #39245
Tu/Th 4:30-5:45pm
ASU Poly Campus, Room SANTN339
Dr. Jnaneshwar Das
jdas5@asu.edu

Is this course a good fit for you?

Take Quiz

This course provides a comprehensive introduction to robotic exploration and AI-driven mapping and sampling techniques, tailored for space exploration and earth observation. Students will gain expertise in key areas such as computer vision, Simultaneous Localization and Mapping (SLAM), multi-robot coordination, and operations in extreme environments. The curriculum emphasizes a strong theoretical foundation leading up to real-world implementation, combining lectures with hands-on projects using mobility autonomy systems, including autonomous ground, aerial, and aquatic robots available as digital twins and physically in the DREAMS Laboratory. The course culminates in a group-based final project, where students design and demonstrate end-to-end robotic systems for future space exploration, planetary science, and earth observation.

Prerequisites

Mathematics

Linear algebra (vectors, matrices, eigenvalues), calculus (derivatives, gradients), and probability theory (Bayes rule, distributions)

Programming

Strong Python programming skills with experience in scientific computing libraries (NumPy, SciPy, PyTorch/TensorFlow)

Computer Vision

Basic understanding of image processing, feature detection, and geometric transformations

Computing Systems

Experience with Linux/Unix systems, version control (Git), and command-line tools

Recommended

Prior exposure to ROS (Robot Operating System), CUDA programming, or parallel computing

Required Software

Students must have a computer capable of running Linux and processing 3D graphics

Course Schedule

Week Topics Lectures & Assignments Related Papers
1-2 (Jan 14-25)
State estimation and Controls
  • Least squares and maximum likelihood estimation (MLE)
  • State space models and linear dynamical systems
  • State-estimation with Kalman and particle filters
  • PID control, linear quadratic regulator (LQR), and model predictive control (MPC)
  • Entry descent and landing (EDL), guidance navigation and control (GNC), and attitude determination and control system (ADCS)
Assignment 1: First-Order Boustrophedon Navigator (Lawnmower pattern) using ROS2 (Due: Jan 27, 2025)
Assignment 2: Optimal Control of Cart-Pole System using ROS2 (Due: Feb 10, 2025)
3-4 (Jan 28-Feb 8)
Computer Vision and 3D Reconstruction
  • Image formation and camera models
  • Feature detection and matching
  • Epipolar geometry and stereo vision
  • Structure from Motion (SfM)
  • Multi-View Stereo (MVS)
Assignment 3: Offline 3D reconstruction pipeline leveraging OpenCV and COLMAP in ROS2
5 (Feb 11-15)
Scene Representation, View Synthesis, and Scene Analysis
  • Scene representation: Orthomaps, pointcloud, mesh models, voxel grids, implicit surface models, and surfels
  • View synthesis: Neural Radiance Fields (NeRF), and Gaussian Splatting
  • Scene analysis: Semantic segmentation of images and point clouds leveraging neural networks
  • Generative scenes: Diffusion models
Assignment 4: View synthesis and scene analysis on Apollo 17 and Lunar analog datasets.
6 (Feb 18-22)
Sampling Strategies and Information Theory
  • Information theory fundamentals
  • Active sampling and exploration
  • Multi-armed bandits and Bayesian optimization
  • Information gain in exploration
Assignment 5: Optimal sampling challenge on James Webb Space Telescope (JWST)datasets.
7-8 (Feb 25-Mar 8)
Digital and Cyber-Physical Twins
  • Decision support systems, geographic information systems (GIS), and digital twins
  • Self-supervised learning of stochastic dynamical processes with physical twins
  • Case study 1 - earthquake geology: Virtual shake robot and particle dynamical studies
  • Case study 2 - ecological digital and physical twins
  • Closing the loop on model improvement with cyber-physical twins
Assignment 6: Adaptive digital twin system involving seismic studies with virtual shake robot and ShakeBot.
9-10 (Mar 17-29)
SLAM and Active Perception
  • Information-theoretic SLAM
  • Active view selection
  • Uncertainty-aware mapping
  • Exploration-exploitation trade-offs
  • Resource-constrained planning
Midterm Project: Information-driven Robot Autonomy Challenge either in digital twins or physical robots.
11-12 (Apr 1-12)
Multi-Robot Coordination and Distributed Learning
  • Distributed bandit algorithms
  • Multi-agent exploration strategies
  • Collaborative information gathering
  • Decentralized decision making
  • Communication-aware sampling
Assignment 7: Multi-robot exploration system in digital twins.
13-14 (Apr 15-26)
Extreme Environment Operations
  • Risk-aware exploration
  • Robust sampling strategies
  • Adaptive resource allocation
  • Environmental uncertainty modeling
  • Safety-constrained learning
Assignment 8: Digital twin exercise on planning under risks and uncertainty.
15-16 (Apr 29-May 3)
Integration & Advanced Applications
  • Meta-learning for exploration
  • Transfer learning in space applications
  • Lifelong learning systems
  • Integrated perception-planning-learning
  • Real-world deployment strategies
Final (group) Project: End-to-end autonomous robotic system themed around space exploration, planetary science, or earth observation.

Grading

Assignments (20%)

Eight assignments throughout the semester to reinforce learning concepts and practical skills.

Midterm Project (20%)

A comprehensive project due mid-semester that integrates core concepts covered in the first half.

Final Project (50%)

A major project that demonstrates mastery of course concepts, including implementation and documentation. This can be a continuation of the midterm project

Class Participation (10%)

Active participation in class discussions, group activities, and engagement with course material.

Resources

Recommended Books

Probabilistic Robotics

Authors: Sebastian Thrun, Wolfram Burgard, Dieter Fox

A foundational text on probabilistic approaches to robotics, covering core algorithms for perception, estimation, and planning under uncertainty.

Optimal State Estimation: Kalman, H Infinity, and Nonlinear Approaches

Authors: Dan Simon

A comprehensive guide to state estimation techniques with practical applications in navigation and control systems.

Pattern Recognition and Machine Learning

Authors: Christopher M. Bishop

The definitive text on modern pattern recognition methods with a focus on Bayesian techniques and machine learning algorithms.

Read Book

Multiple View Geometry in Computer Vision

Authors: Richard Hartley and Andrew Zisserman

The foundational text for understanding geometric relationships between multiple views and 3D reconstruction techniques.

Optimal Control and Estimation

Authors: Robert F. Stengel

A classic text bridging theory and practice in optimal control, estimation, and stochastic systems analysis.

Recommended Papers

Scene Representation and View Synthesis

Gaussian Splatting SLAM

Matsuki, H., Murai, R., Kelly, P.H.J., Davison, A.J.

CVPR 2024 View Paper

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

Mildenhall, B., Srinivasan, P.P., Tancik, M., Barron, J.T., Ramamoorthi, R., Ng, R.

Communications of the ACM 2021 View Paper

Diffusion Models in Vision: A Survey

Croitoru, F.A., Hondru, V., Ionescu, R.T., Shah, M.

IEEE TPAMI 2023 View Paper

PIN-SLAM: LiDAR SLAM Using a Point-Based Implicit Neural Representation for Achieving Global Map Consistency

Pan, Y., Zhong, X., Wiesmann, L., Posewsky, T., Behley, J., Stachniss, C.

IEEE Transactions on Robotics 2024 View Paper

SLAM and Active Perception

Structure-Invariant Range-Visual-Inertial Odometry

Alberico, I., Delaune, J., Cioffi, G., Scaramuzza, D.

IROS 2024 View Paper

Receding Horizon "Next-Best-View" Planner for 3D Exploration

Bircher, A., Kamel, M., Alexis, K., Oleynikova, H., Siegwart, R.

ICRA 2016 View Paper

Active Semantic Mapping and Pose Graph Spectral Analysis for Robot Exploration

Zhang, R., Bong, H.M., Beltrame, G.

IROS 2024 View Paper

Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High-Speed Scenarios

Vidal, A.R., Rebecq, H., Horstschaefer, T., Scaramuzza, D.

IEEE Robotics and Automation Letters 2018 View Paper

Data-Efficient Collaborative Decentralized Thermal-Inertial Odometry

Polizzi, V., Hewitt, R., Hidalgo-Carrió, J., Delaune, J., Scaramuzza, D.

IEEE Robotics and Automation Letters 2022 View Paper

Extreme Environment Operations

Autonomous robotics is driving Perseverance rover's progress on Mars

Verma, V., et al.

Science Robotics 2023 View Paper

Precise pose estimation of the NASA Mars 2020 Perseverance rover through a stereo-vision-based approach

Andolfo, S., Petricca, F., Genova, A.

Journal of Field Robotics 2023 View Paper

Ingenuity Mars Helicopter: From Technology Demonstration to Extraterrestrial Scout

Tzanetos, T., et al.

IEEE Aerospace Conference 2022 View Paper

Sampling Strategies and Information Theory

Data-driven robotic sampling for marine ecosystem monitoring

Das, J., Py, F., Harvey, J.B.J., Ryan, J.P., Gellene, A., Graham, R., Caron, D.A., Rajan, K., Sukhatme, G.S.

International Journal of Robotics Research 2015 View Paper

A 3D drizzle algorithm for JWST and practical application to the MIRI Medium Resolution Spectrometer

Law, D.D., et al.

The Astronomical Journal 2023 View Paper

An information-theoretic approach to optimize JWST observations and retrievals of transiting exoplanet atmospheres

Howe, A.R., Burrows, A., Deming, D.

The Astrophysical Journal 2017 View Paper

Multi-Robot Coordination

Distributed exploration in multi-armed bandits

Hillel, E., Karnin, Z.S., Koren, T., Lempel, R., Somekh, O.

NeurIPS 2013 View Paper

Decentralized cooperative stochastic bandits

Martínez-Rubio, D., Kanade, V., Rebeschini, P.

NeurIPS 2019 View Paper

Digital and Cyber-Physical Twins

Virtual Shake Robot: Simulating Dynamics of Precariously Balanced Rocks for Overturning and Large-displacement Processes

Chen, Z., Arrowsmith, R., Das, J., Wittich, C., Madugo, C., Kottke, A.

Seismica 2024 View Paper

Control and Planning

Model Predictive Contouring Control for Time-Optimal Quadrotor Flight

Romero, A., Sun, S., Foehn, P., Scaramuzza, D.

IEEE Transactions on Robotics 2022 View Paper

Assessment of the Mars 2020 Entry, Descent, and Landing Simulation

Way, D.W., Dutta, S., Zumwalt, C.H., Blette, D.J.

AIAA SciTech 2022 View Paper

Psyche Mission System Level Guidance, Navigation, and Control Off-Nominal Testing

Arthur, P., Navarro, J., Sover, K., Sternberg, D., Twu, P.

IEEE Aerospace Conference 2024 View Paper

Computer Vision and 3D Reconstruction

DUSt3R: Geometric 3D Vision Made Easy

Wang, S., Leroy, V., Cabon, Y., Chidlovskii, B., Revaud, J.

CVPR 2024 View Paper

Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age

Cadena, C., et al.

IEEE Transactions on Robotics 2016 View Paper

Structure-From-Motion Revisited

Schonberger, J.L., Frahm, J.M.

CVPR 2016 View Paper

ORB-SLAM: A Versatile and Accurate Monocular SLAM System

Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.

IEEE Transactions on Robotics 2015 View Paper

Interactive Tutorials

Computer Vision

Tutorial Description Difficulty Action

Multi-View Geometry

 Epipolar geometry, fundamental matrix, and camera calibration Intermediate Start Tutorial →

Bundle Adjustment

 Interactive demo of bundle adjustment with multiple cameras Advanced Start Tutorial →

SLAM Tutorial

 Simultaneous Localization and Mapping fundamentals Advanced Start Tutorial →

Control and Planning

Tutorial Description Difficulty Action

Drone Control Primer

 Interactive introduction to drone control and navigation Advanced Start Tutorial →

Path Planning

 A*, RRT, and potential fields algorithms Intermediate Start Tutorial →

Cart Pole Control

 LQR control for inverted pendulum Advanced Start Tutorial →

Estimation

Tutorial Description Difficulty Action

Parameter Estimation

 Least-squares estimation for linear regression Intermediate Start Tutorial →

Random Sample Consensus (RANSAC)

 Hands-on implementation of RANSAC for robust model fitting with outlier rejection Beginner Start Tutorial →

Gaussian Processes

 Interactive visualization of GP regression for spatial prediction and uncertainty estimation Advanced Start Tutorial →

Information-based Sampling

 Cross-Entropy Sampling Intermediate Start Tutorial →

Sensor Fusion

 Interactive Kalman filter demo showing process vs measurement noise trade-offs in fast-moving systems Advanced Start Tutorial →