Skip to content
Saketh Kamatham

Airport Assignment for Emergency Aircraft using Reinforcement Learning

Master's thesis: a DQN agent in a custom OpenAI Gym airspace routes an emergency aircraft to a safe airport, comparing epsilon-greedy and Boltzmann exploration.

Status
Archived
Dates
Tags
  • Reinforcement Learning
  • DQN
  • OpenAI Gym

Master's thesis, San José State University, 2023. Advised by Dr. Genya Ishigaki.

Problem

Route an emergency aircraft to a safe airport in the fewest steps, without leaving the airspace or hitting obstacles. I modeled it as a sequential decision problem rather than a fixed shortest-path lookup, since the "airport" location is randomized per episode.

Approach

Built a custom OpenAI Gym environment simulating a 500-by-500 2D airspace. Framed it as an MDP: state = aircraft position (and obstacle layout, in the harder scenario), action = move direction, reward = shaped to encourage reaching the airport quickly while penalizing leaving the airspace or hitting an obstacle. Trained a DQN agent with experience replay. Compared two exploration strategies, epsilon-greedy and Boltzmann (softmax), across two scenarios (with and without obstacles), with a hyperparameter sweep to find the best setting for each: epsilon-greedy at ε = 0.7 decaying to 0.1, Boltzmann at τ = 0.7 decaying to 0.5.

Architecture

  1. Gym environment500 × 500 airspace
  2. DQN agentexperience replay
  3. Exploration policyε-greedy vs Boltzmann
  4. Training loop2 scenarios × 2 policies
  5. Evaluationsuccess · reward · steps
Fig. 3 · Training pipeline. The agent and environment exchange state, action and reward each step; the two exploration policies are compared, not combined
Drawn
SK
Rev
A
Status
Archived

Outcome

Epsilon-greedy outperformed Boltzmann exploration on every metric tracked (success rate, mean reward, and how close the agent's path got to the theoretical minimum step count) in both the obstacle and no-obstacle scenarios.