Go Up Autonomous Reinforcement Learning Agent
Custom Gym-Style Reinforcement Learning Environment and Deep Q-Network (DQN) Agent Autonomously Navigating Dynamic Vertical Platforms

System Context & Value Proposition
GoUp-RL is a reinforcement learning platform where an autonomous neural policy learns to navigate procedurally generated vertical platform stages without human intervention. Built from scratch in Python and PyTorch, the project implements a custom Gym-compatible game environment (reset, step, close), a Deep Q-Network (DQN) agent with experience replay, target network stabilization, and custom reward engineering.
Training an autonomous agent in a fast-paced vertical platformer requires balancing exploration vs exploitation in sparse, high-penalty reward landscapes where fatal falls terminate episodes instantly.
System Architecture & Execution Flow
Custom Pygame RL Environment
↓ (Observation State: Player Pos, Velocity, Nearest Platforms)
Deep Q-Network Agent (PyTorch Neural Policy)
↓ (Q-Value Estimation for Discrete Actions)
Action Selection (Epsilon-Greedy Exploration vs Exploitation)
↓ (0: Idle, 1: Left, 2: Right, 3: Jump)
Game Physics Execution & Collision Check
↓
Reward Calculation (+Altitude Gain, -Idle Time, -Fall Penalty)
↓
Experience Replay Buffer (State, Action, Reward, Next State, Done)
↓
Target Network Parameter UpdatesMulti-Tier Architectural Layers
Pygame, Python
Gym-style environment interface (reset, step, close) exposing player kinematics, jump timers, and relative distance vectors to upcoming platforms.
PyTorch Multi-Layer Perceptron
Neural network approximating action-value Q(s, a) functions with fully connected linear layers, ReLU activations, and Mean Squared Error loss.
Circular Deque Buffer
Stores 50,000 transition tuples and samples uncorrelated mini-batches during gradient updates to stabilize learning.
Multi-Stage Environments
Progressive environment stages (rl_env_stage_1.py to stage_4.py) introducing increasing platform spacing and moving hazards.
Core Engineering Modules & Workflows
Custom environment exposing standardized reset() and step(action) interfaces returning numerical observation arrays and rewards.
Encodes player position, velocity (dx, dy), jump state, nearest candidate platforms, and landing zone coordinates.
Horizontal navigation (Stay, Left, Right) combined with binary jump execution.
Penalizes idle hovering, severely penalizes fatal descents, and rewards sustained vertical altitude advancement.
Balances random exploratory jumps in early episodes with learned Q-policy exploitation in mature training stages.
Persistence Schemas & Execution Contracts
Database Schema & Entity Structure
Observation Vector Structure: [player_x, player_y, player_vx, player_vy, can_jump, dist_plat_1_x, dist_plat_1_y, dist_plat_2_x, dist_plat_2_y, dist_plat_3_x, dist_plat_3_y, nearest_hazard_dist] Trained Model Checkpoints: └── models/GoUp_DQN_Model.pt (Trained PyTorch Policy Tensor)
Key Endpoints & Query Interfaces
Engineering Challenges & Specifications
Technical Obstacles Solved
Resolution: Maintained a secondary target network updated every 1,000 steps to decouple target Q-value calculation from active weight gradients.
Resolution: Engineered negative step decay penalties forcing the agent to prioritize upward momentum rather than loitering on initial platforms.
Technology Stack Matrix
| Category | Tooling & Frameworks |
|---|---|
| Language | Python 3.9+ |
| Deep Learning Library | PyTorch (torch, torch.nn) |
| RL Algorithm | Deep Q-Learning (DQN) with Replay Buffer |
| Game Simulation | Pygame |
| Environment Standard | Gym-Style API (reset, step, close) |