Reinforcement learning: Bellman updates, rewards and Q-Learning
How an agent learns from states, actions and rewards, including exploration and safe simulation.
How it differs from supervised learning
Reinforcement learning does not provide a labeled correct answer for every decision. An agent chooses an action in a state, then observes a reward and the next state. The objective is long-term cumulative return, not just immediate reward. For a navigation robot, reaching the target and avoiding collisions both depend on reward design.
The Bellman intuition
The value of a decision combines immediate reward with expected future value. Q-Learning uses sample-based updates: Q(s,a) ← Q(s,a) + α [r + γ max Q(s',a') − Q(s,a)]. Here α controls the update step, γ discounts future returns and the bracketed term is a temporal-difference error.
Exploration versus exploitation
Always selecting the current best action may prevent discovery of better strategies; acting randomly forever wastes learning. Epsilon-greedy policies balance exploration with exploitation. Random actions can be harmful on physical systems, so exploration should start in simulation or under explicit safety constraints.
Test the reward, not just the score
Poorly designed rewards can incentivize unintended shortcuts. Track collisions, task time, safety violations and performance across different initial states, not only average return. A policy that succeeds in simulation may fail in the real world. Safe deployment needs monitoring, human oversight and a way to stop or reverse actions.
Comments
0 real commentsNo real reader comments yet.
✎ Write a comment