MCTS, neural networks, and building mini-AlphaZero.
Login required to access full content.

Minimax & Alpha-Beta (bridging classic AI and RL)

• Why pure RL struggles with deep tactics
• Why humans and engines search
• Teach minimax, alpha-beta
• Connect to MCTS logic (search tree with value estimates)
• Very important bridge: AlphaZero = RL + MCTS + neural nets

Introduction to policy/value functions

• States → value
• States → probability distribution over moves
• Explain why tabular methods fail in full chess
• Show that we need function approximation
• Introduce NNEU concept here:

What is NNEU?

– neural network evaluation unit

– a neural net replacing hand-coded evaluation

– outputs value (who is better)

– outputs policy (what moves are likely good)

The students don’t need deep math; only intuition

Deep Q-Learning (DQN) with simple games

• Use Snake or Catch
• Teach replay buffer, target network
• Show why DQN does NOT work for chess (action space too large), preparing AlphaZero.

Policy gradient & actor-critic basics

(Short overview, not too mathematical)

• Policy gradient: learn policy directly
• Actor-critic: policy (actor) + value estimate (critic)
• Prepare students for AlphaZero’s architecture (policy + value output).

Monte Carlo Tree Search (full lesson)

• Expand nodes
• Simulate
• Backup values
• Choose action using visit counts
• Show how policy prior improves MCTS
• Show how MCTS improves the policy network → the AlphaZero training loop emerges

Putting it all together: the AlphaZero algorithm

• Self-play generates games
• MCTS guided by neural net creates improved policy
• Neural net trained on (state, policy_targets, value)
• Iteration of self-play → training → stronger MCTS → stronger games
• Show how AlphaZero solves simple mini-chess (4×4, 5×5)
• Connect back to your 2-rook checkmate lesson

Students now understand:

– how RL can solve small tasks

– why full chess needs powerful NNEUs, MCTS, and self-play

Build a mini AlphaZero for Connect-4 or MiniChess

• Connect-4 board is perfect for classroom AlphaZero demo
• Implement:

– neural net (small CNN or MLP)

– MCTS

– self-play training

Lc0 is a direct open-source implementation inspired by DeepMind's AlphaZero project, and it has far exceeded its success in chess.
https://lczero.org/