In addition to the basic algorithms like Q-Learning, I also included slightly more advanced ones like the Successor Representation. The tool lets you see the successor maps for each state as the agent moves through the environment, which I think makes them much more intuitive!

I spent some time last week vibe-coding "The Beginner's RL Playground" a simple web tool for exploring many basic concepts in reinforcement learning. Users can adjust all sorts of parameters of the algorithm and environment and see the effects of those changes in realtime.