Reinforcement_learning

# Reinforcement learning ## Microsim (three.js) <div class="microsim-player">  <p class="wt-pending"><strong>Microsim staged, not yet on the CDN.</strong> <code>Reinforcement_learning.html</code> is built and deploy-ready in <code>Microsims for Dissemination/</code>, but the Netlify project still serves the geometry+spintronics set only. The player is disabled until the deploy lands; the explanatory text below is unchanged.</p>   </div> *In the ALGORITHM hub, reinforcement learning is where [[Dynamic_programming]] meets trial and error: an agent bootstraps an [[Optimal_control]] policy from sampled rewards instead of a known model, refining estimates of [[Expected_value]] one step at a time.* > Reinforcement learning (RL) is the branch of machine learning in which an agent learns how to act by interacting with an environment and receiving scalar rewards, with no labelled examples of the correct move. Q-learning, the algorithm this microsim animates, keeps a table of action values $Q(s,a)$ — the long-run reward expected from taking action $a$ in cell $s$ — and sharpens them from experience. In the grid world you watch an agent shuttle between a start and a goal while each cell's arrows brighten as reward information spreads backward from the goal. The controls let you set how boldly the agent trusts new experience (learning rate $\alpha$) and how often it gambles on a random move (exploration rate $\varepsilon$), then watch the value map converge. ## About this microsim The canvas is a small grid world — a start cell, a goal that pays a reward, a penalty cell or two — with each cell's four action values drawn as arrows. Press **Play** and the agent runs episodes, a trail marking its path as the arrows fill in; **Step** advances one action so you can read the update on a single cell, and **Play speed** (80–1200) sets the delay between steps without changing what is learned. Push **exploration rate ε** toward 1 and the agent moves almost at random, charting the grid but committing to nothing; lower it and it exploits the best route so far, sometimes locking onto a poor one before the goal's value has spread. Raise **learning rate α** and each cell lurches toward its newest estimate — fast but jittery; drop it and values settle slowly. **Reset** clears the table. The lesson is visible before it is stated: value ripples outward from the goal, and the exploration–exploitation trade-off is something you can watch stall or succeed. ## Related microsims - [[Dynamic_programming]] — Q-learning is a sampled, model-free version of the same Bellman backups. - [[Monte_Carlo_method]] — an alternative that averages full-episode returns instead of bootstrapping one step. - [[Expected_value]] — each Q-value is the expected discounted return of an action. - [[Optimal_control]] — RL is the learning-based counterpart to classical optimal control. - Dopamine — the temporal-difference error models the reward-prediction-error signal of dopamine neurons. - Convolutional neural network — deep RL uses CNNs to read raw pixel states. - [[Calculus_of_variations]] — related ALGORITHM microsim - [[Combinatorial_optimization]] — related ALGORITHM microsim - [[Inductive_reasoning]] — related ALGORITHM microsim - [[Model_predictive_control]] — related ALGORITHM microsim - Process optimization — related ALGORITHM microsim ## Links (Wikipedia order)  `15.ai` · `1X_Technologies` · `AAAI_Conference_on_Artificial_Intelligence` · `ABB` · `ACM_Computing_Classification_System` · `ACT-R` · `AI_agent` · `AI_alignment` · `AI_anthropomorphism` · `AI_boom` · `AI_bubble` · `AI_data_center` · `AI_effect` · `AI_infrastructure` · `AI_literacy` · `AI_nationalism` · `AI_safety` · `AI_slop` · `AI_takeover` · `AI_warfare` · `AI_winter` · `Action_selection` · `Activation_function` · `Active_learning_(machine_learning)` · `Adaptable_robotics` · `Adobe_Firefly` · `Adversarial_machine_learning` · `Aerobot` · `Agent2Agent` · `AgiBot` · `Agility_Robotics` · `Agricultural_robot` · `Aidan_Gomez` · `Alan_Turing` · `AlexNet` · `Alex_Graves_(computer_scientist)` · `Alex_Krizhevsky` · [[Algorithm]] · [[Algorithmic_efficiency]] · `Allen_Newell` · `AlphaFold` · `AlphaGo` · `AlphaZero` · `Amazon_Robotics` · `Analysis_of_algorithms` · `Andrej_Karpathy` · `Andrew_Barto` · `Andrew_Ng` · `Android_(robot)` · `Animatronics` · `Anomaly_detection` · `Anybots` · `Application_security` · `Applications_of_artificial_intelligence` · `Apprenticeship_learning` · `Articulated_robot` · `Artificial_Intelligence_Act` · `Artificial_Intelligence_Cold_War` · `Artificial_general_intelligence` · `Artificial_human_companion` · [[Artificial_intelligence]] · `Artificial_intelligence_and_elections` · `Artificial_intelligence_arms_race` · `Artificial_intelligence_in_architecture` · `Artificial_intelligence_in_education` · `Artificial_intelligence_in_fiction` · `Artificial_intelligence_in_government` · `Artificial_intelligence_in_healthcare` · `Artificial_intelligence_in_marketing` · `Artificial_intelligence_in_music` · `Artificial_intelligence_in_video_games` · `Ashish_Vaswani` · `Association_rule_learning` · `Attention_(machine_learning)` · `Audio-Animatronics` · [[Augmented_reality]] · `AutoGPT` · `Autoencoder` · `Automata_theory` · `Automated_machine_learning` · `Automated_planning_and_scheduling` · `Automated_reasoning` · `Automated_restaurant` · `Automated_retail` · `Automated_theorem_proving` · `Automatic_summarization` · `Automaton` · `Autonomous_underwater_vehicle` · `Autoregressive_model` · `BEAM_robotics` · `BIRCH` · `BLEU` · `Backgammon` · `Backpropagation` · `Barrett_Technology` · `Basal_ganglia` · `Batch_normalization` · `Bayesian_network` · `Bellman_equation` · `Bernard_Widrow` · `Bias–variance_tradeoff` · `Biorobotics` · `Boltzmann_machine` · `Boosting_(machine_learning)` · `Bootstrap_aggregating` · `Boston_Dynamics` · `Brute-force_search` · `CLARION_(cognitive_architecture)` · `CURE_algorithm` · `Canonical_correlation` · `ChatGPT` · `Chatbot_psychosis` · `Checkers` · `Christopher_D._Manning` · [[Claude_Shannon]] · `Cliff_Shaw` · `Closed-form_expression` · `Cloud_robotics` · `Cluster_analysis` · `Coefficient_of_determination` · `Cognitive_architecture` · [[Communication_protocol]] · `Companion_robot` · `Competition_in_artificial_intelligence` · `Competitions_and_prizes_in_artificial_intelligence` · `Computability_theory` · `Computational_biology` · `Computational_chemistry` · `Computational_complexity` · `Computational_complexity_theory` · `Computational_engineering` · `Computational_geometry` · `Computational_intelligence` · `Computational_learning_theory` · `Computational_mathematics` · `Computational_physics` · `Computational_problem` · `Computational_propaganda` · `Computational_social_science` · `Computer_accessibility` · `Computer_animation` · [[Computer_architecture]] · `Computer_data_storage` · `Computer_graphics` · [[Computer_hardware]] · `Computer_network` · [[Computer_science]] · `Computer_security` · `Computer_vision` · `Computing` · `Computing_platform` · `Concurrency_(computer_science)` · `Concurrent_computing` · `Conditional_random_field` · `Conference_on_Computer_Vision_and_Pattern_Recognition` · `Conference_on_Neural_Information_Processing_Systems` · `Confusion_matrix` · `Conjugate_gradient_method` · `Constraint_satisfaction_problem` · `Continuous_track` · `Continuum_robot` · `Control_flow` · [[Control_theory]] · [[Convolution]] · `Convolutional_neural_network` · `Critique_of_work` · `Cross-entropy_method` · `Cross-validation_(statistics)` · `Crowdsourcing` · `Cryptography` · `Curriculum_learning` · `Cyber-physical_system` · `Cyberwarfare` · `Cyborg` · `DALL-E` · `DBSCAN` · `DBpedia` · `Daniel_Kokotajlo_(researcher)` · `Data_augmentation` · `Data_mining` · `Database` · `David_Silver_(computer_scientist)` · `Daxbot` · `Decision_support_system` · `Decision_tree_learning` · `Deductive_classifier` · `DeepDream` · `DeepSeek_(chatbot)` · `Deep_Robotics` · `Deep_learning` · `Deep_learning_speech_synthesis` · `Deep_reinforcement_learning` · `Delivery_robot` · `Demis_Hassabis` · `Density_estimation` · [[Dependability]] · `Developmental_robotics` · `Dialogue_system` · `Differentiable_neural_computer` · `Diffusion_model` · `Diffusion_process` · `Digital_art` · [[Digital_library]] · `Digital_marketing` · `Dimensionality_reduction` · `Discrete_mathematics` · `Distributed_artificial_intelligence` · `Distributed_computing` · `Document_management_system` · `Domain-specific_language` · `Domestic_robot` · `Doosan_Robotics` · `Dopamine` · `Dopaminergic` · `Double_descent` · `Dream_Machine_(text-to-video_model)` · [[Dynamic_programming]] · `E-commerce` · `ECML_PKDD` · `Echo_state_network` · `Educational_robotics` · `Educational_technology` · `Electric_unicycle` · `Electrochemical_RAM` · `Electronic_design_automation` · `Electronic_publishing` · `Electronic_voting` · `ElevenLabs` · `Embedded_system` · `Empirical_Methods_in_Natural_Language_Processing` · `Empirical_risk_minimization` · `Energid_Technologies` · `Energy_storage` · `Ensemble_learning` · `Enterprise_information_system` · `Enterprise_software` · `Entertainment_robot` · `Ethics_of_artificial_intelligence` · `European_Conference_on_Computer_Vision` · [[Evolutionary_computation]] · [[Evolutionary_robotics]] · `Expectation–maximization_algorithm` · `Expected_shortfall` · [[Expert_system]] · `Explainable_artificial_intelligence` · `Exploration–exploitation_dilemma` · `Exploratory_factor_analysis` · `Extended_reality` · `FANUC` · `Facial_recognition_system` · `Factor_analysis` · `FarmWise` · `Fastbrick_Robotics` · [[Fault_tolerance]] · `Feature_engineering` · `Feature_learning` · `Feedforward_neural_network` · `Fei-Fei_Li` · `Few-shot_learning` · `Figure_AI` · `Flux_(text-to-image_model)` · `Form_factor_(design)` · `Formal_language` · `Formal_methods` · `Foster-Miller` · `Fourier_(company)` · `Frank_Rosenblatt` · `François_Chollet` · `Fuzzy_clustering` · [[Fuzzy_control_system]] · `Fuzzy_rule` · `GPT_Image` · [[Game_theory]] · `Gated_recurrent_unit` · `Gating_mechanism` · `General_game_playing` · `Generative_AI` · `Generative_adversarial_network` · `Generative_engine_optimization` · `Generative_model` · `Genie_(world_model)` · `Geoffrey_Hinton` · `Geographic_information_system` · `Geography_of_robotics` · `Glossary_of_artificial_intelligence` · `Glossary_of_robotics` · `Go_(game)` · `Gradient` · `Gradient_descent` · `Grammar_induction` · `Graph_neural_network` · `Graphical_model` · [[Graphics_processing_unit]] · `Green_computing` · `Gynoid` · `HD_Hyundai_Robotics` · `Hallucination_(artificial_intelligence)` · `Handwriting_recognition` · `Hardware_acceleration` · `Hardware_security` · `Harvest_Automation` · `Health_informatics` · [[Herbert_A._Simon]] · `Hexapod_(robotics)` · `Hidden_Markov_model` · `Hierarchical_clustering` · `Highway_network` · `History_of_artificial_intelligence` · `History_of_robots` · `Honeybee_Robotics` · `Human-centered_computing` · `Human-in-the-loop` · `Human_image_synthesis` · `Humanity's_Last_Exam` · `Humanoid_robot` · `Human–computer_interaction` · `Human–robot_interaction` · `Hyperparameter_(machine_learning)` · `IBM_Watson` · `IBM_Watsonx` · `IRobot` · `Ian_Goodfellow` · `Ideogram_(text-to-image_model)` · `Ilya_Sutskever` · [[Image_compression]] · `Imagen_(text-to-image_model)` · `Imitation_learning` · `Independent_component_analysis` · `Index_of_robotics_articles` · [[Industrial_process_control]] · `Industrial_robot` · `Inference_engine` · `Information_retrieval` · `Information_security` · [[Information_system]] · [[Information_theory]] · `Integrated_circuit` · `Integrated_development_environment` · `Intelligent_agent` · `Interaction_design` · `International_Conference_on_Computer_Vision` · `International_Conference_on_Learning_Representations` · `International_Conference_on_Machine_Learning` · `International_Joint_Conference_on_Artificial_Intelligence` · `Interpreter_(computing)` · `Intrusion_detection_system` · `Intuitive_Surgical` · `Isolation_forest` · `James_Goodnight` · `Jan_Leike` · `John_Hopfield` · `John_McCarthy_(computer_scientist)` · `John_Schulman` · [[John_von_Neumann]] · `Joseph_Weizenbaum` · `Journal_of_Machine_Learning_Research` · `Juggling_robot` · `Jürgen_Schmidhuber` · `K-means_clustering` · `K-nearest_neighbors_algorithm` · `KUKA` · `Kling_AI` · `Knowledge_base` · `Knowledge_representation_and_reasoning` · `Kunihiko_Fukushima` · `LIDA_(cognitive_architecture)` · `Language_model` · `Large_language_model` · `Latent_diffusion_model` · `Laws_of_robotics` · `LeNet` · `Learning_curve_(machine_learning)` · `Learning_to_rank` · `Legged_robot` · `Lethal_autonomous_weapon` · `Library_(computing)` · `Linear_discriminant_analysis` · `Linear_regression` · `List_of_artificial_intelligence_companies` · `List_of_artificial_intelligence_projects` · `List_of_computer_size_categories` · `List_of_datasets_for_machine-learning_research` · `List_of_datasets_in_computer_vision_and_image_processing` · `List_of_fictional_robots_and_androids` · `List_of_large_language_models` · `List_of_robotics_companies` · `List_of_robotics_software` · [[Lists_of_open-source_artificial_intelligence_software]] · `Local_outlier_factor` · `Local_search_(optimization)` · `Logic_in_computer_science` · `Logistic_regression` · `Long_short-term_memory` · `Loss_functions_for_classification` · `Lotfi_A._Zadeh` · `Machine_Learning_(journal)` · [[Machine_learning]] · `Machine_translation` · `Mamba_(deep_learning_architecture)` · `Markov_chain` · `Markov_decision_process` · `Marvin_Minsky` · `Mathematical_analysis` · [[Mathematical_optimization]] · [[Mathematical_software]] · `Mathematics_of_Operations_Research` · `Mean_shift` · `Mechanistic_interpretability` · `Medical_robot` · `Memtransistor` · `Meta-learning_(computer_science)` · `Michael_L._Littman` · `Microbotics` · `Middleware` · `Midjourney` · `Military_applications_of_artificial_intelligence` · `Military_robot` · `Mobile_computing` · `Mobile_robot` · `Model_Context_Protocol` · `Model_of_computation` · [[Model_predictive_control]] · `Modeling_language` · [[Monte_Carlo_method]] · `Monte_Carlo_tree_search` · `Moravec's_paradox` · `Motion_planning` · `MuZero` · `Multi-agent_reinforcement_learning` · [[Multi-agent_system]] · `Multi-armed_bandit` · `Multi-objective_optimization` · `Multi-task_learning` · `Multilayer_perceptron` · `Multimedia_database` · `Multimodal_learning` · [[Multiprocessing]] · [[Multithreading_(computer_architecture)]] · `Mustafa_Suleyman` · `Naive_Bayes_classifier` · `Nanorobotics` · `Nathaniel_Rochester_(computer_scientist)` · `Natural_language_processing` · `Necrobotics` · `Network_architecture` · `Network_performance` · `Network_scheduler` · `Network_security` · `Network_service` · `Networking_hardware` · `Neura_Robotics` · `Neural_Turing_machine` · `Neural_field` · `Neural_machine_translation` · [[Neural_network_(machine_learning)]] · `Neural_radiance_field` · `Neuro-symbolic_AI` · `Noam_Shazeer` · `Non-negative_matrix_factorization` · `Nonparametric_statistics` · `Normalization_(machine_learning)` · `Numerical_analysis` · `OPTICS_algorithm` · `Oasis_(Minecraft_clone)` · `Occam_learning` · `Oliver_Selfridge` · `Online_machine_learning` · `Ontology_learning` · `Open-source_robotics` · `OpenAI` · `OpenAI_Five` · `OpenAI_o1` · `OpenCog` · `Open_source` · `Operant_conditioning` · `Operating_system` · [[Operations_research]] · `Optical_character_recognition` · [[Optimal_control]] · `Oriol_Vinyals` · `Outline_of_computer_science` · `Outline_of_machine_learning` · `Outline_of_robotics` · `Overfitting` · `Parallel_computing` · `Parameter` · `Partially_observable_Markov_decision_process` · `Paul_Werbos` · `Perceptron` · `Perceptual_robotics` · `Peripheral` · `Permutation_test` · `Philosophy_of_artificial_intelligence` · `Photograph_manipulation` · `Photovoltaic_system` · `Physics-informed_neural_networks` · `Policy_gradient_method` · `Post-training_of_large_language_models` · `Precautionary_principle` · `Prentice_Hall` · `Principal_component_analysis` · `Printed_circuit_board` · [[Probability]] · `Probably_approximately_correct_learning` · `Procedural_reasoning_system` · `Processor_(computing)` · `Programming_language` · `Programming_language_theory` · `Programming_paradigm` · `Programming_team` · `Programming_tool` · `Project_Debater` · `Prompt_engineering` · `Proper_generalized_decomposition` · `Q-learning` · [[Quantum_computing]] · `Quantum_machine_learning` · `Quasi-Newton_method` · `Quoc_V._Le` · `Rainbow_Robotics` · `Random_forest` · `Random_sample_consensus` · `Random_utility_model` · `Randomized_algorithm` · [[Real-time_computing]] · `Reasoning_model` · `Reasoning_system` · `Receiver_operating_characteristic` · `Recraft` · `Recurrent_neural_network` · `Recursive_self-improvement` · `Regression_analysis` · `Regret_(decision_theory)` · `Regularization_(mathematics)` · `Regulation_of_artificial_intelligence` · `Regulation_of_artificial_intelligence_in_the_United_States` · `Reinforcement` · `Reinforcement_learning_from_human_feedback` · `Relevance_vector_machine` · `Remotely_operated_underwater_vehicle` · `Rendering_(computer_graphics)` · `Requirements_analysis` · `Reservoir_computing` · `Residual_neural_network` · `Restricted_Boltzmann_machine` · `Retrieval-augmented_generation` · `Richard_S._Sutton` · `Riffusion` · `Robomow` · `Roborock` · `Robot` · `Robot_Hall_of_Fame` · `Robot_competition` · `Robot_control` · `Robot_ethics` · `Robot_kit` · `Robot_locomotion` · `Robot_software` · `Robotic_arm` · `Robotic_mapping` · `Robotic_paradigm` · `Robotic_tech_vest` · [[Robotics]] · `Robotics_simulator` · `Robotics_suite` · `Rule-based_machine_learning` · `Rule-based_system` · `Runway_(company)` · `Security_hacker` · `Security_service_(telecommunication)` · `Seedance_2.0` · `Self-driving_car` · `Self-organizing_map` · `Self-supervised_learning` · `Semantic_analysis_(machine_learning)` · `Semantic_reasoner` · `Seppo_Linnainmaa` · `Service_robot` · `Seymour_Papert` · `Shun'ichi_Amari` · `Sigmoid_function` · `Simulated_annealing` · `Simulation` · `Simulation-based_optimization` · `Simultaneous_localization_and_mapping` · `Situated_robotics` · `Social_computing` · `Social_software` · `Soft_robotics` · `Softmax_function` · `Software_configuration_management` · `Software_construction` · `Software_deployment` · `Software_design` · `Software_development` · `Software_development_process` · [[Software_engineering]] · `Software_framework` · `Software_maintenance` · [[Software_quality]] · `Software_repository` · `Solid_modeling` · `Sora_(text-to-video_model)` · `Space_probe` · `Sparse_dictionary_learning` · `Speech_recognition` · `Spiking_neural_network` · `Stable_Diffusion` · `Starship_Technologies` · `State–action–reward–state–action` · `Statistical_classification` · `Statistical_learning_theory` · `Statistics` · `Stephen_Grossberg` · `Stochastic_computing` · `Stochastic_gradient_descent` · `Stochastic_optimization` · `Structured_prediction` · `Student's_t-test` · `Stäubli` · `Substantia_nigra` · `Suno_(platform)` · `Supervised_learning` · `Support_vector_machine` · `Swarm_intelligence` · `Swarm_robotics` · `Symbolic_artificial_intelligence` · `Symbotic` · `System_on_a_chip` · `T-distributed_stochastic_neighbor_embedding` · `Takeo_Kanade` · `Technological_unemployment` · `Telerobotics` · `Temporal_difference_learning` · `Terrainability` · `Text-to-image_model` · `Text-to-video_model` · `Theoretical_computer_science` · `Theory_of_computation` · `Timeline_of_artificial_intelligence` · `Topological_deep_learning` · `Training,_validation,_and_test_data_sets` · `Transfer_learning` · `Transformer_(deep_learning)` · `U-Net` · `UBtech_Robotics` · `Ubiquitous_computing` · `Ubiquitous_robot` · `Udio` · `Uncanny_valley` · `Unitree_Robotics` · `Universal_Robotics` · `Universal_Robots` · `Unmanned_aerial_vehicle` · `Unmanned_ground_vehicle` · `Unsupervised_learning` · `Value_function` · `Vapnik–Chervonenkis_theory` · `Variational_autoencoder` · `Veo_(text-to-video_model)` · [[Very-large-scale_integration]] · `Vibe_coding` · `Video_game` · `Virtual_machine` · `Virtual_politician` · [[Virtual_reality]] · `Vision-guided_robot_systems` · `Vision_transformer` · `Visual_odometry` · `Visualization_(graphics)` · `Walter_Pitts` · `Warren_Sturgis_McCulloch` · `WaveNet` · `Waymo` · `Weak_artificial_intelligence` · `Weight_initialization` · `Welltec` · `Whisper_(speech_recognition_system)` · `White_Box_Robotics` · `Wikidata` · `Wireless_sensor_network` · `Wolf_Robotics` · `Word_embedding` · `Word_processor` · `Workplace_impact_of_artificial_intelligence` · `Workplace_robotics_safety` · `World_Wide_Web` · `World_model_(artificial_intelligence)` · `YAGO_(database)` · `Yann_LeCun` · `Yaskawa_Electric_Corporation` · `Yoshua_Bengio` · `Zero-shot_learning` · `Zoox` ## Overview RL formalizes goal-directed learning as a Markov decision process: states, available actions, a transition rule, and a reward signal, with the agent seeking a policy $\pi$ that maximizes cumulative discounted reward. It descends from two lineages — the psychology of animal learning (Edward Thorndike's law of effect) and Richard Bellman's dynamic programming of the 1950s. Temporal-difference learning, introduced by Richard Sutton in 1988, let agents learn from incomplete episodes by bootstrapping; Christopher Watkins's Q-learning (1989), proved convergent by Watkins and Dayan (1992), joined this idea to off-policy control. The field drew wide attention when DeepMind's Deep Q-Network learned Atari games from raw pixels (2015) and AlphaGo defeated a human Go champion (2016). Tabular Q-learning endures as the canonical first example: its update is a single interpretable line. ## The algorithm After taking action $a$ in state $s$, observing reward $r$ and next state $s'$, Q-learning updates $Q(s,a) \leftarrow Q(s,a) + \alpha\big[\,r + \gamma\max_{a'}Q(s',a') - Q(s,a)\,\big].$ Here $\alpha\in(0,1]$ is the learning rate, $\gamma\in[0,1]$ the discount factor, and the bracketed **temporal-difference error** measures how surprising the outcome was. Because the target uses $\max_{a'}Q(s',a')$ — the best next action, not the one taken — Q-learning is *off-policy*: it learns the optimal values while exploring with another policy. Actions are picked $\varepsilon$-greedily: with probability $1-\varepsilon$ take $\arg\max_a Q(s,a)$, otherwise a random action. Given enough exploration and a suitably decaying $\alpha$ (the Robbins–Monro conditions $\sum\alpha_t=\infty$, $\sum\alpha_t^2<\infty$), the table converges to the optimal $Q^*$, whose greedy policy $\pi^*(s)=\arg\max_a Q^*(s,a)$ is optimal. With $\gamma=0.9$ and $\alpha=0.5$, a step onto the goal (reward 1) lifts the adjacent cell to $0.5$, then $0.75$ on the next visit, climbing toward $1$; neighbors grow only once that cell is non-zero — the ripple the sim draws. ## Controls -> what each maps to | Control | Maps to | Range / values | Meaning | |---|---|---|---| | Learning rate α | Step size in the update | 0.05–1 | How much each new experience overwrites the old estimate; high is fast but noisy, low is smooth but slow. | | Exploration rate ε | ε-greedy randomness | 0–1 | Probability of a random action instead of the greedy one; sets the exploration–exploitation balance. | | Play speed | Delay between steps | 80–1200 | Animation pace only; it does not affect what is learned. | | Step | One action + one update | button | Advances a single time step so a lone Q update can be inspected. | | Play | Run continuously | button | Auto-steps episodes at the chosen speed. | | Reset | Clear the Q-table | button | Returns all values to their initial state to relearn from scratch. | ## Learning objective After playing, a learner can predict how the learning rate and exploration rate change the speed, stability, and optimality of the Q-table as reward propagates outward from the goal. ## Limits and connections Tabular Q-learning stores one entry per state–action pair, so it suits only small discrete worlds like this grid; images or continuous control need function approximation, as in Deep Q-Networks. The discount factor and reward layout are fixed here yet strongly shape behavior, and $\varepsilon$-greedy is the simplest of many exploration schemes. ## Poster & source <div class="microsim-fallback">  <p><em>Live microsim · <a href="https://wikitube-3d-microsims.netlify.app/Reinforcement_learning.html">open full</a> · source: Microsims for Dissemination/ALGORITHM_microsims/Reinforcement_learning.html</em></p> </div>  *Built to the [[WT!Three_js_Microsim_Master_Class|three.js Master Class]].*  ## Wikipedia : Wikitube **Strict pair:** [Wikipedia](https://en.wikipedia.org/wiki/Reinforcement_learning) : [Wikitube](https://en.wikitube.io/wiki/Reinforcement_learning) ## Previous hub tags Tree parents: [[Game_theory]] · [[Graph_theory]] · [[Information_theory]]. Legacy hubs: `ALGORITHM`. --- *Sources: 1 legacy note. Minted wave 1, 2026-07-30 (v1.6 order).*