Effectiveness of Q-learning as a tool for calibrating agent-based supply network models

Authored by Y. Zhang, S. Bhattacharyya

Date Published: 2007

DOI: 10.1080/17517570701275390

Sponsors: No sponsors listed

Platforms: Swarm

Model Documentation: Other Narrative Flow charts Mathematical description

Model Code URLs: Model code not found

Abstract

This paper examines effectiveness of Q-learning as a tool for specifying agent attributes and behaviours in agent-based supply network models. Agent-based modelling (ABM) has been increasingly employed to study supply chain and supply network problems. A challenging task in building agent-based supply network models is to properly specify agent attributes and behaviours. Machine learning techniques, such as Q-learning, can be a useful tool for this purpose. Q-learning is a reinforcement learning technique that has been shown to be an effective adaptation and searching mechanism in distributed settings. In this study, Q-learning is employed by supply network agents to search for `optimal' values for a parameter in their operating policies simultaneously and independently. Methods are designed to identify the `optimal' parameter values against which effectiveness of the learning is evaluated. Robustness of the learning's effectiveness is also examined through consideration of different model settings and scenarios. Results show that Q-learning is very effective in finding the `optimal' parameter values in all model settings and scenarios considered.

Tags

Agent-based modelling computational techniques Distributed artificial intelligence (DAI) Q-learning supply chain management (SCM)