The tutorial is written for those who would like an introduction to reinforcement learning (RL). The aim is to provide an intuitive presentation of the ideas rather than concentrate on the deeper mathematics underlying the topic. RL is generally used to solve the so-called Markov decision problem (MDP). In other words, the problem that you are attempting to solve with RL should be an MDP or its variant. The theory of RL relies on dynamic programming (DP) and artificial intelligence (AI). We will begin with a quick description of MDPs. We will discuss what we mean by “complex” and “large-scale” MDPs. Then we will explain why RL is needed to solve complex and large-scale MDPs. The semi-Markov decision problem (SMDP) will also be covered.
The tutorial is meant to serve as an introduction to these topics and is based mostly on the book: “Simulation-based optimization: Parametric Optimization techniques and reinforcement learning” [4]. The book discusses this topic in greater detail in the context of simulators. There are at least two other textbooks that I would recommend you to read: (i) Neuro-dynamic programming [2] (lots of details on convergence analysis) and (ii) Reinforcement Learning: An Introduction [11] (lots of details on underlying AI concepts). A more recent tutorial on this topic is [8]. This tutorial has 2 sections: ? Section 2 discusses MDPs and SMDPs. ? Section 3 discusses RL. By the end of this tutorial, you should be able to ? Identify problem structures that can be set up as MDPs / SMDPs. ? Use some RL algorithms.
付費5元查看完整內容
相關內容
強化學習（RL）是機器學習的一個領域，與軟件代理應如何在環境中采取行動以最大化累積獎勵的概念有關。除了監督學習和非監督學習外，強化學習是三種基本的機器學習范式之一。
強化學習與監督學習的不同之處在于，不需要呈現帶標簽的輸入/輸出對，也不需要顯式糾正次優動作。相反，重點是在探索（未知領域）和利用（當前知識）之間找到平衡。
該環境通常以馬爾可夫決策過程（MDP）的形式陳述，因為針對這種情況的許多強化學習算法都使用動態編程技術。經典動態規劃方法和強化學習算法之間的主要區別在于，后者不假設MDP的確切數學模型，并且針對無法采用精確方法的大型MDP。
強化學習
·
強化一詞來源于實驗心理學中對動物學習的研究，它指的是某一事件的發生，與某一反應之間有恰當的關系，而這一事件往往會增加該反應在相同情況下再次發生的可能性。雖然心理學家沒有使用“強化學習”這個術語，但它已經被人工智能和工程領域的理論家廣泛采用，用來指代基于這一強化原理的學習任務和算法。最簡單的強化學習方法使用的是一個常識，即如果一個行為之后出現了一個令人滿意的狀態，或者一個狀態的改善，那么產生該行為的傾向就會得到加強。強化學習的概念在工程領域已經存在了幾十年(如Mendel和McClaren 1970)，在人工智能領域也已經存在了幾十年(Minsky 1954, 1961;撒母耳1959;圖靈1950)。然而，直到最近，強化學習方法的發展和應用才在這些領域占據了大量的研究人員。激發這種興趣的是兩個基本的挑戰:1) 設計能夠在復雜動態環境中在不確定性下運行的自主機器人代理，2) 為非常大規模的動態決策問題找到有用的近似解。
付費5元查看完整內容
深度強化學習
·
Deep Reinforcement Learning via Policy Optimization
付費5元查看完整內容
Meta-learning, or learning to learn, is the science of systematically observing how different machine learning approaches perform on a wide range of learning tasks, and then learning from this experience, or meta-data, to learn new tasks much faster than otherwise possible. Not only does this dramatically speed up and improve the design of machine learning pipelines or neural architectures, it also allows us to replace hand-engineered algorithms with novel approaches learned in a data-driven way. In this chapter, we provide an overview of the state of the art in this fascinating and continuously evolving field.
This paper presents the first two editions of Visual Doom AI Competition, held in 2016 and 2017. The challenge was to create bots that compete in a multi-player deathmatch in a first-person shooter (FPS) game, Doom. The bots had to make their decisions based solely on visual information, i.e., a raw screen buffer. To play well, the bots needed to understand their surroundings, navigate, explore, and handle the opponents at the same time. These aspects, together with the competitive multi-agent aspect of the game, make the competition a unique platform for evaluating the state of the art reinforcement learning algorithms. The paper discusses the rules, solutions, results, and statistics that give insight into the agents' behaviors. Best-performing agents are described in more detail. The results of the competition lead to the conclusion that, although reinforcement learning can produce capable Doom bots, they still are not yet able to successfully compete against humans in this game. The paper also revisits the ViZDoom environment, which is a flexible, easy to use, and efficient 3D platform for research for vision-based reinforcement learning, based on a well-recognized first-person perspective game Doom.
This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. It surveys the general formulation, terminology, and typical experimental implementations of reinforcement learning and reviews competing solution paradigms. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics, perhaps the simplest and best studied problem in optimal control. The manuscript describes how merging techniques from learning theory and control can provide non-asymptotic characterizations of LQR performance and shows that these characterizations tend to match experimental behavior. In turn, when revisiting more complex applications, many of the observed phenomena in LQR persist. In particular, theory and experiment demonstrate the role and importance of models and the cost of generality in reinforcement learning algorithms. This survey concludes with a discussion of some of the challenges in designing learning systems that safely and reliably interact with complex and uncertain environments and how tools from reinforcement learning and controls might be combined to approach these challenges.
注冊地址： 北京市海淀區羊坊店路18號2幢3層301-191