《人工智能/人工神经元—强化学习方法解析 [英语授课]》

AI/ANN -Reinforcement Learning

【Background & Goals】

Reinforcement learning (RL) is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. The problem, due to its generality, is studied in many disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, statistics and genetic algorithms. In this course of lectures, reinforcement learning is being saw as approximate dynamic programming, The approach is studied in the theory of optimal control, though most studies are concerned with the existence of optimal solutions and their characterization, and not with learning or approximation. In machine learning, the environment is typically formulated as a Markov decision process (MDP), as many reinforcement learning algorithms for this context utilize dynamic programming techniques.

Reinforcement learning differs from standard supervised learning in that correct input/output pairs are never presented, nor sub-optimal actions explicitly corrected. Instead the focus is on performance, which involves finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge), The exploration vs. exploitation trade-off has been most thoroughly studied through the multi-armed bandit problem and in finite MDPs.

The main difference between the classical techniques and reinforcement learning algorithms is that the latter do not need knowledge about the MDP (Markov decision process) and they are able to target large MDPs where exact methods become infeasible.

【Trainees】

Programmers and managers engaged in AI/ANN - Reinforcement Learning applications and the managers of the relevant business functions.

Trainees need to have well-understanding to advanced  higher mathematics.

(受训学员必须具备现代高等数学良好基础)

【Timing】 6 class hours (6 Class hrs/day)

【General Content】

PART 1  Necessary & Essential AI Knowledge

PART 2  A smart Robot in a room ——Example

PART 3  Defining a Markov Decision Process

PART 4  Monte Carlo methods

PART 5  RL Substantializing & Strengthening ——Q-learning

【Detailed Content】

PART 1  Necessary & Essential AI Knowledge

1.1 Supervised learning

     classification, regression

1.2 Unsupervised learning

     clustering, dimensionality reduction

1.3 Reinforcement learning

     generalization of supervised learning

     learn from interaction w/ environment to achieve a goal

PART 2  A smart Robot in a room ——Example

What’s the strategy to achieve max reward?

What if the actions were deterministic?

No teacher who would say “good” or “bad”

Explore the environment and learn from the experience

PART 3  Defining a Markov Decision Process

3.1 solving an MDP using Dynamic Programming

states, actions and rewards

solution and policy

Markov Decision Process (MDP)

maximize cumulative reward in the long run

Computing return from rewards

3.2 Value functions

Optimal value functions

Policy evaluation/improvement

Policy/Value iteration

PART 4  Monte Carlo methods

4.1 Monte Carlo methods

don’t need full knowledge of environment

averaging sample returns

4.2 Monte Carlo policy evaluation

want to estimate Vp(s)

first-visit MC

4.3 Monte Carlo control

4.4 Maintaining exploration

4.5 Simulated experience

4.6 Summary of Monte Carlo

PART 5  RL Substantializing & Strengthening ——Q-learning

5.1 off-policy learning

5.2 State representation

5.3 Function approximation

5.4 Features

5.5 Splitting and aggregation

5.6 Designing rewards

5.7 Case study: Back gammon