Undefined

A partially observable Markov decision process (POMDP) is a generalization of a Markov decision process in which the agent cannot directly observe the underlying state of the environment and must act on the basis of uncertain observations. A POMDP is formally a tuple ⟨S, A, T, R, Ω, O⟩ — states, actions, transition model, reward function, observation set, and observation model — and the agent summarizes its action–observation history in a belief state, a probability distribution over states that serves as a sufficient statistic. Variants add a finite time horizon, a discount factor, and an initial belief, giving the extended tuple ⟨h, S, A, Ω, T, O, R, γ, b₀⟩.

One interaction step of a POMDP: the policy π acts on the belief b, and a Bayes filter update τ incorporates the action and the new observation (Lauri et al., 2023).