Delay-corrected Bellman operator + causal attribution for constrained RL contraction proof under unknown stochastic delay [R]

Excerpt

Standard constrained RL assumes consequences are immediate and attributable to the current action. This breaks down whenever violations are delayed and stochastic, which is most real-world settings you end up penalizing whatever action happened to precede the observed violation, not the action that caused it. Working on CCPL (Causal Consequence-Penalized Learning) to address this: - A delay-corrected Bellman operator using an adaptive effective discount learned from the consequence-delay distrib