BEFORE WE CAN FIND A MODEL, WE MUST FORGET ABOUT PERFECTION
look for a model of the world. Typically, we aim to find a model which tells
everything or almost everything. In other words, we hunt for a perfect model (a
total deterministic graph) or for an exhaustive model (MarkovDecisionProcess). Finding such a model
is an overly ambitious task and indeed a practically unsolvable problem with
complex worlds. In order to solve the problem, we will replace perfect and
exhaustive models with Event-Driven models.
Only young children assume
they can fully understand the world and give answers to all questions. We, the adults,
recognise that no one can understand everything and know everything out there.
For that reason, we are prepared to relinquish some of our excessive
The first of these is
that the model cannot be improved any further (an assumption known as Markov property).
Second, we will
abandon the assumption that the model is determinate – which we already did in MarkovDecisionProcess (MDP).Nevertheless, the
MDP assumes that whenever nondeterminism is present, each nondeterministic
branch occurs with exact probability. This is the third assumption we will say
farewell to. We will assume that the probability in question is not
precisely determined, but resides within a certain interval[a,
may also be that we know nothing about the probability, in which case the
interval is [0, 1].
Fourth, we will assume
that the model observes only some of the more important events rather than all
events. The MDP takes into account all actions of the agent. This makes it an
Action-Driven model. We will replace actions with events and will thus move to
an Event-Driven model. Event-Driven models are much more stable as they do not
change their state at each step, but only at the occurrence of one of the
Fifth, we will
dispense with having separate descriptions of the world and of the agent. Instead, we will describe
them as a composite system. The MDP provides only a description of the world
which does not include a description of the agent. There is nothing wrong about
separating the description of the world from that of the agent. This
separation, regretfully, would take a heavy toll on us and prevent our transition
to an Event-Driven model. Accordingly, we will discard that assumption, too.