org.jscience.computing.ai.reinforcement.qlearning
Class QLearner

java.lang.Object
  extended by org.jscience.computing.ai.reinforcement.qlearning.QLearner

public abstract class QLearner
extends java.lang.Object

Q-learning algorithm implementation. You should override this class for your requirements.


Field Summary
static int EpsinonGreedy
          "Epsinon Greedy" selection method for selecting actions
 
Constructor Summary
QLearner()
           
 
Method Summary
 java.util.Collection getBestActions()
          Returns all of the best actions for each states tried so far
 double getBestValue(java.lang.Object state)
          Returns best reinforcement value for given state
 double getDiscountFactor()
          Returns discount factor
 double getLearningRate()
          Returns learning rate
 int getNumberOfStates()
          Returns the number of different states in the q-table
 int getQTableSize()
          Returns the number of entries in q-table
 java.util.Enumeration getStates()
          Returns different states in the q-table.
 void initialize()
          Initializes the learner with default parameters.
 void initialize(double learningRate, double discountFactor, int selectionPolicy)
          Initializes the learner with given parameters.
 java.lang.Object selectAction(java.lang.Object state)
          Selects an action based on given state
protected abstract  java.lang.Object selectRandomAction(java.lang.Object state)
          Selects an action randomly.
 void setDefaultValue(int defaultValue)
          Changes default valee for q-table entries.
 void setDiscountFactor(double dicountFactor)
          Changes the discount factor
 void setLearningRate(double learningRate)
          Changes the learning rate
 void setSelectionPolicy(int selectionPolicy)
          Changes the selection method
 void updateQTable(java.lang.Object state, java.lang.Object action, java.lang.Object newState, double reward)
          Updates q-table with given state, action, new state and reward information
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

EpsinonGreedy

public static final int EpsinonGreedy
"Epsinon Greedy" selection method for selecting actions

See Also:
Constant Field Values
Constructor Detail

QLearner

public QLearner()
Method Detail

initialize

public void initialize()
Initializes the learner with default parameters. Default parameters are listed below:

learningRate = 0.2; discountFactor = 0.9; epsinon = 0.75; defaultValue = 0.0; selectionPolicy = QLearner.EpsinonGreedy;


initialize

public void initialize(double learningRate,
                       double discountFactor,
                       int selectionPolicy)
Initializes the learner with given parameters.

Parameters:
learningRate - learning rate
discountFactor - discount factor
selectionPolicy - selection method

setLearningRate

public void setLearningRate(double learningRate)
Changes the learning rate

Parameters:
learningRate - new learning rate

setDiscountFactor

public void setDiscountFactor(double dicountFactor)
Changes the discount factor

Parameters:
dicountFactor - new discount factor

setSelectionPolicy

public void setSelectionPolicy(int selectionPolicy)
Changes the selection method

Parameters:
selectionPolicy - new selection method

setDefaultValue

public void setDefaultValue(int defaultValue)
Changes default valee for q-table entries.

Parameters:
defaultValue - new default value

getLearningRate

public double getLearningRate()
Returns learning rate

Returns:
learning rate

getDiscountFactor

public double getDiscountFactor()
Returns discount factor

Returns:
discount factor

getQTableSize

public int getQTableSize()
Returns the number of entries in q-table

Returns:
size of the q-table

getNumberOfStates

public int getNumberOfStates()
Returns the number of different states in the q-table

Returns:
number of different states in the q-table

getStates

public java.util.Enumeration getStates()
Returns different states in the q-table.

Returns:
different states in the q-table

getBestActions

public java.util.Collection getBestActions()
Returns all of the best actions for each states tried so far

Returns:
all of the best actions

getBestValue

public double getBestValue(java.lang.Object state)
                    throws QTableEntryNotFoundException
Returns best reinforcement value for given state

Parameters:
state - state to look for best reinforcement value
Returns:
best reinforcement value for given state
Throws:
QTableEntryNotFoundException - if the state is not found in q-table

updateQTable

public void updateQTable(java.lang.Object state,
                         java.lang.Object action,
                         java.lang.Object newState,
                         double reward)
Updates q-table with given state, action, new state and reward information

Parameters:
state - state before the action is done
action - action performed for given reward
newState - new state observed after the action done
reward - reinforcement taken after the action is done

selectAction

public java.lang.Object selectAction(java.lang.Object state)
Selects an action based on given state

Parameters:
state - state to select action for
Returns:
selected action

selectRandomAction

protected abstract java.lang.Object selectRandomAction(java.lang.Object state)
Selects an action randomly. This method should be implemented in subclasses, since possible actions are not known by QLearner class.

Parameters:
state - state to select an action randomly
Returns:
selected action