Reinforcement learning functions. More...

#include "xcsf.h"

Include dependency graph for xcs_rl.h:

This graph shows which files directly or indirectly include this file:

Functions
double	xcs_rl_error (struct XCSF *xcsf, const int action, const double reward, const bool reset, const double max_p)
	Returns the reinforcement learning system prediction error. More...

double	xcs_rl_exp (struct XCSF *xcsf)
	Executes a reinforcement learning experiment. More...

int	xcs_rl_decision (struct XCSF xcsf, const double state)
	Selects an action to perform in a reinforcement learning problem. More...

void	xcs_rl_end_step (struct XCSF xcsf, const double state, const int action, const double reward)
	Ends a step in a reinforcement learning trial. More...

void	xcs_rl_end_trial (struct XCSF *xcsf)
	Frees memory used by a reinforcement learning trial. More...

void	xcs_rl_init_step (struct XCSF *xcsf)
	Initialises a step in a reinforcement learning trial. More...

void	xcs_rl_init_trial (struct XCSF *xcsf)
	Initialises a reinforcement learning trial. More...

void	xcs_rl_update (struct XCSF xcsf, const double state, const int action, const double reward, const bool reset)
	Provides reinforcement to the sets. More...

double	xcs_rl_fit (struct XCSF xcsf, const double state, const int action, const double reward)
	Creates and updates an action set for a given (state, action, reward). More...

Detailed Description

Reinforcement learning functions.

Author: Richard Preen rpree.nosp@m.n@gm.nosp@m.ail.c.nosp@m.om

Copyright: The Authors.

Date: 2015–2020.

Definition in file xcs_rl.h.

Function Documentation

◆ xcs_rl_decision()

int xcs_rl_decision	(	struct XCSF *	xcsf,
		const double *	state
	)

Selects an action to perform in a reinforcement learning problem.

Constructs the match set and selects an action to perform.

Parameters

[in]	xcsf	The XCSF data structure.
[in]	state	The input state.

Returns: The selected action.

Definition at line 247 of file xcs_rl.c.

References clset_match(), pa_best_action(), pa_build(), pa_rand_action(), and rand_uniform().

Referenced by XCS::decision(), and xcs_rl_trial().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_end_step()

void xcs_rl_end_step	(	struct XCSF *	xcsf,
		const double *	state,
		const int	action,
		const double	reward
	)

Ends a step in a reinforcement learning trial.

Parameters

[in]	xcsf	The XCSF data structure.
[in]	state	The current input state.
[in]	action	The current action.
[in]	reward	The current reward.

Definition at line 171 of file xcs_rl.c.

References clset_free(), and pa_val().

Referenced by XCS::end_step(), xcs_rl_fit(), and xcs_rl_trial().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_end_trial()

void xcs_rl_end_trial ( struct XCSF * xcsf )

Frees memory used by a reinforcement learning trial.

Parameters

[in] xcsf The XCSF data structure.

Definition at line 145 of file xcs_rl.c.

References clset_free(), and clset_kill().

Referenced by XCS::end_trial(), xcs_rl_fit(), and xcs_rl_trial().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_error()

double xcs_rl_error	(	struct XCSF *	xcsf,
		const int	action,
		const double	reward,
		const bool	done,
		const double	max_p
	)

Returns the reinforcement learning system prediction error.

Parameters

[in]	xcsf	The XCSF data structure.
[in]	action	The current action.
[in]	reward	The current reward.
[in]	done	Whether the environment is in a terminal state.
[in]	max_p	The maximum payoff in the environment.

Returns: The prediction error.

Definition at line 223 of file xcs_rl.c.

References pa_val().

Referenced by XCS::error(), and xcs_rl_trial().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_exp()

double xcs_rl_exp ( struct XCSF * xcsf )

Executes a reinforcement learning experiment.

Parameters

[in] xcsf The XCSF data structure.

Returns: The mean number of steps to goal.

Definition at line 78 of file xcs_rl.c.

References perf_print(), and xcs_rl_trial().

Referenced by main().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_fit()

double xcs_rl_fit	(	struct XCSF *	xcsf,
		const double *	state,
		const int	action,
		const double	reward
	)

Creates and updates an action set for a given (state, action, reward).

Parameters

[in]	xcsf	The XCSF data structure.
[in]	state	The input state to match.
[in]	action	The selected action.
[in]	reward	The reward for having performed the action.

Returns: The prediction error.

Definition at line 104 of file xcs_rl.c.

References clset_match(), pa_build(), pa_val(), param_set_explore(), xcs_rl_end_step(), xcs_rl_end_trial(), xcs_rl_init_step(), xcs_rl_init_trial(), and xcs_rl_update().

Referenced by XCS::fit().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_init_step()

void xcs_rl_init_step ( struct XCSF * xcsf )

Initialises a step in a reinforcement learning trial.

Parameters

[in] xcsf The XCSF data structure.

Definition at line 157 of file xcs_rl.c.

References clset_init().

Referenced by XCS::init_step(), xcs_rl_fit(), and xcs_rl_trial().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_init_trial()

void xcs_rl_init_trial ( struct XCSF * xcsf )

Initialises a reinforcement learning trial.

Parameters

[in] xcsf The XCSF data structure.

Definition at line 126 of file xcs_rl.c.

References clset_init().

Referenced by XCS::init_trial(), xcs_rl_fit(), and xcs_rl_trial().

Here is the call graph for this function:

Here is the caller graph for this function:

◆ xcs_rl_update()

void xcs_rl_update	(	struct XCSF *	xcsf,
		const double *	state,
		const int	action,
		const double	reward,
		const bool	done
	)

Provides reinforcement to the sets.

Creates the action set, updates the classifiers and runs the EA.

Parameters

[in]	xcsf	The XCSF data structure.
[in]	state	The input state.
[in]	action	The action selected.
[in]	reward	The reward from performing the action.
[in]	done	Whether the environment is in a terminal state.

Definition at line 192 of file xcs_rl.c.

References clset_action(), clset_update(), clset_validate(), ea(), and pa_best_val().

Referenced by XCS::update(), xcs_rl_fit(), and xcs_rl_trial().

Here is the call graph for this function:

Here is the caller graph for this function:

Functions

Detailed Description

Function Documentation

◆ xcs_rl_decision()

◆ xcs_rl_end_step()

◆ xcs_rl_end_trial()

◆ xcs_rl_error()

◆ xcs_rl_exp()

◆ xcs_rl_fit()

◆ xcs_rl_init_step()

◆ xcs_rl_init_trial()

◆ xcs_rl_update()