neural_layer.c File Reference

Interface for neural network layers. More...

#include "neural_activations.h"
#include "neural_layer_avgpool.h"
#include "neural_layer_connected.h"
#include "neural_layer_convolutional.h"
#include "neural_layer_dropout.h"
#include "neural_layer_lstm.h"
#include "neural_layer_maxpool.h"
#include "neural_layer_noise.h"
#include "neural_layer_recurrent.h"
#include "neural_layer_softmax.h"
#include "neural_layer_upsample.h"
#include "utils.h"

Include dependency graph for neural_layer.c:

Go to the source code of this file.

Functions
void	layer_set_vptr (struct Layer *l)
	Sets a neural network layer's functions to the implementations. More...
bool	layer_mutate_eta (struct Layer *l, const double mu)
	Mutates the gradient descent rate of a neural layer. More...
int	layer_mutate_neurons (const struct Layer *l, const double mu)
	Returns the number of neurons to add or remove from a layer. More...
void	layer_add_neurons (struct Layer *l, const int N)
	Adds N neurons to a layer. Negative N removes neurons. More...
bool	layer_mutate_connectivity (struct Layer *l, const double mu_enable, const double mu_disable)
	Mutates a layer's connectivity by zeroing weights. More...
void	layer_ensure_input_represention (struct Layer *l)
	Ensures that each neuron is connected to at least one input and each input is connected to at least one neuron. More...
bool	layer_mutate_weights (struct Layer *l, const double mu)
	Mutates a layer's weights and biases by adding random numbers from a Gaussian normal distribution with zero mean and standard deviation equal to the mutation rate. More...
bool	layer_mutate_functions (struct Layer *l, const double mu)
	Mutates a layer's activation function by random selection. More...
void	layer_weight_print (const struct Layer *l, const bool print_weights)
	Prints a layer's weights and biases. More...
char *	layer_weight_json (const struct Layer *l, const bool return_weights)
	Returns a json formatted string representation of a layer's weights. More...
void	layer_weight_rand (struct Layer *l)
	Randomises a layer's weights and biases. More...
void	layer_weight_clamp (const struct Layer *l)
	Clamps a layer's weights and biases in range [WEIGHT_MIN, WEIGHT_MAX]. More...
void	layer_calc_n_active (struct Layer *l)
	Recalculates the number of active connections within a layer. More...
void	layer_init_eta (struct Layer *l)
	Initialises a layer's gradient descent rate. More...
void	layer_defaults (struct Layer *l)
	Initialises a layer to default values. More...
const char *	layer_type_as_string (const int type)
	Returns a string representation of a layer type from an integer. More...
bool	layer_receives_images (const int type)
	Returns a whether a layer type expects images as input. More...
int	layer_type_as_int (const char *type)
	Returns the integer representation of a layer type given a name. More...
void	layer_guard_biases (const struct Layer *l)
	Check number of biases is within bounds. More...
void	layer_guard_outputs (const struct Layer *l)
	Check number of outputs is within bounds. More...
void	layer_guard_weights (const struct Layer *l)
	Check number of weights is within bounds. More...

Detailed Description

Interface for neural network layers.

Author

Richard Preen rpree.nosp@m.n@gm.nosp@m.ail.c.nosp@m.om

Copyright

The Authors.

Date

2016–2020.

Definition in file neural_layer.c.

Function Documentation

layer_add_neurons()

void layer_add_neurons	(	struct Layer *	l,
		const int	N
	)

Adds N neurons to a layer. Negative N removes neurons.

Precondition

N must be appropriately bounds checked for the layer.

Parameters

[in]	l	The neural network layer to mutate.
[in]	N	The number of neurons to add.

Definition at line 130 of file neural_layer.c.

References Layer::bias_updates, Layer::biases, Layer::delta, layer_calc_n_active(), LAYER_EVOLVE_CONNECT, layer_guard_outputs(), layer_guard_weights(), Layer::n_biases, Layer::n_inputs, Layer::n_outputs, Layer::n_weights, Layer::options, Layer::output, rand_normal(), rand_uniform(), Layer::state, Layer::weight_active, WEIGHT_SD, Layer::weight_updates, and Layer::weights.

Referenced by mutate_neurons(), and neural_layer_connected_mutate().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_calc_n_active()

void layer_calc_n_active

(

struct Layer *

l

)

Recalculates the number of active connections within a layer.

Parameters

[in]

l

The layer to recalculate the number of active connections.

Definition at line 384 of file neural_layer.c.

References Layer::n_active, Layer::n_weights, and Layer::weight_active.

Referenced by layer_add_neurons(), neural_layer_connected_resize(), neural_layer_convolutional_add_filters(), and neural_layer_convolutional_resize().

Here is the caller graph for this function:

layer_defaults()

void layer_defaults

(

struct Layer *

l

)

Initialises a layer to default values.

Parameters

[in]

l

The layer to initialise.

Definition at line 413 of file neural_layer.c.

References Layer::bias_updates, Layer::biases, Layer::c, Layer::cell, Layer::channels, Layer::dc, Layer::decay, Layer::delta, Layer::eta, Layer::eta_max, Layer::eta_min, Layer::f, Layer::function, Layer::g, Layer::h, Layer::height, Layer::i, Layer::indexes, Layer::input_layer, Layer::layer_vptr, Layer::max_neuron_grow, Layer::max_outputs, Layer::momentum, Layer::mu, Layer::n_active, Layer::n_biases, Layer::n_filters, Layer::n_inputs, Layer::n_outputs, Layer::n_weights, Layer::o, Layer::options, Layer::out_c, Layer::out_h, Layer::out_w, Layer::output, Layer::output_layer, Layer::pad, Layer::prev_cell, Layer::prev_state, Layer::probability, Layer::recurrent_function, Layer::scale, Layer::self_layer, Layer::size, Layer::state, Layer::stride, Layer::temp, Layer::temp2, Layer::temp3, Layer::type, Layer::uf, Layer::ug, Layer::ui, Layer::uo, Layer::weight_active, Layer::weight_updates, Layer::weights, Layer::wf, Layer::wg, Layer::wi, Layer::width, and Layer::wo.

Referenced by layer_init(), layer_load(), neural_layer_avgpool_copy(), neural_layer_connected_copy(), neural_layer_convolutional_copy(), neural_layer_dropout_copy(), neural_layer_lstm_copy(), neural_layer_maxpool_copy(), neural_layer_noise_copy(), neural_layer_recurrent_copy(), neural_layer_softmax_copy(), and neural_layer_upsample_copy().

Here is the caller graph for this function:

layer_ensure_input_represention()

void layer_ensure_input_represention

(

struct Layer *

l

)

Ensures that each neuron is connected to at least one input and each input is connected to at least one neuron.

Parameters

[in]

l

A neural network layer.

Definition at line 204 of file neural_layer.c.

References Layer::n_active, Layer::n_inputs, Layer::n_outputs, rand_normal(), rand_uniform_int(), Layer::weight_active, WEIGHT_SD, and Layer::weights.

Referenced by neural_layer_connected_mutate(), and neural_layer_connected_resize().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_guard_biases()

void layer_guard_biases

(

const struct Layer *

l

)

Check number of biases is within bounds.

Parameters

[in]

l

Layer to check.

Definition at line 581 of file neural_layer.c.

References layer_print(), Layer::n_biases, and N_OUTPUTS_MAX.

Referenced by guard_malloc().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_guard_outputs()

void layer_guard_outputs

(

const struct Layer *

l

)

Check number of outputs is within bounds.

Parameters

[in]

l

Layer to check.

Definition at line 595 of file neural_layer.c.

References layer_print(), Layer::n_outputs, and N_OUTPUTS_MAX.

Referenced by guard_malloc(), layer_add_neurons(), malloc_layer_arrays(), and realloc_layer_arrays().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_guard_weights()

void layer_guard_weights

(

const struct Layer *

l

)

Check number of weights is within bounds.

Parameters

[in]

l

Layer to check.

Definition at line 609 of file neural_layer.c.

References layer_print(), Layer::n_weights, and N_WEIGHTS_MAX.

Referenced by guard_malloc(), layer_add_neurons(), and malloc_layer_arrays().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_init_eta()

void layer_init_eta

(

struct Layer *

l

)

Initialises a layer's gradient descent rate.

Parameters

[in]

l

The layer to initialise.

Definition at line 399 of file neural_layer.c.

References Layer::eta, Layer::eta_max, Layer::eta_min, LAYER_EVOLVE_ETA, Layer::options, and rand_uniform().

Referenced by neural_layer_connected_init(), and neural_layer_convolutional_init().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_mutate_connectivity()

bool layer_mutate_connectivity	(	struct Layer *	l,
		const double	mu_enable,
		const double	mu_disable
	)

Mutates a layer's connectivity by zeroing weights.

Parameters

[in]	l	The neural network layer to mutate.
[in]	mu_enable	Probability of enabling a currently disabled weight.
[in]	mu_disable	Probability of disabling a currently enabled weight.

Returns

Whether any alterations were made.

Definition at line 176 of file neural_layer.c.

References Layer::n_active, Layer::n_inputs, Layer::n_outputs, Layer::n_weights, rand_normal(), rand_uniform(), Layer::weight_active, WEIGHT_SD, and Layer::weights.

Referenced by mutate_connectivity(), neural_layer_connected_mutate(), and neural_layer_convolutional_mutate().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_mutate_eta()

bool layer_mutate_eta	(	struct Layer *	l,
		const double	mu
	)

Mutates the gradient descent rate of a neural layer.

Parameters

[in]	l	The neural network layer to mutate.
[in]	mu	The rate of mutation.

Returns

Whether any alterations were made.

Definition at line 88 of file neural_layer.c.

References clamp(), Layer::eta, Layer::eta_max, Layer::eta_min, and rand_normal().

Referenced by mutate_eta(), neural_layer_connected_mutate(), and neural_layer_convolutional_mutate().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_mutate_functions()

bool layer_mutate_functions	(	struct Layer *	l,
		const double	mu
	)

Mutates a layer's activation function by random selection.

Parameters

[in]	l	The neural network layer to mutate.
[in]	mu	The rate of mutation.

Returns

Whether any alterations were made.

Definition at line 283 of file neural_layer.c.

References Layer::function, LSTM, NUM_ACTIVATIONS, rand_uniform(), rand_uniform_int(), Layer::recurrent_function, and Layer::type.

Referenced by mutate_functions(), neural_layer_connected_mutate(), neural_layer_convolutional_mutate(), and neural_layer_lstm_mutate().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_mutate_neurons()

int layer_mutate_neurons	(	const struct Layer *	l,
		const double	mu
	)

Returns the number of neurons to add or remove from a layer.

Parameters

[in]	l	The neural network layer to mutate.
[in]	mu	The rate of mutation.

Returns

The number of neurons to be added or removed.

Definition at line 106 of file neural_layer.c.

References clamp(), Layer::max_neuron_grow, Layer::max_outputs, Layer::n_outputs, rand_normal(), and rand_uniform().

Referenced by mutate_neurons(), and neural_layer_connected_mutate().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_mutate_weights()

bool layer_mutate_weights	(	struct Layer *	l,
		const double	mu
	)

Mutates a layer's weights and biases by adding random numbers from a Gaussian normal distribution with zero mean and standard deviation equal to the mutation rate.

Parameters

[in]	l	The neural network layer to mutate.
[in]	mu	The rate of mutation.

Returns

Whether any alterations were made.

Definition at line 252 of file neural_layer.c.

References Layer::biases, clamp(), Layer::n_biases, Layer::n_weights, rand_normal(), Layer::weight_active, WEIGHT_MAX, WEIGHT_MIN, and Layer::weights.

Referenced by mutate_weights(), neural_layer_connected_mutate(), and neural_layer_convolutional_mutate().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_receives_images()

bool layer_receives_images

(

const int

type

)

Returns a whether a layer type expects images as input.

Parameters

[in]

type

Integer representation of a layer type.

Returns

Whether images (height × width × channels) are expected as inputs.

Definition at line 521 of file neural_layer.c.

References AVGPOOL, CONVOLUTIONAL, MAXPOOL, and UPSAMPLE.

Referenced by layer_args_json_export_inputs(), and layer_args_validate_inputs().

Here is the caller graph for this function:

layer_set_vptr()

void layer_set_vptr

(

struct Layer *

l

)

Sets a neural network layer's functions to the implementations.

Parameters

[in]

l

The neural network layer to set.

Definition at line 42 of file neural_layer.c.

References AVGPOOL, CONNECTED, CONVOLUTIONAL, DROPOUT, layer_avgpool_vtbl, layer_connected_vtbl, layer_convolutional_vtbl, layer_dropout_vtbl, layer_lstm_vtbl, layer_maxpool_vtbl, layer_noise_vtbl, layer_recurrent_vtbl, layer_softmax_vtbl, layer_upsample_vtbl, Layer::layer_vptr, LSTM, MAXPOOL, NOISE, RECURRENT, SOFTMAX, Layer::type, and UPSAMPLE.

Referenced by layer_init(), and layer_load().

Here is the caller graph for this function:

layer_type_as_int()

int layer_type_as_int

(

const char *

type

)

Returns the integer representation of a layer type given a name.

Parameters

[in]

type

String representation of a layer type.

Returns

Integer representing the layer type.

Definition at line 540 of file neural_layer.c.

References AVGPOOL, CONNECTED, CONVOLUTIONAL, DROPOUT, LSTM, MAXPOOL, NOISE, RECURRENT, SOFTMAX, STRING_AVGPOOL, STRING_CONNECTED, STRING_CONVOLUTIONAL, STRING_DROPOUT, STRING_LSTM, STRING_MAXPOOL, STRING_NOISE, STRING_RECURRENT, STRING_SOFTMAX, STRING_UPSAMPLE, and UPSAMPLE.

Referenced by layer_args_json_import().

Here is the caller graph for this function:

layer_type_as_string()

const char* layer_type_as_string

(

const int

type

)

Returns a string representation of a layer type from an integer.

Parameters

[in]

type

Integer representation of a layer type.

Returns

String representing the name of the layer type.

Definition at line 486 of file neural_layer.c.

References AVGPOOL, CONNECTED, CONVOLUTIONAL, DROPOUT, LSTM, MAXPOOL, NOISE, RECURRENT, SOFTMAX, STRING_AVGPOOL, STRING_CONNECTED, STRING_CONVOLUTIONAL, STRING_DROPOUT, STRING_LSTM, STRING_MAXPOOL, STRING_NOISE, STRING_RECURRENT, STRING_SOFTMAX, STRING_UPSAMPLE, and UPSAMPLE.

Referenced by layer_args_json_export().

Here is the caller graph for this function:

layer_weight_clamp()

void layer_weight_clamp

(

const struct Layer *

l

)

Clamps a layer's weights and biases in range [WEIGHT_MIN, WEIGHT_MAX].

Parameters

[in]

l

The neural network layer to clamp.

Definition at line 365 of file neural_layer.c.

References Layer::biases, clamp(), Layer::n_biases, Layer::n_weights, Layer::weight_active, WEIGHT_MAX, WEIGHT_MIN, and Layer::weights.

Referenced by neural_layer_connected_update(), and neural_layer_convolutional_update().

Here is the call graph for this function:

Here is the caller graph for this function:

layer_weight_json()

char* layer_weight_json	(	const struct Layer *	l,
		const bool	return_weights
	)

Returns a json formatted string representation of a layer's weights.

Parameters

[in]	l	The layer to return.
[in]	return_weights	Whether to return the values of weights and biases.

Returns

String encoded in json format.

Definition at line 324 of file neural_layer.c.

References Layer::biases, Layer::n_active, Layer::n_biases, Layer::n_weights, and Layer::weights.

Referenced by layer_weight_print(), neural_layer_connected_json_export(), neural_layer_convolutional_json_export(), neural_layer_lstm_json_export(), and neural_layer_recurrent_json_export().

Here is the caller graph for this function:

layer_weight_print()

void layer_weight_print	(	const struct Layer *	l,
		const bool	print_weights
	)

Prints a layer's weights and biases.

Parameters

[in]	l	The neural network layer to print.
[in]	print_weights	Whether to print each individual weight and bias.

Definition at line 309 of file neural_layer.c.

References layer_weight_json().

Here is the call graph for this function:

layer_weight_rand()

void layer_weight_rand

(

struct Layer *

l

)

Randomises a layer's weights and biases.

Parameters

[in]

l

The neural network layer to randomise.

Definition at line 348 of file neural_layer.c.

References Layer::biases, Layer::n_active, Layer::n_biases, Layer::n_weights, rand_normal(), Layer::weight_active, WEIGHT_SD_RAND, and Layer::weights.

Referenced by neural_layer_connected_rand(), and neural_layer_convolutional_rand().

Here is the call graph for this function:

Here is the caller graph for this function: