Analysis

Introduction

Assuming that all relevant values have been written into a HDF5 file (e.g. using CollectingADHDP), the controller intrinsics can be analyzed. To work with such a file, some functions are provided.

It’s assumed that a group represents one episode. There may be several groups in a single HDF5 file, which would then be interpreted as several runs (episodes) of the same experiment.

It is often usefull to look at the overall characteristics of a controller, for example development of the error. This analysis is limited, yet very quickly executed. To support this processes, Analysis provides a collection of simple functions for plotting various data characteristics.

More complex analysis is likely to be dependent on the concrete experiment. For the architectures defined in this module, more involved code is provided (see ePuck, Puppy). As a preliminary, critic() offers a compact representation of the learning machine.

Example

Reference

class HDPy.analysis.Analysis(pth, grid=False, min_key_length=0)[source]

Collection of simple plotting functions to analyze a HDF5 data file at pth.

The interface is similar for most plotting functions, consisting of an axis argument and allowing additional keyword arguments. If axis is None, a new figure will be created, otherwise the curve is plotted into the provided axis. The additional keywords are passed to the pylab.plot(). Note that some functions overwrite certain keywords (e.g. label or color).

pth
Path to the HDF5 data file or H5CombinedFile instance.
grid
True gray bars should be plotted, showing the episode boundaries.
min_key_length
Threshold for the number of datasets in an episode. This is usually used to filter out episodes which terminated during the initialization phase.
episode_len()[source]

Return the lengths of all episodes

get_data(key)[source]

Return a list of all data belonging to key.

get_episode(num)[source]

Return the raw data of episode num.

num_steps(key=None)[source]

Return the number of steps per experiment.

plot_absolute_error(axis=None, **kwargs)[source]

Plot the absolute error over time in axis.

plot_accumulated_reward(axis=None, **kwargs)[source]

Plot the accumulated reward per episode in axis.

plot_actions(axis=None, **kwargs)[source]

Plot the current (blue) and next action (red) over time in axis.

plot_cumulative_readout_diff(axis=None, **kwargs)[source]

Plot the cumulative difference of absolute readout weights in axis.

plot_derivative(axis=None, **kwargs)[source]

Plot the derivative dJ/da over time in axis.

plot_episode_len(axis=None, **kwargs)[source]

Plot the length of the episodes in axis.

plot_error(axis=None, **kwargs)[source]

Plot the error over time in axis.

plot_error_avg_over_episodes(axis=None, step=1, median=False)[source]

Plot the evolution of the error over multiple episodes.

This function assumes that the same path (same action sequence) was applied in all experiments.

plot_error_over_episodes(axis=None, step=1)[source]

Plot the evolution of the error over multiple episodes.

This function assumes that the same path (same action sequence) was applied in all experiments.

plot_grid(axis, key=None)[source]

Add a vertical bar to axis to mark experiment boundaries.

plot_input(axis=None, **kwargs)[source]

Plot the reservoir input in axis.

plot_input_over_episode(axis, episode)[source]

Plot the reservoir input of a single episode in axis.

plot_mean_reward(axis=None, **kwargs)[source]

Plot the accumulated reward per episode in axis.

plot_neuron_activation(axis=None, **kwargs)[source]

Plot the absolute sum of the neuron state (=activation potential) of the current (red) and next (blue) timestep in axis.

plot_node_over_episode(axis, episode, node)[source]

Plot the reservoir input and output of node on a tanh shape over one episode in axis. If node is None, all nodes are plotted. If axis is a list, each node is plotted in its own axis.

plot_node_over_episode_time_input(axis, episode, node)[source]

Plot the time over input of node in axis. Only episode is considered.

plot_node_over_episode_time_output(axis, episode, node)[source]

Plot the output of node over time in axis. Only episode is considered.

plot_node_weight_over_episode(episode, axis=None, **kwargs)[source]

Plot the readout weights of a single episode in axis.

plot_path_return_prediction(expno, axis=None, **kwargs)[source]

Plot the predicted return of a simulated path.

The return prediction is specific to the path of the experiment expno. The plots will be written to axis.

Five curves are shown:

  • Offline MSE
  • Online MSE (no bias)
  • Online MSE (with bias)
  • Predicted Value
  • Target

For the first three curves, the respective method was trained on the path and then evaluated (thus displaying a training behaviour). The fourth curve shows the evaluation at the robot state of the estimated value function, based on the whole experiment. In contrast to the other curves, it incorporates all experiments up to expno. Specifically, note that early time steps may not reflect the target well, since the inspected path only counts towards the expected value but doesn’t exclusively define it. The target is the discounted return for any step in the path of expno.

plot_predicted_return_over_episodes(axis=None, step=1)[source]

Plot the evolution of the predicted return over multiple episodes.

This function assumes that the same path (same action sequence) was applied in all experiments.

plot_readout(axis=None, func=<function <lambda> at 0x3844668>, **kwargs)[source]

Plot the readout nodes as individual curves in axis. The func allows for an operation to applied to the data before plotting. The main intention for this is abs(). Default is the identity.

plot_readout_diff(axis=None, **kwargs)[source]

Plot the difference of absolute readout weights in axis.

plot_readout_sum(axis=None, **kwargs)[source]

Plot the absolute readout weight over time in axis.

plot_reservoir_input(axis=None, **kwargs)[source]

Plot the reservoir input over time in axis. Per input, one line is drawn.

plot_return_prediction(axis=None, plot=('j_curr', 'j_next'), **kwargs)[source]

Plot the predicted return of the current (red) and next (blue) state/action pair in axis. plot defines which of the two predicted return are plotted.

plot_reward(axis=None, **kwargs)[source]

Plot the reward over time in axis.

stack_all_data()[source]

Return a dict with all concatenated data, sorted by the data key.

stack_data(key)[source]

Return data related to key of all experiments in a single array.

HDPy.analysis.overview(analysis, figure)[source]

Plot some characteristics of analysis in figure.

  • Sum of readout weights
  • Reward
  • Derivative
  • Actions
  • TD error
  • Accumulated reward
HDPy.analysis.node_inspection(analysis, figure, episode, node)[source]

Plot input and output of reservoir node node at episode according to analysis in figure. Note that this function assumes \(tanh\) as reservoir node function.

HDPy.analysis.critic(plant, reservoir, readout, norm=None)[source]

Use the simulation parts to set up a simpler to use critic. The critic is a function of the state and action. Furthermore, it takes simulate as argument to control if the reservoir state is actually advanced.

Table Of Contents

Previous topic

Utility functions

Next topic

Plants and Policies

This Page