For the ePuck robot, a small simulator is provided. It allows to place ePuck in an arena, with unpassable walls and obstacles at (almost) arbitrary locations. Some environment objects are predefined in HDPy.epuck.env. The class Robot provides the implementation of the simulated ePuck. Obstacles are directly inserted into this instance, hence it combines the robot with the environment.

As for other problems, a ADHDP instance can be used on top of this to control the robot motion. A plant and a policy have to be provided (see Plants and Policies). Due to historical reasons, the interpretation of the policy (i.e. action) is done in the robot. In Robot, the action is a relative heading, AbsoluteRobot implements an absolute one.

The robot and adhdp instances are combined in the simulation_loop() function to run the simulation for a fixed amount of time.

The analysis of ePuck experiments is conducted by looking at isolated time steps of a testing or training situation. Three tools have been implemented to support this procedure: epuck_plot_all_trajectories() plots all training trajectories, epuck_plot_value_over_action() creates a graph of the expected return in a state as a function of the action. epuck_plot_snapshot() plots the predicted return over an example trajectory for several actions.


import numpy as np
import HDPy
import os
import pylab


# Robot
obstacles = [

robot = HDPy.epuck.AbsoluteRobot(
    walls       = HDPy.epuck.env.obstacles_box,
    obstacles   = obstacles,
    tol         = 0.0,
    speed       = 0.5,
    step_time   = 0.5,

# Plant and Policy
policy = HDPy.epuck.policy.HeadingRandInit()
plant = HDPy.epuck.plant.CollisionAvoidanceFrontal(
    theta       = 1.0,
    obs_noise   = 0.05

# Set up reservoir
reservoir = HDPy.ReservoirNode(
    output_dim      = 50,
    input_dim       = policy.action_space_dim() + plant.state_space_dim(),
    spectral_radius = 0.95,
    input_scaling   = 1.0/3.0,
    bias_scaling    = -3.0,
    fan_in_w        = 20


# Set up readout
readout = HDPy.StabilizedRLS(
    with_bias   = True,
    input_dim   = reservoir.get_output_dim() + policy.action_space_dim() + plant.state_space_dim(),
    output_dim  = 1,
    lambda_     = 1.0

# Custom ADHDP
class ExperimentingHDP(HDPy.CollectingADHDP):
    def _next_action_hook(self, a_next):
        """Project action into the interval [0,2pi]."""
        return a_next % (2*np.pi)

# Remove old data file
if os.path.exists('/tmp/epuck_data.hdf5'):

# Create ADHDP instance
acd = ExperimentingHDP(
    # Demanded by CollectingADHDP
    expfile     = '/tmp/epuck_data.hdf5',
    # Demanded by ADHDP
    reservoir   = reservoir,
    readout     = readout,
    # Demanded by ActorCritic
    plant       = plant,
    policy      = policy,
    gamma       = 0.5,
    alpha       = 1.0,
    init_steps  = 5,


# Execute the simulation for 10 episodes, with 100 steps tops each
    max_step        = 100,
    max_episodes    = 10,
    max_total_iter  = -1


# Load the data file
analysis = HDPy.Analysis('/tmp/epuck_data.hdf5')

# Plot the trajectories and obstacles
axis = pylab.figure().add_subplot(111)
HDPy.epuck.plot_all_trajectories(analysis, axis)

# Show the figure


class HDPy.epuck.Robot(walls=None, obstacles=None, speed=0.5, step_time=1.0, tol=0.0)

Simulated ePuck robot.

The robot may be steered by means of change in its orientation (i.e. the heading relative to the robot). Every time an action is executed, the robot turns to the target orientation, then moves forward. How much it moves is proportional to the speed and step_time. In between, infrared sensor readouts can be taken. The robot is placed in an arena, with some obstacles and walls it can collide with but not pass. Upon collision, the robot stops moving.

List of wall lines which cannot be passed. The lines are to be given by their endpoints.
List of obstacles which cannot be passed. In contrast to walls, the obstacles are closed polygons. They have to be given as list of corner points. Obstacles may not include the origin (0, 0).
Speed of the robot.
Time quantum for movement, i.e. for how long the robot drives forward.
Minimal distance from any obstacle or wall which counts as collision.


Obstacles may not include the origin (0, 0).


wall tolerance does not operate correctly.


Read all sensors. A dict is returned.


Execute an action and move forward (speed * step_time units or until collision). Return True if the robot collided.


Reset the robot to the origin.

reset_random(loc_lo=-10.0, loc_hi=10.0)

Reset the robot to a random location, outside the obstacles.

plot_trajectory(wait=False, with_tol=True, tol=None, full_view=True, axis=None)

Plot the robot trajectory in a pylab figure.

True for blocking until the figure is closed.
Plot obstacle tolerance lines.
Overwrite the obstacle tolerance.
Keep the original clipping of the window. If false, the clipping will be adjusted to the data.
A pylab axis, which should be used for plotting. If not provided, the first axis of the first figure is used.
class HDPy.epuck.AbsoluteRobot(walls=None, obstacles=None, speed=0.5, step_time=1.0, tol=0.0)

Bases: HDPy.epuck.epuck.Robot

Simulated ePuck robot.

In contrast to Robot, the heading is with respect to the arena instead of the robot - i.e. it is absolute, not relative to the robot.

HDPy.epuck.simulation_loop(acd, robot, max_step=-1, max_episodes=-1, max_total_iter=-1)

Simulate some episodes of the ePuck robot.

This method handles data passing between the acd and robot instances in two loops, one for the episode and one for the whole experiment.

Actor-Critic instance (ADHDP).
Robot instance (Robot).
Maximum number of steps in an episode. Negative means no limit.
Maximum number of episodes. Negative means no limit.
Maximum number of steps in total. Negative means no limit.
HDPy.epuck.epuck_plot_snapshot(*args, **kwargs)

Alias of plot_snapshot()

Deprecated since version 1.0: Use plot_snapshot() instead

HDPy.epuck.epuck_plot_value_over_action(*args, **kwargs)

Alias of plot_value_over_action()

Deprecated since version 1.0: Use plot_value_over_action() instead

HDPy.epuck.epuck_plot_all_trajectories(*args, **kwargs)

Alias of plot_all_trajectories()

Deprecated since version 1.0: Use plot_all_trajectories() instead

Table Of Contents

Previous topic

Plants and Policies

Next topic


This Page