ePuck

Introduction

For the ePuck robot, a small simulator is provided. It allows to place ePuck in an arena, with unpassable walls and obstacles at (almost) arbitrary locations. Some environment objects are predefined in HDPy.epuck.env. The class Robot provides the implementation of the simulated ePuck. Obstacles are directly inserted into this instance, hence it combines the robot with the environment.

As for other problems, a ADHDP instance can be used on top of this to control the robot motion. A plant and a policy have to be provided (see Plants and Policies). Due to historical reasons, the interpretation of the policy (i.e. action) is done in the robot. In Robot, the action is a relative heading, AbsoluteRobot implements an absolute one.

The robot and adhdp instances are combined in the simulation_loop() function to run the simulation for a fixed amount of time.

The analysis of ePuck experiments is conducted by looking at isolated time steps of a testing or training situation. Three tools have been implemented to support this procedure: epuck_plot_all_trajectories() plots all training trajectories, epuck_plot_value_over_action() creates a graph of the expected return in a state as a function of the action. epuck_plot_snapshot() plots the predicted return over an example trajectory for several actions.

Example

import numpy as np
import HDPy
import os
import pylab

## INITIALIZATION ##

# Robot
obstacles = [
    HDPy.epuck.env.train_lower,
    HDPy.epuck.env.train_middle,
    HDPy.epuck.env.train_left,
    HDPy.epuck.env.train_upper
]

robot = HDPy.epuck.AbsoluteRobot(
    walls       = HDPy.epuck.env.obstacles_box,
    obstacles   = obstacles,
    tol         = 0.0,
    speed       = 0.5,
    step_time   = 0.5,
)

# Plant and Policy
policy = HDPy.epuck.policy.HeadingRandInit()
plant = HDPy.epuck.plant.CollisionAvoidanceFrontal(
    theta       = 1.0,
    obs_noise   = 0.05
)

# Set up reservoir
reservoir = HDPy.ReservoirNode(
    output_dim      = 50,
    input_dim       = policy.action_space_dim() + plant.state_space_dim(),
    spectral_radius = 0.95,
    input_scaling   = 1.0/3.0,
    bias_scaling    = -3.0,
    fan_in_w        = 20
)

reservoir.initialize()

# Set up readout
readout = HDPy.StabilizedRLS(
    with_bias   = True,
    input_dim   = reservoir.get_output_dim() + policy.action_space_dim() + plant.state_space_dim(),
    output_dim  = 1,
    lambda_     = 1.0
)

# Custom ADHDP
class ExperimentingHDP(HDPy.CollectingADHDP):
    def _next_action_hook(self, a_next):
        """Project action into the interval [0,2pi]."""
        return a_next % (2*np.pi)

# Remove old data file
if os.path.exists('/tmp/epuck_data.hdf5'):
    os.unlink('/tmp/epuck_data.hdf5')

# Create ADHDP instance
acd = ExperimentingHDP(
    # Demanded by CollectingADHDP
    expfile     = '/tmp/epuck_data.hdf5',
    # Demanded by ADHDP
    reservoir   = reservoir,
    readout     = readout,
    # Demanded by ActorCritic
    plant       = plant,
    policy      = policy,
    gamma       = 0.5,
    alpha       = 1.0,
    init_steps  = 5,
)

## SIMULATION LOOP ##

# Execute the simulation for 10 episodes, with 100 steps tops each
HDPy.epuck.simulation_loop(
    acd,
    robot,
    max_step        = 100,
    max_episodes    = 10,
    max_total_iter  = -1
)

## EVALUATION ##

# Load the data file
analysis = HDPy.Analysis('/tmp/epuck_data.hdf5')

# Plot the trajectories and obstacles
axis = pylab.figure().add_subplot(111)
robot._plot_obstacles(axis=axis)
HDPy.epuck.plot_all_trajectories(analysis, axis)

# Show the figure
pylab.show(block=False)
_images/epuck_result.png

Reference

class HDPy.epuck.Robot(walls=None, obstacles=None, speed=0.5, step_time=1.0, tol=0.0)

Simulated ePuck robot.

The robot may be steered by means of change in its orientation (i.e. the heading relative to the robot). Every time an action is executed, the robot turns to the target orientation, then moves forward. How much it moves is proportional to the speed and step_time. In between, infrared sensor readouts can be taken. The robot is placed in an arena, with some obstacles and walls it can collide with but not pass. Upon collision, the robot stops moving.

walls
List of wall lines which cannot be passed. The lines are to be given by their endpoints.
obstacles
List of obstacles which cannot be passed. In contrast to walls, the obstacles are closed polygons. They have to be given as list of corner points. Obstacles may not include the origin (0, 0).
speed
Speed of the robot.
step_time
Time quantum for movement, i.e. for how long the robot drives forward.
tol
Minimal distance from any obstacle or wall which counts as collision.

Note

Obstacles may not include the origin (0, 0).

Todo

wall tolerance does not operate correctly.

read_sensors()

Read all sensors. A dict is returned.

take_action(action)

Execute an action and move forward (speed * step_time units or until collision). Return True if the robot collided.

reset()

Reset the robot to the origin.

reset_random(loc_lo=-10.0, loc_hi=10.0)

Reset the robot to a random location, outside the obstacles.

plot_trajectory(wait=False, with_tol=True, tol=None, full_view=True, axis=None)

Plot the robot trajectory in a pylab figure.

wait
True for blocking until the figure is closed.
with_tol
Plot obstacle tolerance lines.
tol
Overwrite the obstacle tolerance.
full_view
Keep the original clipping of the window. If false, the clipping will be adjusted to the data.
axis
A pylab axis, which should be used for plotting. If not provided, the first axis of the first figure is used.
class HDPy.epuck.AbsoluteRobot(walls=None, obstacles=None, speed=0.5, step_time=1.0, tol=0.0)

Bases: HDPy.epuck.epuck.Robot

Simulated ePuck robot.

In contrast to Robot, the heading is with respect to the arena instead of the robot - i.e. it is absolute, not relative to the robot.

HDPy.epuck.simulation_loop(acd, robot, max_step=-1, max_episodes=-1, max_total_iter=-1)

Simulate some episodes of the ePuck robot.

This method handles data passing between the acd and robot instances in two loops, one for the episode and one for the whole experiment.

acd
Actor-Critic instance (ADHDP).
robot
Robot instance (Robot).
max_step
Maximum number of steps in an episode. Negative means no limit.
max_episodes
Maximum number of episodes. Negative means no limit.
max_total_iter
Maximum number of steps in total. Negative means no limit.
HDPy.epuck.epuck_plot_snapshot(*args, **kwargs)

Alias of plot_snapshot()

Deprecated since version 1.0: Use plot_snapshot() instead

HDPy.epuck.epuck_plot_value_over_action(*args, **kwargs)

Alias of plot_value_over_action()

Deprecated since version 1.0: Use plot_value_over_action() instead

HDPy.epuck.epuck_plot_all_trajectories(*args, **kwargs)

Alias of plot_all_trajectories()

Deprecated since version 1.0: Use plot_all_trajectories() instead

Table Of Contents

Previous topic

Plants and Policies

Next topic

Puppy

This Page