Degraded Communications Emulator (DCE)
DARPA Robotics Challenge Finals - June 2015

$LastChangedDate: 2015-03-16 12:50:35 -0700 (Mon, 16 Mar 2015) $
$Rev: 74 $

About This Document

This is a living document - it will almost certainly evolve over time to become more complete and to correct errors.

InterWorking Labs must be, and will be, strictly neutral and impartial.  This note will be the primary vehicle though which information about the use of the Maxwell Pro devices as the Degraded Communications Emulators will be published.  Official information about the DRC Finals, its network, and the DCE settings is disseminated by DARPA; please see the DRC Testbed Operations Guide for that information.

Recent Changes In This Document

DCE Device

Network Interfaces
Network Interfaces.

Each DCE is constructed using one Maxwell® Pro from InterWorking Labs (http://iwl.com/)  Supplementary software will be used to control and monitor these units.  That supplementary code is being made available to all DRC Finals participants via this website.

For details how the DCEs will be interposed between each team and its robot see the DARPA document Communications between Operator and Robot for the DRC Finals.

The software on the DCE is the base Maxwell Pro software from InterWorking Labs.  Only the base version of that software is loaded or used.

The Maxwell Pro system software can be operated with or without a graphical user interface.  At the DRC Finals the graphical user interface will not be used.  Rather, the Maxwell Pro will be operated remotely using a control program.  This program and related materials are available via links below.

Note: The DCE device and the network degradation settings and algorithms being used at the DRC Finals event are significantly different from those used at the DRC Trials in 2013.

Which Network Interface Goes Where?

There are three network interfaces on the back of the DCE/Maxwell Pro:

Launching the DCE Software on the Maxwell Pro

Normally the Maxwell Pro software is operated via a graphical user interface.  That GUI automatically launches when one logs into the DCE/Maxwell Pro machine.  However, at the DRC Finals we will not be using that graphical user interface; and even if that GUI launches, it will be disconnected from the underlying packet handling engine when the DRC control software (described below) is activated.

Before the DRC control software can interact with the DCE/Maxwell Pro the packet handling engine must be initiated.  That can be done either by logging into the desktop and allowing the graphical user interface to do the work.  Or one may log in via SSH and run a small shell script (startstdiserver.sh provided below) to do the launch.  That latter approach is the preferred approach.

The packet handling engine reads and writes to the standard output, so the startstdiserver.sh should not be run in the background.

DCE Control Platform

At the DRC Finals the DCEs will be operated remotely.  A copy of that control software is provide here.

It is strongly urged that you run this control software on a platform separate from the Maxwell Pro itself.

These remote control programs are written in the Python language (version 2.7) and are intended to be run on a Linux™ platform.  This code has been run on these Linux distributions:

It is likely that the code will run on many other Linux distributions and desktops, however those have not been tested.

The Python code does require packages that may not be part of the base versions of Linux distributions, but are readily available.  These include:

Note: This list is likely to be extended.

The principal program of interest is drcctrl.py.  This is a small wxPython graphical program that will be used by the DCE operator to initiate a and operate a run of a robot through the course.  This program makes uses of several of the other items listed below.

Operating drcctrl.py

It is possible to operate drcctrl.py in either of two modes: manual or automatic.

Manual state mode allows the operator to control the start of the run.

Automatic start mode fetches a schedule web page that lists the run start times.  The next available start time will be selected.  Automatic mode requires that a properly formatted scheduling page be available using a URI.  This will generally mean that the file is stored on a web server, however, for testing a "file:<pathname>" URI may be used to fetch a local file.  An example of this file is available below.

The start mode is selected using the command line parameters to drcctrl.py.

The steps below are for manual start mode.

  1. Log in to an X11 based desktop.  On Linux this encompasses all the standard desktops - KDE, Gnome, etc.
  2. Create a new directory from which you will run the software.  Change your working directory to that directory.
  3. Copy all of the files below into that directory.
  4. Launch the drcctrl.py program using the following command line:

    python drcctrl.py --trackname <trackname> --trackcolor <bgcolor> -b <blackouts_csvfile> <DCEhost>

    <trackname>: Track name (use quotation marks if necessary.)

    <bgcolor>: The background color name or RGB value.

    <blackouts_csvfile>: The name of the CSV format file that contains the blackout schedule.

    <DCEhost>: The host name or IPv4 address of the DCE.

    Example:

    python drcctrl.py --trackname "Red Track" --trackcolor CORAL -b 2015-2-2_Degraded_Comms_Schedule_Example.csv 192.168.1.100

    If things are working a small graphical application should appear on your screen.

    DCE Control Application
  5. The graphical application has various buttons and indicators.  One of the most important indicators is the status line on the bottom; status and errors will be shown there.

Standard Output Messages

A fair number of messages will be displayed to the standard output.  Most of these represent internal events.  These messages probably will not be particularly useful to you; they are mainly for audit trails, and for code development and debugging

Supplementary Utility and Control Software

NOTE: These files may change without notice.  So check here for updates.

Note: When updating a Python (.py) file you should first remove any compiled (.pyc) version of that file that may be lingering in your directory.

Module Name Current Version Description
drcctrl.py 2015-03-10 16:24:12Z This is the main program; everything else is subsidiary.  This module displays the graphical user interface shown previously in this note.
dce_ctl.py 2015-03-08 14:09:31Z This is an internal library of utility functions.
servercomm.py 2015-01-23 01:10:14Z This is an internal library of utility functions.
track.py 2015-03-10 16:35:27Z This module is invoked drcctrl.py as a sub-process to handle the details of scheduling.  This module is not intended to be separately and has no graphical user interface.
starttrack.sh 2015-03-11 20:55:17Z This is a shell script that is useful when launching the drcctrl.py tool.  It is also a useful place to control the Link 3 and ICMP rate limits when doing testing. 
blackball-16.png 2015-01-15 This is an icon used by the drcctrl.py program.
blueball-16.png 2015-01-15 This is an icon used by the drcctrl.py program.
greenball-16.png 2015-01-15 This is an icon used by the drcctrl.py program.
redball-16.png 2015-01-15 This is an icon used by the drcctrl.py program.
yellowball-16.png 2015-01-15 This is an icon used by the drcctrl.py program.
startstdiserver.sh 2015-02-02 23:11:51Z This is a very short shell script that is run on the DCE/Maxwell Pro to start the packet handling engine without its usual graphical user interface.
2015-2-2_Degraded_Comms_Schedule_Example.csv 2015-02-02

This is a DARPA provided text file (in CSV - Comma Separated Value) format.  This file defines the timing of the blackout periods that will be applied to Link 2.

This is a usable sample; however it is not the blackout specification that will be used at the DRC Finals.

schedule.yaml 2015-01-22 07:49:52Z

This is an example of the YAML formatted text file that is fetched by the drcctrl.py program when running in automatic start mode.

This is a usable sample; however it is not the schedule that will be used at the DRC Finals.

Help and Assistance

If you need help with any of these, drop an email to karl at iwl.com.

If you are stuck and need help immediately, send a text message to +1 831-246-0418.

Please note that due to our obligation to be neutral the answer to your question may be added to this note.

Copyrights and Licenses

InterWorking Labs offers these programs under MIT open source license:

Copyright (c) 2015 InterWorking Labs, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

Notes and Questions

  1. It has been noted that the Maxwell Pro graphical user interface (GUI) becomes inoperative and disconnected from the impairment engine ("stdiserver") once the DRC control program (drcctrl.py) takes control.  This is not an indication of a problem or fault.  The GUI and the drcctrl.py control program should be considered as mutually exclusive ways to control the system.

    You may chose to operate the Maxwell Pro manually from its GUI.  However, when switching to the GUI you should do the following steps:

    1. Exit from the drcctrl.py control program.
    2. Use the killstdiserver command to make sure that the running impairment engine is terminated.
    3. Use the icon on the desktop to launch the GUI (which will, in turn, re-launch the impairment engine.)
  2. Many people use the iperf utility to generate traffic loads and measure bandwidth.  Iperf is a useful tool, however, one ought to be aware of how it measures bandwidth.  We have written a short note on the topic - Does-IPERF-Tell-White-Lies.pdf

    For purposes of counting bits, the DCE/Maxwell Pro rate limitation includes the 14 byte Ethernet header, IP header, UDP/TCP headers, and UDP/TCP data.  Iperf counts only the UDP/TCP data.

  3. There has been some question about the method used to limit bandwidth.  The method being used strives to mimic a physical path with limited buffering; bandwidth that is not utilized does not create a pool that later traffic may draw upon.  The following note may help explain the differences.  In this note the method we are using is called "Reduced Bandwidth Link Emulation".  Rate-Limitation-vs-Bandwidth-Limitation.pdf

  4. Some clarification may be useful regarding the classification of TCP and UDP packets by port numbers onto Links 2 and 3 as those links are shown on "Figure 1. Simplified Logical Communications Diagram." in the DRC Testbed Operations Guide.

    The dce_ctl.py module has been updated to reflect the text below.  Previously the dce_ctl.py module was slightly divergent from the DRC Testbed Operations Guide in that it formerly put TCP and UDP packets with source ports between 16384 and 24575 and flowing in the Team to Robot direction onto Link 2 (which consequently caused them to be silently discarded because Link 2 is only for use for Robot to Team packets.)  That sometimes meant that a packet that should properly have been sent to Link 3 was improperly discarded.

    How a TCP or UDP packet is handled depends on the direction in which that packet is flowing.  Also note that this section is concerned only with TCP and UDP packets; ICMP, ARP, and other packet types are handed as described in the DRC Testbed Operations Guide.

    1. Packets flowing from the Robot to the Team
      1. Packets that have destination UDP or TCP port numbers in the range 16384 - 24575 (inclusive) will be classified onto Link 2.  The source port number is not used when evaluating whether a packet should be sent via Link 2.

        Note that this port number test is applied before the test for Link 3, below.

      2. Packets that have either source or destination UDP or TCP port numbers in the range 0 to 2047 (inclusive) will be classified onto Link 3.
      3. All other TCP or UDP packets will be dropped.  (No ICMP packet will be generated to signal the drop.)
    2. Packets flowing from the Team to the Robot
      1. Packets that have either source or destination UDP or TCP port numbers in the range 0 to 2047 (inclusive) will be classified onto Link 3.
      2. All other TCP or UDP packets will be dropped.  (No ICMP packet will be generated to signal the drop.)
  5. When updating one of the Python (.py) files listed above, remember to remove the compiled version (the .pyc file) first.  Otherwise the Python interpreter may sometimes use the older, pre-compiled version rather than the newer, yet-to-be-compiled, version.