Mark Carson
mark.carson@nist.gov
Internetworking Technologies Group
NIST
September, 1997
- Quality of service guarantees (min bandwidth, max latency) are harder to test,
compare than simple best-effort delivery
- Adaptive protocols and applications are harder to "pin down" and test thoroughly
- New capabilities such as multicast delivery require larger, more complex test
environments for any sort of reasonable test coverage
- New link technologies (HFC, XDSL, DBS) can be highly asymmetric (and
expensive to test in a lab setting)
- Simulation: Run a model/ representation of the code in a synthetic environment.
- Live test: Run the real code in the real (or near real) environment
- Advantages of simulation
- (Can be) cheap and quick to assemble
- Given sufficient computing resources, can do large-scale tests
- Tests are controlled, reproducible
- Problems with simulation
- Need to redo code for simulation environment
- Simulation implementation may differ considerably from real one
- Synthetic environment may also poorly represent real one.
- Advantages of live testing
- Real code in real environment
- Problems with live testing
- May only be possible very late in development cycle
- Often difficult or too expensive to create a real test environment of any
significant size
- "Real environment" tests also tend not to be reproducible, making it difficult
to analyze problems when found
- A controlled, reproducible environment for running live code
- Emulates any degree of "real-world badness" in a small, lab-environment network
- Provides a means for simulations to interact with a live environment
- Implemented via "hooks" into a real, working networking implementation (in our
case, TCP/IP on Linux 2.0.30)
- Hitbox pseudo-device for SunOS
- Orchestra fault injection tool
- Trace-based mobile network emulation
- MOST Radio network emulator
- X-kernel simulator/live test
- Commercial ATM emulators
- NIST Network performance/testing projects
- Collaboration over highly heterogeneous networks, wireless to gigabit
(DARPA ICV)
- Test bed for digital video delivery over IP (ATP)
- RSVP monitoring/validation
- Web content delivery experimentation (UI group)
- Web server performance instrumentation
- Network emulation for the people!
- Based on freely available code (Linux)
- Simple (hopefully) installation and setup
- Runs on commodity PCs, with stock Ethernet cards
- Total cost $0 -- a few thousand
- 10Mb Ethernet emulation on $0 machine (486/50 with junk Ethernet cards) (< 50
sec/packet overhead)
- 100Mb Ethernet emulation on < $2500 machine (200MHz Pentium with standard
10/100 cards) (< 5 sec/packet overhead)
- Performance constrained only by network capacity, not number of emulated
connection pairs (within reason)
- Simple user entry of network parameters (packet loss, delay)
- Allow record/playback of network conditions
- Suitable for testing multicast, high-bandwidth applications
- Emulate a wide range of network types with a small lab setup
- Emulate asymmetric networks over symmetric ones
- Emulator constraints may be updated continuously while emulator operates
- Emulator may be taken up or down, patched and reloaded without interrupting any
active connections
- Packet delay
- Fixed
- Variable (loadable delay probability distribution, settable mean/variance)
- Default distribution is pareto-normal, empirically parametrized to match
observed packet delays
- Packet reordering
- Through large delay variances
- Packet loss
- Uniform probability
- Congestion-dependent (DRD-type algorithm)
- Packet duplication
- Bandwidth limitation
- Use in conjunction with congestion-dependent packet loss
- Packet diversion to other, user-supplied handlers
- Gather packet type, size, etc. info for Web server monitoring project
- Intercept and spoof RTP replies
- Monitor QoS performance
- Each source/destination pair may receive separate handling from emulator