Block I logo
University of Illinois Urbana-Champaign

Imaging Technology Group

Syzygy Background

Syzygy Background

Syzygy Architecture

Syzygy has five layers, from bottom to top as follows. 1, an OS-portability layer. 2, a network communications layer. 3, communication protocols for things like graphics and audio. 4, a distributed operating system that forms things like sound players and remote scene graph renderers ("media objects"). 5, application frameworks for various styles of cluster programming.

 syzygyarchitecture

Portability Layer

Windows, Unix, etc., differently implement threads, locks, sockets, and serial I/O. This layer hides these differences. This necessarily omits some idiosyncratic features, but that doesn't hurt Syzygy as a whole. Details are in the Portability Layer chapter.

Communications Infrastructure

This layer provides tools for constructing communications protocols that are easy to change, automatically convert data formats, and robustly connect/disconnect/reconnect in an unreliable network environment. A protocol is defined by a dictionary of message templates, which defines the semantics of the sendable messages. Data transfer objects then manage connections and data transfer. Eventually, this layer will contain an abstract interface to the underlying communications medium, allowing the same code to work over TCP, UDP, or shared memory communications pipes.

Media Protocols

Protocols for manipulating media information implement synchronization; distributing and rendering a scene graph; manipulating and playing a sound database; and communicating with input devices.

Network-based synchronization primitives provide "active stereo" swap lock across displays.

The distributed database itself is simple, while the graphics and sound implementations built on it are elaborate.

A master database accepts network connections from remote slave databases. At connection, the master database dumps itself in the slave database, creating two synchronized copies. Subsequent changes to the master database are automatically mirrored to the slave, creating consistency without programmer effort.

The general input device framework lets distinct physical devices aggregate into larger virtual ones. The data streams flowing through these devices can be filtered arbitrarily.

Distributed Operating System

One program, szgserver, manages basic services of a distributed operating system. One instance of this program runs per Syzygy installation. All Syzygy programs connect to it. To avoid bottlenecks, it only sets up connections between elements in the distributed system. Another program, szgd ("Syzygy Daemon"), runs on each computer to provide remote execution services.

The distributed operating system manages processes. Applications can be started or killed on any computer, by typing commands at any other computer. Applications send messages to one another through a standard interface for interprocess communication. A parameter database, managed by szgserver, acts as a config file maintable over the network. This replaces the text configuration files that might otherwise be scattered across many computers.

The distributed operating system supports multiple users.

Details are in the Cluster Mode chapter.

Media Objects

These objects include SoundRender, for the distributed sound database, and szgrender, for the distributed graphics database. They turn information from the network into sensory events.

Application Frameworks

See Programming and Application Frameworks

Goals

Syzygy's first web release was in September 2000. Its goal was to run a wide range of VR applications on a PC cluster, with performance matching shared-memory SMP computers like the SGI Onyx. Since then, it has become portable, robust, and easier to use. The Syzygy application model has also been adapted to diverse environments such as standalone laptops, wide-area networks, and even the older shared-memory SMP systems.

Hardware changes drive software changes. Here, developing for PC clusters requires explicit library support and an updated application model. Monolithic shared memory systems can ignore many data-sharing issues, leaving these to the applications programmer and encouraging machine-dependent code.

Consider the detailed requirements for PC cluster software. Inter-processor communication is a bottleneck. Tools are needed to efficiently manage and develop cluster applications. The developer needs an API that supports complex communications between cluster machines. These information exchanges must be fault-tolerant and conducted in a way that keeps the cluster producing frame-by-frame synchronized views of the virtual world. During the early stages of application development, the application code may be unstable, so the overall distributed system must tolerate the failure of individual components. The system needs the ability to portably launch applications across a wide range of target clusters, both homogeneous and heterogenous in nature. It also needs mechanisms for steering running applications and monitoring tools that allow users to remotely determine its state.

The goal of creating VR applications on a PC cluster is best served by increasing the level of abstraction and writing a toolkit with broader applicability. Syzygy aims to provide simple software tools for building heterogeneous distributed systems. While the system's main focus is PC clusters on high performance LANs, it is also applicable to WAN-based systems. Syzygy is designed to be a lightweight toolkit that can operate on PC clusters, SMP boxes, and laptops, while connecting meaningfully with smaller devices. This presents several challenges. Much system infrastructure is unnecessary for a standalone laptop being used to provide a portable road demo, and, to maintain ease of use, the sytem's operation should scale down gracefully. Furthermore, we need to deploy communication methods suitable for SMP, namely shared memory instead of sockets, and do this while reusing our low-level communications infrastructure. Finally, performance is very important, as many of the toolkit's functions are OS-level and must be efficient.

Syzygy solves several problems: distributed graphics, distributed sound, distributed systems management, distributed data access, and general I/O device management. There are systems out there that do each of these things, but no one system addresses them all. However, each problem relies on a similar infrastructure for its solution. Syzygy aims to identify that common infrastructure and provide an implementation. Thus, each problem can be solved more easily, and the resulting code base is well-integrated, maintainable, and smaller. In effect, Syzygy is a middleware solution for distributed multimedia.

Change Log

Changes in version 1.3

As of revision number 1794 of July '09, the environment variables for building the python bindings have changed. Please read the linked section.

Make now inserts bazaar revision number and other info into a source file, szg/src/language/arVersion.cpp. This requires both bzr and python. If they're not available, set the environment variable SZG_BZR_INFO=FALSE to skip this step.

Added support for alternate input simulators, built as shared libraries and loaded based on a database parameter (SZG_INPUTSIM/sim_type) at runtime. An example is included in szg/skeleton.

Added support for Python input devices. This required changes to the arAppLauncher to allow Python input devices in virtual computer definitions, so all programs will need to be recompiled for any setup where you want to use a Python input driver. Hence the bump to 1.3.

Added the ability to set the application frameworks' navigation matrix by copying a matrix directly from the input event stream. This allows control of navigation by e.g. a Python script.

Re-worked the Makefiles and conditional compilation such that input device drivers are built as shared libraries even if szg is built with SZG_LINKING=STATIC--except with Visual C++. This means that you can load input devices in standalone mode on Linux and MacOS.

Renamed 'python' and 'sip_python' directories to 'python_swig' and 'python_sip', respectively.

Changes in version 1.2

Syzygy 1.2 happened internally.

Previously, application launch on a virtual computer would fail if even one of the constituent computers didn't have an szgd running on it. Now the application will launch on the remaining computers (unless of course the missing szgd was on the computer that the application's trigger instance was supposed to run on).

Bugfixes

Fixed a bug in szgd that could lead to the current working directory not being set properly if an renderer and one or more services were all launched on the same computer.

Fixed a crash bug for daisy-chained input devices (arInputFactory netinput slot #).

utilities/PictureViewer now runs properly on multi-display computers.

Fixed a bug that caused certain path variables to grow over time.

Fixed the documentation stylesheet so that the docs can now be printed legibly.

We've found some bugs in the handling of multiple networks. A bug that occurred when multiple networks were used in a virtual computer, but some computers were only attached to one of them has been fixed.

Previously, if a connection between an szgserver and szgd were broken due to e.g. network unlreliability, the szgserver would notice and remove the szgd from the process table but the szgd would not notice (and hence would not attempt to reconnect). This has been fixed.

Extraction of Euler angles from matrices was incorrect & has been fixed.

Diagnostics

ar_log_error() and company now print a process ID and timestamp.

Compiling

Got things compiling with Visual Studio 8.

Ditto for g++ 4.3.1 (on Sidux).

Added support for joysticks/gamepads when compiling with MinGW. Ditto for speech synthesis. Ditto for .3ds model files.

Windows builds have been speeded up with all compilers.

Input Devices

Added ar5DTGloveDriver (5th dimension technologies dataglove).

arFOBDriver attempts to query the flock about its configuration instead of using the SZG_FOB/config database parameter.

Added a prototype wiimote driver on Linux using wiiuse-0.9 (1.0 and later won't work without a re-write).

Added some more PForth words.

Standalone Mode

Programs in standalone mode can now load input device drivers, but only if they were built with dynamic linking.

Programming

The biggest change is much improved .obj support.

Python

Began the switch from SWIG to SIP. Currently two sets of bindings available, in szg/python and szg/sip_python. The new bindings are easier to read and maintain and sometimes faster, but don't yet include support for the more object-oriented version of the scene graph API.

Moved the scene graph demos out of the 'doc/python folder to python/demos/myriad and sip_python/demos/myriad. As mentioned above, the latter don't work yet, although sip_python/demos/blobby/blobby.py is a working scene-graph program.

Commands

The ouput of dex has been changed a bit, and the -v and  -q (verbose and quiet) options have been added. Also, some of the application output during startup that was just being lost before is now returned to dex.

Added dkillall -a <virtual_computer> option, which kills all non-szgd processes associated with a virtual computer.

Changes in version 1.1

Syzygy 1.1. is primarily a bug-fix release (too many to list here). The most important changes:

Documentation

...Has been largely re-written for clarity. A few major messes remain, however.

Demos

Some demos (ones that were insufficiently instructive or that depended on very large data sets) have been removed from the main "szg" repostitory. We're moving towards packaging each of these individually together with their data.

Skeleton/Build Template

The new szg/skeleton directory is a directory tree template for building your own programs. Copy the directory somewhere, rename, modify the source files, etc.

Commands

szgd

szgd now takes a non-optional <base paths> argument. The full form of the command is:

szgd <base paths> [-r]

The <base paths> argument is a semicolon-delimited list of paths to directories or native executables. This allows you to restrict remote execution to particular directories or programs. On Windows the ".exe" suffix of executables should be omitted. You'll probably need to enclose the argument in quotes if it contains a semicolon. Each item must exist or szgd will exit. All SZG_EXEC and SZG_PYTHON paths and SZG_PYTHON/executable must begin with an item from the <base paths> argument or the dex command will fail.

The optional -r causes it to repeatedly attempt to reconnect to the Syzygy server on failure (instead of quitting as formerly).

dex

One variant of the dex command has been eliminated (to clarify error messages). You may no longer run an application on the local computer by typing:

dex <app> <args>

Instead, type:

dex <computer> <app> <args>

 

Precompiled SDKs

Are no longer available; we're too under-staffed to support them. Weren't too useful anyway.

Sound

Syzygy 1.1 uses FMOD 4 (ex) instead of FMOD 3. This change is transparent to the application programmer.

Configuration

XML parameter files

The code that parses XML config files now reports errors (quite well) instead of failing silently as it did in 1.0 (we hang our heads in shame).

Resource Layout

Native executables can now be packaged together with their data files (as Python programs could before). szgd now looks for executables in subdirectories of directories on SZG_EXEC/path, and when it finds an executable it sets the current working directory to the directory containing the executable. This means that data files that the application opens directly can be placed in the directory with the executable (or a subdirectory of it) and opened using the normal file-access functions with application-relative paths. And applications and their data can be bundled up into their own directory. See also the description of  setDataBundlePath() in the Programming chapter.

Python Executable

The location of the Python executable should be specified using the database variable SZG_PYTHON/executable instead of having its directory added to SZG_EXEC/path. You can pass arguments to this executable by delimiting them with bars, e.g.  c:\Python24\python.exe|<arg1>|<arg2>.

Programming

Master/Slave
Scene Graph

Is now extensible via new node types defined in shared library plugins (see szg/src/graphics/arTeapotGraphicsPlugin.cpp, called from szg/src/demo/cubes/cubes.cpp). No build template for creating these plugins yet, alas.

Python
All

Fixed bug in arGUIWindow preventing application access to window pixel dimensions. One of these days we'll even document the arGUI classes...

Fixed a bug in arTexture::readPPM() that was causing textures to be read in left/right reversed.

PForth

Added new words less, greater to work around XML parser bug ("<" and ">" don't work well in XML and the parser was choking on the escaped versions).

Changes in szg-1.0

Changes in szg-0.8

Changes in szg-0.7

Changes in szg-0.6

Changes in szg-0.55

Changes in szg-0.5

Changes in szg-0.4

Changes in szg-0.3

IMPORTANT NOTE: Endian-conversion is currently broken! The arStructuredData::parse call breaks it. The next version of Syzygy will do endian-conversion upstream so that this is not a problem.

Changes in szg-0.2

Testing

Supported Configurations

Syzygy uses the following combination of technologies, influencing the platforms on which it can compile and run. As of 2004, this requirement set has become mainstream, and, because a development goal is not to expand it further, building and running Syzygy should require only trailing edge features of the one's development environment in the future, a desirable characteristic.

Syzygy is portable across a range of modern operating systems, Linux, Win32, Mac OS X, and Irix. It is stable on these systems, with the stability subjected to rigorous testing, as noted below. Furthermore, it is stable and tested across heterogenous combinations of these systems: your clusters need not be comprised of one type of machine.

Any new (as of summer 2004) system should compile and run Syzygy. Some specific information about a wider range of systems follows.

Special Notes for Windows 98

The problem with Windows 98 derives from the way in which Syzygy components automatically determine their names for connecting to the Syzygy distributed OS. On Windows 98, it is impossible for an executable to get its own name (other than in all capital letters) from argv[0] when run from cmd.exe (the default Windows shell). Since Syzygy uses the command line argv[0] (in most cases) to set the component name, this causes problems with automated operation of the distributed system, for instance when running an application on a "virtual computer", one of whose nodes is Win98. As a work-around, three components (szgd, DeviceServer, and SoundRender) manually set their component names. Thus, Win98 nodes in a virtual computer can host input devices (via DeviceServer) and can also emit sounds (via SoundRender). Finally, components can retrieve the correct name from the argv[0] when launched via the cygwin shell. Consequently, normal operation of the distributed system is assured if you only launch components from the cygwin shell when on Win98. Running a syzygy program from an incorrectly configured Win98 DOS shell can fail. If this happens, try unchecking "Prevent MS-DOS-based programs from detecting Windows".

Testing Procedures

Syzygy is stable enough that it can now be used to reliably detect bugs and misconfigurations in the hardware/OS upon which it is being run. The following tests thoroughly exercise the system, though they have yet to be automated.

Bugs

Documentation Errors

Setup/Building

- Thread-safety issues... The Irix port does, indeed, seem to be working. However, it seems sensitive to having an appropriately configured compiler. (or maybe a recent enough version). Specifically, if the following test program compiles and runs for a minute or two, then syzygy will run and be stable.

#include
#include
#include
using namespace std;
void* task1(void*){
  cout << "YYYY\n";
  while (true){
    string temp("yyy_test_name");
  }
  return NULL;
}
void* task2(void*){
  cout << "XXXX\n";
  while (true){
    string temp("death_by_xxx");
  }
  return NULL;
}
int main(int, char**){
  cout << "#####################################################\n";
  pthread_t ID1, ID2;
  pthread_create(&ID1,NULL,task1,NULL);
  pthread_create(&ID2,NULL,task2,NULL);
  // wait forever
  pthread_join(ID1, NULL);
}

 

The compile script for Irix is simple. Here, we are assuming that the program name is test.cpp.

CC -n32 -LANG:std -c test.cpp
CC -n32 -LANG:std -o test test.o -lpthread

 

NOTE: This would probably catch the thread safety problem on RedHat 7.x as well (?). Haven't really felt like re-installing something that ancient to find out.

Configuration/Cluster Mode

Graphics/Scene Graph/Database

Miscellaneous

405 North Mathews Avenue, Urbana, IL 61801 USA

(217) 300-0566

University of Illinois Urbana-Champaign

Imaging Technology Group

405 North Mathews Avenue, Urbana, IL 61801 USA

(217) 300-0566

Around Campus

Universitywide Resources

About Illinois