The Solaris-compatible Thread Library (ScTL) source kit

Release Notes v1.1.1    July 2003

Changes from v1.1.0 are highlighted in red. In summary, the changes from v1.1.0 to v1.1.1 are:

About ScTL

The Solaris-compatible Thread Library (ScTL) is a piece of open-source software that enables applications that use the Solaris threads API (thr_create(), mutex_lock(), etc.) to be ported to other systems that have a POSIX threads library. ScTL provides a layer between the application and the POSIX threads library that maps between the Solaris and POSIX thread data types and functions, and adds thread functionality.

ScTL was originally developed as part of the Solaris Compatibility Libraries (SCL) for Tru64 UNIX. See http://tru64unix.compaq.com/complibs/ for information about SCL. There is an on-line Users Guide for SCLat http://tru64unix.compaq.com/complibs/documentation/html/scl_ug.html, of which chapters 3 Solaris Threads and 5 Error Logging are relevant to ScTL.

There is a paper about ScTL that was presented at the Usenix 2001 conference at http://www.opensource.compaq.com/the_source/linux_papers/scl_solaris.htm. There is also a web-site for ScTL at http://www.opensource.compaq.com/the_source/linux_papers/stl.htm.

The ScTL source-code is hosted in a SourceForge project at http://sourceforge.net/projects/sctl. The Sourceforge home-page for ScTL is http://sctl.sourceforge.net.

ScTL was known as STL, and many of the build-files, etc. still refer to STL instead of ScTL.

ScTL is provided "as-is", with no formal support. Any problems or queries can be sent to solaris.complib@compaq.com where they will be dealt with on a best-efforts basis.

ScTL License

ScTL v1.0.0 used the GNU General Public License: see http://www.gnu.org/copyleft/gpl.html. This has been changed for ScTL v1.1.0 to use the GNU Lesser General Public License: see http://www.gnu.org/copyleft/lesser.html. This license choice is more suitable for libraries, because it means that ISVs could use the open-source library without being required to open-source their application code. The intention behind ScTL was always to help make it easier for ISVs to port from Solaris, and so this license change should make ScTL more acceptable for ISVs to use.

Build platforms

ScTL has been built on the following platforms:
 
Operating System Compiler
Tru64 UNIX v4.0F Compaq C
Tru64 UNIX v5.1 Compaq C
Red Hat Linux 7.0 (Alpha, 64-bit) GNU C
Red Hat Linux 7.0 (Intel, 32-bit) GNU C
Linux on IBM S/390 GNU C ?
FreeBSD 4.2 (Intel, 32-bit) GNU C
HP-UX 11i (B.11.11) (PA-RISC, 32-bit & 64-bit) HP C
HP-UX 11i 1.6 (B.11.22) (Itanium, 32-bit & 64-bit) bundled C

New Features

Introduced in SCTL 1.1.1:

Known Problems / Features

  1. On all platforms, thr_setprior() and thr_getprior(), to manipulate thread priority, do not actually affect thread scheduling. ScTL just records the value set by thr_setprior() so that it can be returned by thr_getprior().
  2. On all platforms, system-wide mutexes, condition variables and read/write locks are not supported by ScTL, even if the operating system supports them (e.g. Tru64 UNIX v5 and above).
  3. On all platforms, ScTL does not support the mixing of Solaris thread and POSIX thread calls within the same program. Solaris allows this.
  4. On Linux, the threads implementation uses the clone() system call. Generally this does not cause ScTL any problems, but note that a result of using clone() is that each thread of the same virtual process has a different process identifier.
  5. On Linux, a threaded ScTL process can hang when one or more threads are suspended, and the last non-suspended thread calls exit(). This does not happen on Tru64 UNIX or FreeBSD, where the process exits.
  6. On FreeBSD 4.2, there is no pthread_setconcurrency() nor pthread_getconcurrency() function. Thus calls to the Solaris equivalent routines thr_setconcurrency() and thr_getconcurrency() do nothing on FreeBSD other than maintain an internal-to-ScTL concurrency value, such that the value set by thr_getconcurrency() can be obtained by calling thr_getconcurrency().
  7. On FreeBSD 4.2, mutexes always seem to do some error-checking. E.g. if one thread locks a mutex, and another thread tries to unlock it, then this will fail and return an error status on FreeBSD. On Linux and Tru64 UNIX, (and on Solaris), the unlock would succeed. (Note that it is bad practice for one thread to unlock a mutex that another thread locked: the results may be unpredictable).
  8. On HP-UX on Itanium, if one thread calls mutex_lock(), and another thread subsequently calls mutex_unlock(), mutex_unlock(), mutex_lock() on the same mutex, then the program hangs at the second mutex-lock. This program is essentially broken in that the second thread should not unlock the mutex that the first thread locked, but you can get away with this on Solaris, Tru64 UNIX and Linux.
  9. On HP-UX, the ScTL man-pages will not work unless you perform the following steps:
  10. The ScTL man-pages require the sml & rsml macro-files for correct display. These macros seem to originate from OSF, and don't seem to be available for Linux. I'm not sure what the proper solution is: I worked around it by copying the sml & rsml files from a UNIX system to my linux system, and edited man/man3/linux_conf.sed & run the man/man3/linux_conf.csh script.

The ScTL source kit

The ScTL source kit contains the source-code and scripts to build ScTL on one of the aforementioned platforms.

ScTL is written in the C programming language, and requires an ANSI89 C compiler to build it from source.

ScTL requires that the target operating system has a POSIX 1003.1-1996 -compliant threads library. ScTL also uses POSIX threads extensions from the Single UNIX Specification v2, which defines the UNIX98 brand.

ScTL was originally developed on Tru64 UNIX. It has been ported to Linux and FreeBSD.

Building ScTL from source-code

Pre-requisite: GNU make

Makefiles are used to build ScTL components. You need to use GNU make with the ScTL makefiles, due to the syntax used within the makefiles for platform-specific dependencies and actions. On Linux, the make utility is GNU make, so you're all set. On Tru64 UNIX, FreeBSD and HP-UX, you will need to obtain a version of GNU make, as GNU make is not bundled. GNU make can be build from the GNU make source-code, which can be down-loaded from a link on the GNU make web-page at http://www.gnu.org/software/make/make.html.

The ScTL development makefiles are all named gnumakefile, to make it more obvious that GNU make is required.

To use an ScTL gnumakefile, you must specify a command-line parameter OS that indicates the operating system platform. Recognized values for OS are:

Thus to invoke an ScTL makefile on say Tru64 UNIX, you would use a command such as:
% gnumake -f gnumakefile OS=t64

The tool/MakeSTL script

To build ScTL, from the top-level source directory invoke the tool/MakeSTL script. Note that this script uses the gnumake command to process the makefiles, to distinguish the GNU make utility from the platform's standard make utility. Use either an alias command to map gnumake onto the GNU make executable, or create a symbolic link called gnumake that links to the GNU make executable.

The tool/MakeSTL script checks that some environment variables have been set, and if they have not then it will prompt you to set them. The environment variables needed are:

SCL_SOURCE_ROOT
  -top-level directory for the ScTL source-code.

SCL_BUILD_ROOT
  -top-level directory for the ScTL build (object files, libraries, executables).

Note that the environment variables are prefixed by SCL, not ScTL, which betrays ScTL's origin as part of SCL.

SCL_SOURCE_ROOT must point to the ScTL source code's top-level directory. This directory will have sub-directories of:

If you are going to use the ScTL source in a heterogeneous environment, i.e. to build on multiple platforms from one set of NFS-mounted source-code, then you should set the SCL_BUILD_ROOT environment variable to be a platform-specific directory. For example, if SCL_SOURCE_ROOT is /home/stl, then you could set SCL_BUILD_ROOT on a Linux Intel system to /home/stl_build/linux_intel, and set SCL_BUILD_ROOT on a Linux Alpha system to /home/stl_build/linux_alpha.

If you are building for a single platform only, then you can set SCL_BUILD_ROOT to be the same as SCL_SOURCE_ROOT.
 

When you run tool/MakeSTL, it may ask if you want to build the ScTL libraries and/or the tests. Currently you can only build the libraries, because the test source code has not been released.

MakeSTL should build two libraries and a message catalogue:

SCL_BUILD_ROOT/shlib/libthread.so
SCL_BUILD_ROOT/shlib/libsolutil.so
SCL_BUILD_ROOT/lib/nls/msg/scl_message.cat
Actually MakeSTL also builds debug-built versions of the two libraries (named libthreaddbg.so and libsolutildbg.so), but the debug libraries are mainly used during development.
 

The MakeSTL script does not require that OS is set, as it sets OS by running the common/scl_os script.

The MakeSTL script can be invoked with a make-target parameter. For example:

% tool/MakeSTL clean
will invoke all of the gnumakefiles with the target clean, which removes all re-creatable files.
 
 

Summary of commands to build ScTL from source

gnumake     -check that the gnumake command works
setenv SCL_SOURCE_ROOT /dir/containing/ScTL/sources
setenv SCL_BUILD_ROOT  /dir/for/build/files
chmod +x $SCL_SOURCE_ROOT/MakeSTL
cd $SCL_BUILD_ROOT
$SCL_SOURCE_ROOT/tool/MakeSTL


Using a newly-built ScTL

The following files are needed to enable Solaris-threaded applications to be built and run using ScTL:
$SCL_SOURCE_ROOT/include/thread.h          Solaris thread declarations
$SCL_SOURCE_ROOT/include/synch.h           Solaris thread synchronization-object declarations
$SCL_BUILD_ROOT/shlib/libthread.so         Solaris thread shared object library
$SCL_BUILD_ROOT/shlib/libsolutil.so        SCL utility library
$SCL_BUILD_ROOT/lib/nls/msg/scl_message.cat  SCL message catalogue
$SCL_BUILD_ROOT/bin/tru64_version          needed for Tru64 UNIX only
You can build your Solaris applications against the ScTL development files, or you may choose to use a suitable root directory to copy the necessary ScTL files to, so that the building of your Solaris-threaded applications is not interfered with by modifications to the ScTL development builds.

Assume that you are developing on Tru64 UNIX, and that the ScTL development files are copied to /usr/opt/STL. Set the environment variable SCL_ROOT_DIR to this directory. Then copy the above files into (beneath) the target directory, such that you now have the following files on your system:

$SCL_ROOT_DIR/include/thread.h
$SCL_ROOT_DIR/include/synch.h
$SCL_ROOT_DIR/shlib/libthread.so
$SCL_ROOT_DIR/shlib/libsolutil.so
$SCL_ROOT_DIR/lib/nls/msg/scl_message.cat
$SCL_ROOT_DIR/bin/tru64_version             needed on Tru64 UNIX only


When building Solaris-threaded applications, note the following points:

  1. Tell the compiler where to find the ScTL header files
  2. Tell the linker where to find the ScTL libraries
  3. Tell the loader where to find the ScTL libraries
  4. Ensure that any ScTL messages are seen
For example, to build and run the threads example program on Linux, the steps are shown below, with the aforementioned points highlighted:
  1. Compile the program:
      2. cc -c -ansi -Wall -fPIC -pthread -I$SCL_ROOT_DIR/include -D_XOPEN_SOURCE=500 -D_POSIX_C_SOURCE=199506L stl_example.c -o stl_example.o
  2. Link the program:
      3. cc stl_example.o -L$SCL_ROOT_DIR/shlib -lthread -lsolutil -pthread -lrt -o stl_example
  3. Run the program:
      4. setenv LD_LIBRARY_PATH $SCL_ROOT_DIR/shlib
      setenv NLSPATH $SCL_ROOT_DIR/lib/nls/msg/%N
      setenv SCL_LOG_FILE stdout
      ./stl_example

To build and run the ScTL example program

There is an example ScTL program called stl_example.c in the threads/examples, or examples/threads, directory.

The stl_example.c program is supplied with the following makefiles:

The gnumakefile must be invoked in the same way as for the ScTL source. For example, assuming that SCL_SOURCE_ROOT and SCL_BUILD_ROOT are set appropriately, and that you are building on Linux:
gnumake -f gnumakefile OS=lnx
The makefile.xxx files do not require GNU make, but rather will work with the bundled make on the respective system. To build using the makefile.xxx files, you need to set an environment variable so that the C compiler can find the ScTL include-files, and the linker can find the ScTL libraries. The environment variable SCL_ROOT_DIR needs to be set to the parent directory of the ScTL kit. For example, if the ScTL files reside beneath /home/stl, and C-shell is being used, and you are building on Linux, then do the following:
setenv SCL_ROOT_DIR /home/stl
cd $SCL_ROOT_DIR/examples/threads
make -e -f makefile.lnx
Alternatively you could set SCL_ROOT_DIR on the make command-line. E.g.:
make -f makefile.lnx SCL_ROOT_DIR=/home/stl
But then you do not have SCL_ROOT_DIR as an environment variable.

Before running the stl_example program, you should set some environment variables so that the shared libraries, etc., can be found. E.g.

setenv LD_LIBRARY_PATH $SCL_ROOT_DIR/shlib
setenv NLSPATH         $SCL_ROOT_DIR/lib/nls/msg/%N
setenv SCL_LOG_FILE    stdout
Then run the example program to sum the square of the numbers 1..10 by using worker-threads to calculate each square.
./stl_example 10
The output should look similar to that below. Note the first and last message about SCL Logfile: you will not see this (and hence not see any potential error messages) if you have not set the SCL_LOG_FILE environment variable.
% ./stl_example 10
   0.00000|000003ffc01b3e88: SCL Logfile initialized

Will create 10 new threads

New thread 0x0000020000f17600 running; argument is 1
New thread 0x000002000141f600 running; argument is 2
New thread 0x0000020001927600 running; argument is 3
New thread 0x0000020001e2f600 running; argument is 4
New thread 0x0000020002337600 running; argument is 5
New thread 0x000002000283f600 running; argument is 6
New thread 0x0000020002d47600 running; argument is 7
New thread 0x000002000324f600 running; argument is 8
New thread 0x0000020003757600 running; argument is 9
New thread 0x0000020003c5f600 running; argument is 10

main thread: all new threads running;
will release them in 2 seconds.

Thread 0x0000020000f17600 with argument 1 computed square = 1
Thread 0x000002000141f600 with argument 2 computed square = 4
Thread 0x0000020001927600 with argument 3 computed square = 9
Thread 0x0000020001e2f600 with argument 4 computed square = 16
Thread 0x0000020002337600 with argument 5 computed square = 25
Thread 0x000002000283f600 with argument 6 computed square = 36
Thread 0x0000020002d47600 with argument 7 computed square = 49
Thread 0x000002000324f600 with argument 8 computed square = 64
Thread 0x0000020003757600 with argument 9 computed square = 81
Thread 0x0000020003c5f600 with argument 10 computed square = 100

All threads joined; total sum of squares is = 385

   2.05078|000003ffc01b3e88: SCL Logfile being closed
%

Distributing a binary ScTL kit

The following files are required to be distributed with an executable program that has been linked against ScTL, to enable the program to run on another system: If you also require the ability to build applications against ScTL, (not just run them), then you need the additional files:


You may also want to distribute the ScTL example program, stl_example.c, along with the appropriate makefile.xxx for the platform, so that ScTL can be tested.