Why doesn’t my job run under TWS?

by Pete Meechan

One of the most common questions asked when adding new workloads to TWS is “Why doesn’t my job run under TWS when it runs perfectly okay from the command line?” This tip explains possible reasons and provides troubleshooting techniques that should help to answer that question.

The login environment

If you are familiar with login procedures you may want to skip to the “Sourcing the environment” section.

When you login to a computer, irrespective of whether it is a Windows, Linux, UNIX, Mac or any other operating system, there are a number of actions that are performed automatically in the background to prepare the computer “environment” for your use. What actions are performed will vary greatly from one operating system to another, as well as from one organisation to another as the actions performed are usually specified by the system administrator responsible for the computer systems being used.

Typically, actions such as setting up the location of the programs that will be used are defined so that they can be automatically located when required. If your organisation uses shared resources such as printers or storage area networks (SAN), these will also be allocated and defined automatically at login time. How these are defined will also vary depending upon the operating system and command environment being used, but generically speaking they will be allocated using a “login profile”.

The “login profile” consists of a series of commands that define, amongst other things, the printer to be used by default, which locations need to be referenced when looking for the commands or programs that you will execute, etc. This process is generally referred to as “setting up the environment”.

When your login profile has executed it will usually set up the location of programs or commands that are executed using an environment variable known as the “PATH”. The PATH variable contains multiple locations (paths) that contain programs and commands that are available to your login session. This allows you to execute programs and commands by only specifying their name (e.g. copy) and the operating system will automatically locate and execute the requested program.

The “environment” contains many other settings that are required either by the operating system itself or set by the system administrator responsible for the management of the computers in your organisation. In the example shown below (taken from a Linux system) the “HOSTNAME” environment variable contains the name of the computer (e.g. clynelish.scotchwhisky.local) and the “PATH” environment variable contains the names of the paths (or directories) that will be searched for any programs or commands that you may execute. On UNIX and Linux systems the “environment” settings can be listed using the “env” command as shown below.

For the Windows “environment” similar information can be obtained using the “set” command from a Windows command prompt as shown below. The “COMPUTERNAME” environment variable contains the hostname of the computer (e.g. BALBLAIR) and the “PATH” environment variable contains the names of the paths (or directories) that will be searched for any programs or commands that you may execute.

Sourcing the environment

Not all programs that you may use are automatically added to the “PATH” environment variable so that they can be automatically located when required. On Windows systems for example, the program shortcut used to start a program (e.g. Microsoft Word) contains the location of the program and the directory path where the program is located. Similar shortcut definitions can be created on UNIX or Linux systems and are usually known as program or application launchers.

A program can also be executed by entering the full path to the program e.g. /usr/orb/bin/program1 (UNIX/Linux) or C:OrbProgramsprogram1.exe (Windows), but this technique requires knowledge of where the program is located and is therefore considered rather user unfriendly.

A common practice to avoid having to enter the full path to programs is to provide a “sourcing” script that sets up the correct environment to allow the program to execute successfully. Whilst the individual actions performed by the “sourcing” script are determined by the requirements of the individual program, they will almost always temporarily modify the “PATH” environment variable to include the location of the program for which it was supplied. Common examples of products that use “sourcing” scripts are database applications and java programs.

TWS is another program that employs a “sourcing” script as part of the installation process. If you need to use the TWS command line interface (for command such as composer or conman), you will need to “source” the TWS environment to provide this capability. This is achieved by executing the TWS supplied script – twa_env.sh (UNIX/Linux) or twa_env.cmd (Windows) found within the TWS installation directory. Part of the processing performed by these scripts is to add the location of the TWS programs to the “PATH” environment variable so that the “conman” and “composer” commands can be executed successfully from the command line.
Note: The TWS “sourcing” script is only required when using the TWS command line interface – it is not required for execution of non-TWS programs (e.g. database, sales forecast update, etc.) that execute under TWS.

In summary, successful execution of a program within an interactive login command prompt/shell is achieved by using one or more of the techniques listed above.

TWS Environment

So why doesn’t my program run under TWS? In most cases, it is because the TWS environment is not the same as the interactive login environment described above. The simplest way to determine what is different between the two environments is to list the environment settings in both the TWS and the interactive login environments.

The previous section showed how to use the “env” (UNIX/Linux) and “set” (Windows) command to list the interactive login environment settings. Within TWS these same commands can be used in a TWS job definition to obtain the TWS environment settings.

The sample jobs shown below can be used to do this for the UNIX/Linux and Windows environments respectively. It is very important that the user specified in the job definition STREAMLOGON is the same user that is logged in to the interactive login described earlier in this article. If the user specified is not the same, the differences in the environment settings may be caused by the different user environments.

UNIX/Linux job definition

Windows job definition

As can be seen from the results of these jobs below, the TWS environment does not contain all of the environment settings that are found within the interactive login environment, but contains a lot of additional environment variable settings that are set and used by TWS (e.g. all of the UNISON_xxx variables).

UNIX/Linux TWS environment settings

Windows TWS environment settings

Modifying TWS environment settings

If it is necessary to change the default TWS environment settings to make specific settings available for ALL jobs executed under TWS, the TWS supplied jobmanrc (UNIX/Linux) or jobmanrc.cmd (Windows) script can be modified accordingly. The script can be found within the TWS installation directory.

If the changes affect only the jobs executed by a specific user such as a database user, instead of modifying the common jobmanrc script the user specific script can be updated. This script does not exist and must be created in the home directory of the user (e.g. /home/user for UNIX/Linux or the value in the %USERPROFILE% environment variable for Windows). For UNIX/Linux the script must be named “.jobmanrc” and be marked as executable. For Windows the script must be named “djobmanrc.cmd”.

Troubleshooting the TWS environment

When trying to determine why a job does not execute successfully under TWS, the first option to consider is the different environmental settings as described earlier. If correction to the environment settings does not resolve the issue, review the following items to see if they could be the cause of further inconsistencies.

• Ensure that the user logged in to the system where the program executes successfully from an interactive command line, is the same user specified in the TWS job definition STREAMLOGON parameter. If they are different, this can cause inconsistency in the environment settings in addition to permission issues
• Particularly within the Windows environment, consider the following

1. 1. Most programs have been developed for use in an interactive login environment and consequently assume that certain environmental variables will be set when the program executes. These may not be set by default within the TWS environment.
   2. Some programs make use of the Windows desktop heap and require the “interactive job” setting to be specified. Normally this will be indicated by the error message in the job log or the Windows event log mentioning errors relating to the “desktop heap”
   3. Some programs assume that temporary files are set to the user home directory, but this is not always the case in background processing. It may be necessary to check the program documentation to see if it is possible to override this assumption with a program specific parameter. An example of this is for the SAS Institute business analytics program, where the “-WORK C:Temp” parameter can be specified when executing SAS jobs under TWS.

• Use the program and operating system log files to assist with the problem determination process and identify the error. Typical places for locating errors are

1. TWS job log output
2. TWS TWMERGE log file in the TWS installation directory stdlist/logs directory
3. Windows event log
4. UNIX/Linux /var/log/messages
5. home directory for the user executing the job