Workload Automation – Part 5b: Vexing Variables – variable tables

In the next part of this blog series we will look at variable tables. As mentioned in my previous blog, variable tables were introduced with TWS 8.5 and supplement the use of parameter values in earlier TWS versions.

Variable tables

Variables are held in variable tables which are stored in the TWS database. This provides the advantage of being able to manage the variable tables and variable values centrally through the TWS web console or composer.

Installing or upgrading to TWS 8.5 or later versions of TWS, creates a default variable table named MAIN_TABLE. This is the default name for the default variable table, but can be changed by making another variable table the default. For the purposes of this blog, it is assumed that the MAIN-TABLE is the default variable table.

In addition to the default variable table, variable tables can be specified as part of the following

  1. Run cycle definition
  2. Job stream definition
  3. Workstation definition

Variables syntax

Variables can be specified in two different formats

  • ^variable^ for resolution when the job is added to the plan
  • ${variable} for resolution at runtime when the job executes

Note Runtime resolution of variables is supported from TWS 8.6 and higher

Variable resolution

When specified using the format ^variable^, the value of a variable is resolved at the time the job is submitted to the plan. This means that it is either resolved when the plan is being built during JnextPlan processing or when the job or job stream is submitted.

TWS checks each of the variable tables, if specified in the job stream being submitted, in the order shown below to locate the variable to be resolved

  1. Run cycle variable table (if specified)
    1. First the run cycle variable table
    2. Then the variable table in the run cycle of the run cycle group
    3. Then the variable table in the run cycle group
  2. Job stream variable table (if specified in the job stream definition)
  3. Workstation variable table (if specified in the workstation definition where the job executes)
  4. Default variable table (named MAIN_TABLE by default)

If a job stream is submitted manually (or via an event rule) that contains references to a variable, TWS will check for existence of the variable in any variable table specified during submit, in the variable table associated with the Job Stream, workstation or default variable table. Any variable table associated with the run cycle is ignored for manually submitted job streams.

If a job is submitted manually (or via an event rule) that contains references to a variable, TWS will check for existence of the variable in any variable table specified during submit followed by the variable table associated with the workstation or default variable table.

Variables specified using the format ${variable} are added to the plan unchanged and the value of the variable is resolved at the time the job is executed. This format is only supported with advanced job types that support the “Variable resolution at runtime” option on the General tab e.g. executable, method, DataStage of the job definition in the workload console.

Variables specified in this format are resolved at runtime using the variable table specified in the order shown below

  1. Run Cycle variable table (if specified)
    1. First the run cycle variable table
    2. Then the variable table in the run cycle of the run cycle group
    3. Then the variable table in the run cycle group
  2. Job stream variable table (if specified in the job stream definition)
  3. Workstation variable table (if specified in the workstation definition where the job executes)

Note that the default variable table is not used when resolving variables at execution time.

Why use variable tables?

Variable tables can be used to supply different values to jobs, thereby allowing the same job definition to be reused such that the job definition effectively becomes a template job definition.

Taking an example of the TWS housekeeping process using the rmstdlist command to clean up older job log output, we can show the benefits of creating a template job to execute rmstdlist and using variable tables to deal with differences in how the job is executed.

It is common practice to use a workstation class to execute the “same” job on groups of workstations and this is especially true for jobs like the rmstdlist job. Ideally a single workstation class covering all TWS agents could be used, but differences in the OS, the job log retention period and possibly the installation path for TWS mean that this requires multiple workstation class definitions and multiple job definitions.
Using correctly configured variable tables this can be reduced to a single workstation class, a single template job definition and one or more variable tables. The remainder of this blog will show how this can be achieved.

First, we will need to understand what differences there are in the TWS environment such as

  • Different rmstdlist command – in Windows rmstdlist.cmd, UNIX/Linux rmstdlist
  • Different retention requirements – default is 14 days, but some workstations require 30 days
  • Different installation paths – some older TWS agents do not have the standard installation paths
  • Different TWS Instance owners that will need to execute the rmstdlist job
  • Any other differences can be listed here as each of these will need to become a variable value

Next decide on the default values that are the most common across the TWS estate as these values will be defined in the default variable table MAIN_TABLE.

Creating variable tables

The screenshot below shows the most common values for the variables added to the default variable table MAIN_TABLE. These values will be used by all the housekeeping jobs unless overridden by another variable table.

Next, we need to create a variable table for each workstation where there is a difference from the default values we entered above. For example, the TWS instance owner on my TWS backup MDM is the user twsbmdm and the installation path for TWS is different so I need to create a variable table to override those settings as shown below.

Add the variable table to the workstation definition – I use the workstation name as the name of the variable table to keep it simple! Remember that it will need JnextPlan to run before the variable table addition comes in to affect.

On my test Windows FTA the instance owner, path, retention period, command suffix and delimiter need to be overridden as shown below. Don’t forget to add the variable table to the workstation if required.

Repeat the above process to create any additional variable tables required to override the default values and add them to their respective workstation definition if required.

Using variables

Now the variable tables are set up, create a template job to execute the rmstdlist command. The sample job below is setup to execute on a workstation class named ORB_HKEEP_AGT that contains all of the agents in my TWS estate. The Login has been set to use the variable defined earlier.

On the Task screen, the command is made up primarily of the variables that we defined in the variables tables as can be seen below.


The full command is shown below

^ORB_HKEEP_PATH^^ORB_HKEEP_DLM^bin^ORB_HKEEP_DLM^rmstdlist^ORB_HKEEP_CMD_SUFFIX^ ^ORB_HKEEP_STDL^

As described in the section Variable resolution the variables will be resolved in a specific order which means that

  • The ORB_HKEEP_PATH will be taken from the workstation variable table and only if it does not exist there, will the default value be used from the MAIN_TABLE variable table
  • Likewise the ORB_KEEP_DLM (path delimiter) will be taken from the workstation table (using back slashes for Windows) if it exists. Otherwise it will use the default forward slash (for UNIX/Linux) from the MAIN_TABLE
  • The remaining variables will be resolved in the same way, thereby allowing for different retention periods for joblog output files at the workstation level

Summary

Although the example used above takes a little time to set up, once it is place every time a new workstation is added to TWS, adding an accompanying variable table with the housekeeping variables set is all that needs to be done to implement the joblog rmstdlst process.

Variable tables can also be used in a similar way to override the usual BAU activity. For example, if the MAIN_TABLE has a path setting to the current production version of a program e.g.

PGMPATH=/opt/orbdata/prod/program1

and you want to test a new version of “program1” adding a job stream level variable table to selected job streams with a setting of

PGMPATH=/op/orbdata/prod_new_version/program1

would override the path only for those selected job streams. Once testing is complete, removing the job stream variable table (or the settings with it) and updating the MAIN_TABLE variable setting to the new value would implement the new version for all jobs in production. Thus avoiding a manual update to multiple job streams.

Advantages

  1. Variable tables are centrally managed through the Dynamic Workload Console (DWC) or using the composer command line
  2. As variables within variable tables are resolved in a specific order, multiple levels of variable tables can be used to override generic values to more specific values
  3. Variable value is substituted at the time the job is submitted to the plan e.g. when JnextPlan runs or when submitted manually. Depending upon your requirements this could be considered as a disadvantage! If this isn’t what you want, see the next blog in the series covering “runtime variable resolution”

Disadvantages

  1. Variable value is substituted at the time the job is submitted to the plan e.g. when JnextPlan runs or when submitted manually. Depending upon your requirements this could be considered as an advantage!

The next blog in the series will look at TWS variables using runtime resolution.

Visits: 159

Author:

Ina Costin

Lorem Ipsum is simply dummy text of the printing and typesetting industry.