automating after hour cube builds

8/8/2019 Automating After Hour Cube Builds

1/3

Transformer 8.3 - Automating After Hours Cube BuildsBy John Anderson, Senior Consultant, CapTech Ventures

Transformer 8.3 represents the first major change to the application in years. It is now fully integrated

with Cognos 8. This includes data sources, cube access, and authentication. While this integrationmakes it easier to feed Transformer data, it complicates the process of automating cube builds. Withinthe UI, the steps to create secured views have changed, and the data sources are displayed slightlydifferently. The binary models now have .PYJ file extensions.This is a technique for building cubes on a regular basis, preferably after hours. It uses three sharedfolders on each server that runs Transformer in each environment (Dev, Test, and Prod). The folders arereferenced within Cognos by their UNC name. Cognos Administrators should have access to thesefolders.This article is based on a Windows environment; a similar concept can be applied in UNIX.Share Name | Purpose\cubes | Cubes are accessed here\cubes\build | Cubes are built here\cubes\model | Batch (other than cogconfig.bat) and PYJ filesTo enable Transformer to log into the server to build the cube, you must create an OS Sign on in themodel, store the user ID and password (using the local environments service credentials), then set it asan auto logon.

When you build the model in Development, you create the data source(s) using saved Query or ReportStudio reports, or you can select items directly from a package. Remember that the reports andpackage must exist in the same location in each environment, or you will need to adjust the data sourceto find them. The first time you build a cube in an environment, you need to create a data source forcube access within Cognos Connection by the studios.

CautionWhen creating an export, or performing an import, be careful not to include data sources. If you do, all data source connections in the target environment (not just the one for the package you are exporting /importing) will be replaced with the connections from the source environment. In other words, all Testdata sources will suddenly point to Dev databases, as will Production data sources. Also, do not includeCognos Roles, as it will have the same effect on security.

The .BAT filesSince the model contains environment-specific signons and build targets, you will need to modify it asyou promote it.

y create the model with the cube build target location set as\\servername\cubes\build\cubename.mdc

y as you promote the model, you will need to change servername in the target location entry, andthe OS signon, to match the new environment

y the model data source (reports feeding the model) properties should not need changing as longas their name and location does not change as you promote from Dev to Test to Prod.

y Copy the model to \\servername\cubes\model\. Transformer models are not handled withinCognos Connection. They must be manually copied from environment to environment, just likethe deployment package for your model data source.

You will need two system environment variables on each server used to build cubes. One iscog8_drive , whos value is the drive letter where the \Cube share is located (for example, D:). Theother is cog8_root , which points to the bin folder of the Cognos install (for example, d:\programfiles\cognos\c8\bin).There are four batch files you will use. One is a Cognos-provided file: \ProgramFiles\cognos\c8\bin\cogconfig.bat. You will need to edit this file, adding an EXIT command as the lastline to return control to the calling batch job. Be sure to save a backup first.The second is cubebuild.bat , which will run Transformer to build the enabled cubes in a model. You willneed to call this for each model you want run. This file stays the same in each environment, as it getsthe drive letter and Cognos\bin location from environment variables and the model name is passed as aparameter when it is called.


2/3

Blat, used below in a commented example for error trapping, is a Win32 command line utility that sendsemail using SMTP or posts to Usenet using NNTP. Blat can be downloaded from www.blat.net . Be sure tochange the settings below to match your configuration, if you use it.@ECHO OFF:::: = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =:: = CubeBuild.bat

:: = John Anderson, CapTech Ventures:: = This is the batch file for automating Transformer Cube Builds:: = It uses environment system variables %cog8_root% and %cog8_drive%:: = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =:::: Bug in 8.3, requires changing to Bin before running Transformer -::CD %cog8_root%:::: Build the cube using the argument passed in the calling command:: (from scheduler), save model when done::cogtr -n2 -p"%cog8_drive%\cubes\model\%1" -s"%cog8_drive%\cubes\model\%1":::: Check return code for error indication

::if %errorlevel% == 0 goto noerror:::error:::: Errors were encountered, so send email advising same:: this example uses BLAT installed on the server:::: set BLAT variables::::set [email protected]::set [email protected]::set subj=-s "! Transformer Encountered An Error !"::set note=Please check the model %1 for errors::set server=-server Emailserver:25::set debug=-debug -log c:\ja\blat.log -timestamp:::: run Blat::::blat - -to %eMailTo% -f %eMailFrom% %subj% -body "%note%" %server% %debug%:::noerror:::: No errors were encountered, so exit this batch job::Exit

The third batch file is run once nightly, and will stop Cognos, wait for the f ile locks to be released, callthe copy_cubes batch job, and restart Cognos.@ECHO OFF:::: = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =:: = restart_cognos.bat:: = John Anderson, CapTech Ventures:: = This is the batch file for stopping Cognos, copying new cubes:: = to the cubes folder, and restarting Cognos:: = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =:::: Change to the bin folder::


3/3

cd %cog8_root%:::: Stop Cognos, wait for job to finish::start /w cogconfig -stop:::: Use PING to delay execution of copy step for 20 seconds

:: while system drops file locks::PING 1.0.0.0 -n 20 -w 1000 >NUL:::: Call the job to copy all cubes from staging up to cubes,:: verifying the write and suppressing prompts,:: ` wait for copy to finish::start /w %cog8_drive%\cubes\model\copy_cubes:::: Start Cognos, saving current configuration in UTF-8 format,:: wait for job to finish::start /w cogconfig -s -utf8::

:: Exit this batch job::exitThe fourth batch file is a short one called from restart_cognos.bat that copies the cubes from\\cubes\build to \\cubes while Cognos is stopped.@ECHO OFF:::: = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =:: = copy_cubes.bat:: = John Anderson, CapTech Ventures:: = This batch file copies new cubes to the cubes folder:: = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =:::: Copy all cubes from build up to cubes, verifying the write and:: suppressing prompts, wait for copy to finish::copy /v /y %cog8_drive%\cubes\build\*.mdc %cog8_drive%\cubes\ :::: Exit this batch job::exitTo put all this together, you use Windows Scheduler. Create a scheduled task running cubebuild.bat foreach model you want to automate. The command line should look likeD:\ cubes \ model \ CubeBuild.bat model.pyj where m odel.pyj is the name of the model to run.Remember, the models must reside in \\cubes\model. Add the model name to the task name so you cankeep them straight.You will also need to schedule restart_cognos.bat to run at whatever frequency you need to refresh thecubes. Remember, this will bring Cognos down while it runs typically for about five minutes test iton your server to find out the downtime you will experience. In practice, these will all be scheduled forthe early hours of the morning.Although this looks like a hassle to maintain, its not. The majority of the work is in setting up theenvironment the first time. After that, its little more than the standard cube build-and-promote process.The batch files should not need to be modified once theyre in place, unless the environment, such as anemail server address, changes. If you move the Cog8 install location or cube drive, the environmentvariables will need updating.

automating after hour cube builds

Documents