Command Line Batch Jobs - EcoSys - Administration & Configuration - Hexagon

EcoSys System Administration

Language
English
Product
EcoSys
Search by Category
Administration & Configuration
EcoSys Version
9.0

Some data import and processing jobs can be scheduled to run outside of the Java application server. This is useful for long-running jobs that need to recur on a regular schedule, for example, importing forecasts from Primavera each week. You should work with your implementation team to determine whether batch jobs need to be configured, and if so, what their parameters and schedules should be.

Details on the import file format for XML-based external system imports can be found in the EcoSys Integration Guide.

Follow these steps to configure a command line batch script and job. The batch job script can be scheduled to run using the scheduling tools native to your server’s operating system. For Windows, you can use Control Panel > Scheduled Tasks. For Unix, you can configure a cron job.

Configuring the Files and Settings

  1. Create folder for the batch job files. Typically this is under your existing ESFM_HOME folder, perhaps called ‘batch’. We will refer to this folder as BATCH_HOME in this section.

  2. Copy the following files from ESFM_HOME to BATCH_HOME:

    FMServerSettings.properties, log4j2.xml , FMLicense.properties.

  3. In the copied FMServerSettings.properties file saved to BATCH_HOME, find the section titled Database Connection Pool and place a pound (#) symbol in front of the following line to comment it: fm.datasource.factory=com.ecosysmgmt.framework.context.JNDIDatasourceFactory

  4. In the copied FMServerSettings.properties file saved to BATCH_HOME, find the section titled Batch Job Connection Pool. Remove the pound (#) symbol to uncomment the following lines. Edit the values if needed. :

    fm.datasource.factory=com.ecosysmgmt.framework.context.FMPooledDataSourceFactory
    connectionPool.minSize=2
    connectionPool.maxSize=20
    datasource.JDBCUser=fmuser
    datasource.JDBCPass=fmuser
    datasource.JDBCDriver=oracle.jdbc.OracleDriver
    datasource.JDBCConnectString=jdbc:oracle:thin:@MYSERVERHOSTNAME:1521:ESFM

    OR if using sql server leave the above two Oracle specific lines commented and uncomment the SQL Driver and Connect string as shown below.

    datasource.JDBCDriver=com.microsoft.sqlserver.jdbc.SQLServerDriver
    datasource.JDBCConnectString=jdbc:sqlserver://MYSERVERHOSTNAME:1433;database=esfm

  5. Edit log4j2.xml, and change value of "ESFM-Application.log" to point to a location under your BATCH_HOME folder. For example, "BatchImport.log"

  6. Begin with the template batch script, located under batch\ecosys_batch_template.bat. You can copy and modify this file to perform the specific batch commands you desire using the steps below.

  7. Build the CLASSPATH for the job. To build a CLASSPATH for the job,

    1. Add all jar files from the EcoSys <BATCH_HOME>\lib directory.

    2. Add all jar files from the Oracle Primavera P6 Integration API\lib directory to the CLASSPATH, if the Primavera P6 Adapter is used for the job.

    3. Add the intgserver-wrapper.jar file from the EcoSys\alt\lib directory to the CLASSPATH, if the Primavera P6 Adapter is not used.

    4. Ensure that the CLASSPATH contains a supported database driver. It should be based on the EcoSys database platform, irrespective of whether the Primavera P6 Adapter is used or not.

Exporting System Settings to XML

For maximum flexibility, batch jobs run with their own system settings file that is independent of the settings in the EcoSys ("ESFM") database. The settings are stored in an XML file, which you need to export from the EcoSys application. To do this:

  1. Log on to EcoSys as an administrative user.

  2. Configure the import settings as desired. You do this by navigating to the Integration menu, then selecting each of the Import… menu items. For each one, configure the import settings as desired, and click Save. This saves the system settings to the database.

  3. After you have configured all the import settings as desired, use the System Administration > Export Application Settings. When prompted, save the systemsettings.xml file locally.

  4. Copy the resulting systemsettings.xml file to the BATCH_HOME folder where your batch job will be run. This file will be used when running batch jobs, and the settings in that file will apply only to that batch job.When changes to the batch job’s import settings are required, you can either edit the systemsettings.xml file directly or follow the steps above to export a new file.

Configuring Batch Job Logging

The settings for logging in batch jobs invoked from the command line (outside of the application GUI) can be controlled through the systemsettings.xml file specified for each batch command. This gives full control over the details logged to the batch job log (visible in the application GUI) as well as the log file written when a batch job runs. (These settings take the place of any properties specified in FMServerSettings.properties)

  • Settings that begin with "LOGGING_" control the level of detail written to the log file. These are primarily intended for system administrators.

  • Settings that begin with "BATCH_JOB_LOGGING_" control the level of detail written to the batch job log visible in the Reports > Batch Job Log screen within the application. These are intended for end users and application administrators.

For each command line batch job you are using, you may adjust the logging settings individually by editing the corresponding systemsettings.xml file. The settings in question and their recommended default values are:

<LOGGING_DEBUG_ENABLED>N</LOGGING_DEBUG_ENABLED>
<LOGGING_XML_ENABLED>Y</LOGGING_XML_ENABLED>
<LOGGING_ERROR_ENABLED>Y</LOGGING_ERROR_ENABLED>
<LOGGING_INFO_ENABLED>Y</LOGGING_INFO_ENABLED>
<LOGGING_WARN_ENABLED>Y</LOGGING_WARN_ENABLED>
<LOGGING_USER_ACTION_ENABLED>Y</LOGGING_USER_ACTION_ENABLED>
<LOGGING_USER_ACTION_PARAMS_ENABLED>Y</LOGGING_USER_ACTION_PARAMS_ENABLED>
<LOGGING_METHOD_TRACE_ENABLED>N</LOGGING_METHOD_TRACE_ENABLED>
<LOGGING_METHOD_PARAMS_ENABLED>N</LOGGING_METHOD_PARAMS_ENABLED>
<LOGGING_DATABASE_QUERY_ENABLED>N</LOGGING_DATABASE_QUERY_ENABLED>
<LOGGING_DB_QUERY_PARAMS_ENABLED>N</LOGGING_DB_QUERY_PARAMS_ENABLED>
<LOGGING_INCLUDE_DATE_FLAG>Y</LOGGING_INCLUDE_DATE_FLAG>
<LOGGING_INCLUDE_TIMESTAMP>Y</LOGGING_INCLUDE_TIMESTAMP>
<LOGGING_INCLUDE_LOG_TYPE>Y</LOGGING_INCLUDE_LOG_TYPE>
<LOGGING_INCLUDE_SERVER_THREAD_IDENTIFIER>Y</LOGGING_INCLUDE_SERVER_THREAD_IDENTIFIER>
<LOGGING_INCLUDE_USER_IDENTIFIER>Y</LOGGING_INCLUDE_USER_IDENTIFIER>
<LOGGING_INCLUDE_CLASS_NAME>Y</LOGGING_INCLUDE_CLASS_NAME>
<BATCH_JOB_LOGGING_ACTION_ENABLED>Y</BATCH_JOB_LOGGING_ACTION_ENABLED>
<BATCH_JOB_LOGGING_DEBUG_ENABLED>N</BATCH_JOB_LOGGING_DEBUG_ENABLED>
<BATCH_JOB_LOGGING_ERROR_ENABLED>Y</BATCH_JOB_LOGGING_ERROR_ENABLED>
<BATCH_JOB_LOGGING_INFO_ENABLED>Y</BATCH_JOB_LOGGING_INFO_ENABLED>
<BATCH_JOB_LOGGING_WARN_ENABLED>Y</BATCH_JOB_LOGGING_WARN_ENABLED>
<LOGGING_PROGRESS_ENABLED>Y</LOGGING_PROGRESS_ENABLED>
<LOGGING_PROGRESS_REPORT_INTERVAL_SECONDS>120</LOGGING_PROGRESS_REPORT_INTERVAL_SECONDS>
<BATCH_PROGRESS_REPORT_INTERVAL_SECONDS>60</BATCH_PROGRESS_REPORT_INTERVAL_SECONDS>
<LOGGING_SERVER_METRICS_ENABLED>Y</LOGGING_SERVER_METRICS_ENABLED>
<LOGGING_SERVER_METRICS_INTERVAL_SECONDS>120</LOGGING_SERVER_METRICS_INTERVAL_SECONDS>

Creating and Scheduling a Batch Script

Scheduling of batch import process basically involves creating a batch file (or shell script under Unix) which will set up appropriate CLASSPATH, then invoke the java command and using a platform dependent scheduler to execute the batch file on a predefined schedule. A sample batch file for windows platform is provided – copy and modify batch/ecosys_batch_template.bat as a starting point.To run several import process simultaneously you will need to duplicate the above mentioned folder and script for each import job. Since there will be a separate batch file for each process, they can be scheduled to run separately or independent of each other.

Command Line Parameter Reference

The following parameters always need to be provided when calling a command line batch job:

-a {Action} -u {Username} -p {Password} -f {SystemSettings.xml file}

The available {Action} values are listed below along with any additional parameters. Required parameters are listed in bold. All others are optional. Default values are underlined.

BulkCopy

BulkPublish

BulkRollup

CloseoutMinorPeriod

ExternalOperation
-o {operation}
-xml {parameter XML file used to override defined operation parameters}

GenXML – Deprecated. Use the EcoSys Web Service API.

ImportActionDef
-importDir {directory containing action XML files}

ImportActivityCode
-ds {datastore id}

ImportActual

ImportCompletePercent
-cv {cost object category value id}

ImportCostAccount
-ds {datastore id}

ImportDates
-cv {cost object category value id}

ImportForecast
-cv {cost object category value id}

ImportFormulaDef
-importDir {directory containing formula XML files}

ImportProject
-ds {datastore id}

ImportReportDef
-importDir {directory containing report XML files}

ImportResource
-ds {datastore id}

ImportRole
-ds {datastore id}

ImportSpreadsheet
-spreadsheet {name or internal id of EcoSys spreadsheet}
-file {file path to Excel spreadsheet to import}
-format {"excel" or "delimited"}
-importLogic {"update", "insert", or "ignore"}
-runtimeparam {runtime parameter}

ImportSpreadsheetDef
-importDir {directory containing spreadsheet XML files}

ImportXML – Deprecated. Use the EcoSys Web Service API.

RunAction
-action {action id or internal id}
-runtimeparam {runtime parameter}

RunActionBatch
-actionbatch {action batch id or internal id}
-runtimeparam {runtime parameter}

RunReportBatch
-rb {report batch id}
-runtimeparam {runtime parameter}

To pass runtime parameter values, use -runtimeparam specifying the parameter ID(s) and value(s). For example: -runtimeparam "location=SA" -runtimeparam "version=345"

System Settings Input Value Reference

The following list enumerates the input parameters used by batch processes. These values will appear in the systemsettings.xml file automatically when exported from the GUI. If you add or edit them manually, they need to be in XML tag format: <tagname>value</tagname>

BULK COPY

BULK_COPY_ROOT_SPREADSHEET_TYPE
BULK_COPY_SPREADSHEET_TYPE
BULK_COPY_TGT_SPREADSHEET_TYPE
BULK_COPY_SOURCE_VERSION
BULK_COPY_TARGET_VERSION
BULK_COPY_SOURCE_MAJORPERIOD
BULK_COPY_TARGET_MAJORPERIOD
BULK_COPY_SOURCE_VERSION_SHORT_NAME
BULK_COPY_TARGET_VERSION_SHORT_NAME
BULK_COPY_SOURCE_MAJORPERIOD_SHORT_NAME
BULK_COPY_TARGET_MAJORPERIOD_SHORT_NAME
BULK_COPY_FUNDING_SOURCE_ID
BULK_COPY_FUNDING_SOURCE_SHORT_NAME
BULK_COPY_INCLUDE_FS_DESCENDANTS
BULK_COPY_CATEGORY_TYPE_ID
BULK_COPY_CATEGORY_TYPE_SHORT_NAME
BULK_COPY_ADJUST_VALUES_FLAG
BULK_COPY_ADJUST_VALUES_BY
BULK_COPY_DELETE_TARGET
BULK_COPY_RETAIN_STATUS
BULK_COPY_CO_STATUS_FILTER
BULK_COPY_CO_STATUS_FILTER_FLAG
BULK_COPY_CO_CAT_VAL_FILTER
BULK_COPY_CO_CAT_VAL_FILTER_SHORT_NAME
BULK_COPY_CO_CAT_VAL_FILTER_FLAG
BULK_COPY_CO_FS_FILTER
BULK_COPY_CO_FS_FILTER_SHORT_NAME
BULK_COPY_CO_FS_FILTER_FLAG
BULK_COPY_CO_FS_FILTER_RELATION
BULK_COPY_CO_FS_FILTER_RELATION_VERSION_ID
BULK_COPY_CO_FS_FILTER_RELATION_VERSION_SHORT_NAME
BULK_COPY_PHASE_FILTER
BULK_COPY_PHASE_FILTER_SHORT_NAME
BULK_COPY_PHASE_FILTER_FLAG
BULK_COPY_NUM_MAJOR_PERIOD

BULK PUBLISH

BULK_PUBLISH_SOURCE_VERSION
BULK_PUBLISH_TARGET_VERSION
BULK_PUBLISH_SOURCE_MAJORPERIOD
BULK_PUBLISH_TARGET_MAJORPERIOD
BULK_PUBLISH_SOURCE_VERSION_SHORT_NAME
BULK_PUBLISH_TARGET_VERSION_SHORT_NAME
BULK_PUBLISH_SOURCE_MAJORPERIOD_SHORT_NAME
BULK_PUBLISH_TARGET_MAJORPERIOD_SHORT_NAME
BULK_PUBLISH_CO_STATUS_FILTER
BULK_PUBLISH_CO_CAT_VAL_FILTER
BULK_PUBLISH_CO_FS_FILTER
BULK_PUBLISH_CO_CAT_VAL_FILTER_SHORT_NAME
BULK_PUBLISH_CO_FS_FILTER_SHORT_NAME
BULK_PUBLISH_CO_STATUS_FILTER_FLAG
BULK_PUBLISH_CO_CAT_VAL_FILTER_FLAG
BULK_PUBLISH_CO_FS_FILTER_FLAG
BULK_PUBLISH_CO_FS_FILTER_RELATION
BULK_PUBLISH_CO_FS_FILTER_RELATION_VERSION_ID
BULK_PUBLISH_CO_FS_FILTER_RELATION_VERSION_SHORT_NAME
BULK_PUBLISH_PHASE_FILTER
BULK_PUBLISH_PHASE_FILTER_SHORT_NAME
BULK_PUBLISH_PHASE_FILTER_FLAG
BULK_PUBLISH_CURRENT_PHASE_ONLY
BULK_PUBLISH_NUM_MAJOR_PERIOD

BULK ROLLUP

BULK_ROLLUP_SPREADSHEET_TYPE
BULK_ROLLUP_TGT_SPREADSHEET_TYPE
BULK_ROLLUP_SOURCE_VERSION
BULK_ROLLUP_TARGET_VERSION
BULK_ROLLUP_SOURCE_MAJORPERIOD
BULK_ROLLUP_TARGET_MAJORPERIOD
BULK_ROLLUP_SOURCE_VERSION_SHORT_NAME
BULK_ROLLUP_TARGET_VERSION_SHORT_NAME
BULK_ROLLUP_SOURCE_MAJORPERIOD_SHORT_NAME
BULK_ROLLUP_TARGET_MAJORPERIOD_SHORT_NAME
BULK_ROLLUP_CO_STATUS_FILTER
BULK_ROLLUP_CO_CAT_VAL_FILTER
BULK_ROLLUP_CO_FS_FILTER
BULK_ROLLUP_CO_CAT_VAL_FILTER_SHORT_NAME
BULK_ROLLUP_CO_FS_FILTER_SHORT_NAME
BULK_ROLLUP_CO_STATUS_FILTER_FLAG
BULK_ROLLUP_CO_CAT_VAL_FILTER_FLAG
BULK_ROLLUP_CO_FS_FILTER_FLAG
BULK_ROLLUP_CO_FS_FILTER_RELATION
BULK_ROLLUP_CO_FS_FILTER_RELATION_VERSION_ID
BULK_ROLLUP_CO_FS_FILTER_RELATION_VERSION_SHORT_NAME
BULK_ROLLUP_TO_FS_OWNER_ONLY
BULK_ROLLUP_FUNDING_SOURCE_ID
BULK_ROLLUP_FUNDING_SOURCE_SHORT_NAME
BULK_ROLLUP_INCLUDE_FS_DESCENDANTS
BULK_ROLLUP_NUM_MAJOR_PERIOD