Creating a Custom Metric to Check SAPRouter SNC Certificate Expiry

This metric and script checks the SAPRouter SNC certificate expiry and gives an alert depending on how many days left until expiry.

Define SAProuter in Focused Run LMDB

Prerequisite: Make sure you have installed the SAP Host Agent and performed Outside Discovery on the SAPRouter host.

Go to the LMDB of you Focused Run system, then go to Single Customer Network:

Switch Namespace and go to “Technical Systems” and choose “Unspecific Standalone Application System” from the drop-down, then hit Create.

Enter:

Enter the Extended SID:

Add Software Component Version:

Search on the software, and then select the correct release, then Add. E.g.:

Save

Add Installation Path and Required LMDB Attributes. NOTE: Find out the relevant software installation path of the SAPRouter on the server.

Perform Simple System Integration.

Configure Script and Custom SAPHostCtrl Operation

Prerequisite: The script requires the SAPRouter SNC Certificate to be properly installed, and the PSE credentials in the cred_v2 file for the account that runs the SAPRouter service.

A script is scheduled to run daily as the same user that runs the SAPRouter service. The script reads the expiry date of the from the PSE and writes it to a file in JSON format so that the FRUN metric can read and interpret it. The script by default has SAPRouter home directory as /usr/sap/saprouter or <drive>:\usr\sap\saprouter, and it can be changed. This will be denoted as <SAPROUTER_HOME>. The PSE file variable can also be changed.

Linux/Unix

Create a new executable file in the SAPRouter home directory and copy the code in Appendix A. Ensure it is owned and executable by the user that runs the SAPRouter service and belongs to group sapsys. E.g.,

chown <saprouter user>:sapsys saprouter_expiry_days.sh
chmod 744 saprouter_expiry_days.sh

In the script, adjust the following variables:

  • SAPROUTER_HOME
  • SECUDIR (The SECUDIR folder is where the PSE file resides)
  • SAPROUTER_USER
  • PSE

Test the Script. Switch to the user that runs the SAPRouter service, then execute the script:

/usr/sap/saprouter/saprouter_expiry_days.sh

Look for file NUMDAYS.json. It should contain e.g.:

{type:integer, name:NumDays, value:136}

Schedule script in the root crontab as follows:

# Run SAProuter SNC Certificate expiry check
00 09 * * * su - <saprouter user> -c /usr/sap/saprouter/saprouter_expiry_days.sh

Windows

Create a batch file (*.bat) in the SAPRouter home directory. In the script, adjust the following variables:

  • SAPROUTER_HOME
  • SECUDIR (The SECUDIR folder is where the PSE file resides)
  • PSE

Test the Script. Logon as the user that runs the SAPRouter service. Right-click on the bat script and “run as Administrator”. Alternatively, if you are not logged as the SAPRouter service account, you can “run-as” the user that runs the SAPRouter service and execute the script that way.

runas /user:<domain or hostname>\<SAPRouter service user> <SAPROUTER_HOME>\saprouter_expiry_days.bat

NOTE: The SAPRouter service user requires “Allow log on locally” user rights.

Look for file NUMDAYS.json. It should contain e.g.:

{type:integer, name:NumDays, value:136}

Schedule the Script. Open a command prompt as administrator. Execute the command:

SCHTASKS /CREATE /RU <DOMAIN>\<saprouter user> /RP <password> /SC DAILY /TN "Run SAPRouter SNC Certificate expiry check" /TR "C:\Windows\System32\cmd.exe /C \"<SAPROUTER_HOME>\saprouter_expiry_days.bat\"" /ST 09:00 /RL HIGHEST

Note: The start time e.g., 09:00 is in 24 hour format. You should see:

Run the new task to test it:

SCHTASKS /RUN /TN "Run SAPRouter SNC Certificate expiry check"

You should see:

Check the timestamp of the file NUMDAYS.json.

Create Custom Operation for saphostctrl

To load these values into Focused Run, a custom operation for saphostctrl needs to be created.

Linux/Unix

As Root: Create the following custom operations conf file:

/usr/sap/hostctrl/exe/operations.d/checksnccert.conf

Enter the following into the conf file:

Command: cat /usr/sap/saprouter/NUMDAYS.json
Description: Check number of days to SNC certificate expiry
ResultConverter: flat
Platform: Unix

Test the custom operation as follows:

/usr/sap/hostctrl/exe/saphostctrl -function ExecuteOperation -name checksnccert

Result should be:

Webmethod returned successfully
Operation ID: 06C635D6863A1EEDB6BC5C819EE199D7

----- Response data ----
description=Check number of days to SNC certificate expiry
{type:integer, name:NumDays, value:169}
exitcode=0

Windows

Create the following custom operations conf file:

C:\Program Files\SAP\hostctrl\exe\operations.d\checksnccert.conf

Enter the following into the conf file:

Command: FOR /F "tokens=* delims=" %x in (<SAPROUTER_HOME>\NUMDAYS.json) DO @echo %x
Description: Check number of days to SNC certificate expiry
ResultConverter: flat
Platform: Windows

In a Command Prompt as Administrator, test the custom operation as follows:

"C:\Program Files\SAP\hostctrl\exe\saphostctrl" -function ExecuteOperation -name checksnccert

Result should be:

Webmethod returned successfully
Operation ID: 00155D657F901EEDBCF9E32BC564F964

----- Response data ----
description=Check number of days to SNC certificate expiry
{type:integer, name:NumDays, value:360}
exitcode=0

Create Custom Alert in Focused Run

Create a new monitoring template for SAPRouter here:

Enter into Expert Mode. Create a new Alert with the following settings:

NOTE: If the tick box “Do not Group Individual Occurrences” is ticked, it will alert at each data collection regardless of its previous rating, and not only at a change of rating.

Create Custom Metric In Focused Run

Create a new Metric with the following settings:

Data Collection

All parameters of the Data Collector which are fixed should have the “Configure” box unticked, and the common parameters pre-filled with the correct values.

The “Custom Operation” parameter is the saphostctrl operation “checksnccert” created earlier. The “Metric Name” parameter is the name of the metric in the JSON output file produced by the script, which is “NumDays”.

Threshold

The threshold picks up the integer as returned by the script; if it falls to below or equals 10 days, then raise a yellow alert, if below or equals 5 days, then raise a red alert. Choose whatever threshold values suit you.

Assignment

Assign the Metric to the Alert:

Activate the Alert

Apply SAPRouter Template and Check Monitoring

TIP: Initially set the collection interval to 5 minutes and apply template to see whether it is collecting data. Then you can set the collection interval back to daily.

You should see this in the Monitoring Application:

Appendix A: saprouter_expiry_days.sh

#!/bin/bash
# Outputs the expiry date of the SNC Certificate and calculates the number of days
# If the number of days falls below threshold, it sends an alert
# Written by Tony Swietochowski

SAPROUTER_USER=saprouter
SAPROUTER_HOME=/usr/sap/saprouter
SECUDIR=$SAPROUTER_HOME/sec
PSE=local.pse
HOSTNAME=$(hostname -f)

# Check  for saprouter user
[[ ! "$USER"=="$SAPROUTER_USER" ]] && echo "This script must be run using the $SAPROUTER_USER user. Exiting." && exit 1

EXPIRYDATE=$(${SAPROUTER_HOME}/sapgenpse get_my_name -p $PSE -n validity 2>&1 | grep NotAfter | awk -F\( '{print$2}' | cut -c -6)
NUMDAYS=$(echo $(( ($(echo $(date --date="$EXPIRYDATE" +%s) - $(date -d $(date +%y%m%d) +%s)) )/86400 )))
# Above method based on https://stackoverflow.com/questions/4946785/how-to-find-the-difference-in-days-between-two-dates

echo $NUMDAYS > $SAPROUTER_HOME/NUMDAYS
echo \{type:integer, name:NumDays, value:$NUMDAYS\} > $SAPROUTER_HOME/NUMDAYS.json

Appendix B: saprouter_expiry_days.bat

@echo off

REM Outputs the expiry date of the SNC Certificate and calculates the number of days
REM If the number of days falls below threshold, it sends an alert
REM Written by Tony Swietochowski

set SAPROUTER_HOME=D:\usr\sap\saprouter
set SECUDIR=%SAPROUTER_HOME%\sec
set SNC_LIB=%SECUDIR%\sapcrypto.dll
set PSE=local.pse

for /f "tokens=2 delims=\" %%i in ('whoami') do set THISUSER=%%i
FOR /F "tokens=* USEBACKQ" %%F IN (`hostname`) DO (SET HOSTNAME=%%F)

chdir /d %SAPROUTER_HOME%

for /f "tokens=2 delims=(" %%a in ('%SAPROUTER_HOME%\sapgenpse.exe get_my_name -p %PSE% -n validity ^2^>^&^1 ^| findstr /l "NotAfter"') do set DATESTRING=(%%a

set expiry_year=20%DATESTRING:~1,2%
set expiry_month=%DATESTRING:~3,2%
set expiry_day=%DATESTRING:~5,2%

set current_year=%date:~-4%
set current_month=%date:~4,2%
set current_day=%date:~7,2%

set "from=%current_month%-%current_day%-%current_year%"
set "to=%expiry_month%-%expiry_day%-%expiry_year%"
echo Wscript.Echo DateDiff("d", "%from%", "%to%") > %TEMP%\tmp.vbs

for /f %%a in ('cscript /nologo %TEMP%\tmp.vbs') do set /a "numdays=%%a"

del %TEMP%\tmp.vbs

echo Number of days to SAPRouter certificate expiry: %numdays% > %SAPROUTER_HOME%\NUMDAYS.log
echo {type:integer, name:NumDays, value:%numdays%} > %SAPROUTER_HOME%\NUMDAYS.json

<< This blog was originally posted on SAP Focused Run Guru by Tony Swietochowski (DXC). Repost done with permission. >>

SAP Focused Run certificate expiry monitoring

SAP Focused Run can be used to monitor expiry of a lot of different type of certificates:

  • ABAP PSE certificates (read below)
  • Host agent PSE certificates (read below)
  • SSL certificates (see this blog)
  • Cloud Connector BTP certificate expiry (read below)
  • SNC certificate expiry (read below)
  • SNC expiry in SAP router

ABAP PSE certificate monitoring

In system monitoring you can monitor if ABAP system certificates are expiring or are already expired. SAP provides a standard metric for this based on configuration validation rule SAP ABAP PSE CERTIFICATES CHECK. This rule is taking all certificates that are to be expired in the next 7 days and all expired certificates.

This rule has two setbacks:

  • 7 days upfront warning might be too short
  • It reports all expired certificates that are already years expired and don’t harm the system

New rule for ABAP PSE certificate monitoring

For the improved setup we will defined 2 new rules:

  • To be expired certificates, with date selection -30 till tomorrow
  • Expired certificates, with date selection today till next 30 days.

Start with creating the policy in configuration validation (more on configuration validation in this blog):

Add a new policy Z_ABAP_PSE_TO_BE_EXP for the to be expired, and give it this syntax:

Add a new policy Z_ABAP_PSE_EXP for the expired ones:

Both are basically a copy of the standard SAP one with the changed dates.

Please note you need to change both the dates in the Compliant and Non-Compliant section.

Create custom metric

Now you can create a custom metric using the newly created rule. For full description on how to create custom metrics, read this blog. Create the custom metric in the ABAP system template:

Refer to the newly create custom CoVa (configuration validation) rule:

And set the threshold:

For the to be expired certificates, we only want warning.

We repeat the same for the expired certificates, but now we make it a red alert by setting the thresholds differently.

Make sure both are now activated for monitoring:

Trouble shooting

There might be false alerts raised for this metric. This is for both the custom and standard SAP metric. The source is usually the data collection.

SAP note 3138046 – False Alert ABAP PSE certificates expiring in Focused Run describes the checks to perform.

Go to the configuration and configuration validation FIORI tile and select the system and search for the PSE certificates data collection details:

Monitoring Host Agent PSE Certificate Expiry

You have configured SSL for enabling secure connections to SAP Host Agent. In such a case you need to regularly update or extend the validity of the SAP Host Agent PSE certificate. With Focused run System Monitoring you can create custom metric for monitoring SAP Host Agent PSE certificate expiry.

Setup Monitoring Host Agent PSE Certificate Expiry

Step1: Navigate to Host Template maintenance

SAP Host Agent resides at OS level of each host of systems that you are monitoring using Focused Run. Hence you need to monitor the PSE certificate for SAP Host Agents of each host in your customer network.

In our recent blog we have explained how you can configure URL Certificate monitoring in Health Monitoring in SAP Focused Run. However configuring certificate monitoring for each SAP Host Agent in your customer network using Health Monitoring will be very cumbersome.

An easier way will be to setup a custom metric in the host level system monitoring template which when activated will be automatically applied to all hosts for which the monitoring template is used.

For template maintenance you can navigate to System Monitoring Template Maintenance app on the Focused Run launch pad.

In the template maintenance app navigate down to the Host (Server) node and then to the respective template which you want to edit and click on Edit button.

Step2: Create Custom Metric

In order to create a custom metric you need to activate expert mode. For this click on the Expert Mode button as shown below.

After enabling expert mode you can click on Create button. Select Metric in the drop down from Create button.

In the Metric creation pane, under Overview tab enter the details as shown below.

Then in the Data Collection tab, provide the details as shown below.

For the URL field you need to mention the URL https://$SAP_ComputerSystem.FQDName$:1129/SAPHostControl/?wsdl in which the expression $SAP_ComputerSystem.FQDName$ will dynamically resolve the respective FQDN of the host picked up from LMDB. Hence it is important to select the check box under Placeholder and select LMDB under Placeholder type.

Then navigate to the Threshold tab and provide the threshold as shown below. In this example the threshold is Yellow if certificate is expiring in 30 days and Red if its expiring in 15 days.

Then click on Next button to navigate to the Assignment tab where we assign this metric to an existing alert. In this step just click on the Finish button to save the metric. In this step we don’t assign to any alerts yet as we are yet to create a custom alert for this custom metric.

Step3: Create custom Alert and assign to custom Metric

In the expert mode maintenance go to the Create button and then from the drop down select Alert.

Enter the details as shown below and click on Next button.

Then in the Assignments tab you will see the custom metric you just created. Select the check box for this metric and click on Finish to save the alert.

Now your custom metric is ready together with alerting active.

To activate this template update on all hosts navigate to Managed Objects tab for the template and click on Apply and Activate button.

Upon activation the new metric will be available in system monitoring as shown below.

For more details on how to setup SSL for SAP Host Agent you can refer to the SAP documentation here.

SNC certificate expiry monitoring

SNC Certificate expiry is very critical for ABAP systems where SNC logon is used. As soon as the certificate expires users will no more be able to login to system.

In System Monitoring for ABAP systems SAP provides a standard metric called Expiring PSE Certificates which monitors all PSE certificates that includes SNC certificates. When this metric turns red it will tell how many certificates are expiring/expired but not which one. Hence its beneficial to monitor separately the status of SNC Certificates.

This monitoring can be achieved by using Configuration Validation.

You can create a custom policy with the below XML code that checks the PSE_CERT store to check on status of SNC Certificate.

<?xml version="1.0" encoding="utf-8"?>
<targetsystem desc="SNC Certificate about to expire" id="ASML_SNCERT_TOBE_EXP" multisql="Yes" version="0000">
<configstore name="PSE_CERT" system_type="ABAP">
<checkitem desc="Valid in range" id="1" operator="">
<compliant>APPLICATION like '%SNCS%' and CONTEXT like '%' and TYPE like '%' and SUBJECT like '%sapci%' and ISSUER like '%' and SERIALNO like '%' and VALID_FROM like '%' and ( ( ( replace_regexpr ('^$' IN valid_to WITH (replace(current_date, '-', '')||replace(current_time, ':', '')))) &gt; to_number(replace(add_days(current_date, 30), '-', '')||replace(current_time, ':', '') ))
or (( replace_regexpr ('^$' IN valid_to WITH (replace(current_date, '-', '')||replace(current_time, ':', '')))) &lt; to_number(replace(add_days(current_date, 0), '-', '')||replace(current_time, ':', '') )) )</compliant>
<noncompliant>APPLICATION like '%SNCS%' and CONTEXT like '%' and TYPE like '%' and SUBJECT like '%sapci%' and ISSUER like '%' and SERIALNO like '%' and VALID_FROM like '%' and not ( ( ( replace_regexpr ('^$' IN valid_to WITH (replace(current_date, '-', '')||replace(current_time, ':', '')))) &gt; to_number(replace(add_days(current_date, 30), '-', '')||replace(current_time, ':', '') ))
or (( replace_regexpr ('^$' IN valid_to WITH (replace(current_date, '-', '')||replace(current_time, ':', '')))) &lt; to_number(replace(add_days(current_date, 0), '-', '')||replace(current_time, ':', '') )) )</noncompliant>
</checkitem>
</configstore>
</targetsystem>

Note: In the above XML code, SUBJECT like ‘%sapci%’, you have to replace sapci with a hostname pattern that you use for all your SAP ABAP system hostnames.

Once you have created the policy, you can directly activate alerting on the policy.

For that go to the Configuration and Security Analytics app in the Advanced Configuration Monitoring area.

In the app navigate to Configuration Validation Alert Management under the the Related links tab.

In the Alert Management app click on Create button to create the alerting for the policy you just created.

In the next popup, enter details as shown below and then click on Select policy to select the policy for which you want to activate the alert.

In the next popup select the policy by clicking on the policy name.

Now click on Save to activate the alerting.

Additionally you can also activate email notification on the alert. For this in the Alert Management main screen, select the alert and then click on Notification/Outbound.

In the next popup select the notification variant and save.

Cloud Connector BTP certificate expiry

With SAP Focused Run 3.0 FP 2 SAP has delivered a configuration validation policy using which you can monitor Cloud Connector BTP Certificate expiry.

Note: For using this functionality you just have to ensure that you have registered your cloud connector managed system to your Focused Run System and have performed SSI for the cloud connector managed system.

You can do the monitoring via the Configuration & Security Analytics app in the Advanced Configuration Monitoring area of Focused Run Launchpad.

In the app click on the select button to select the policy to monitor Cloud Connector BTP Certificates.

In the pop-up you can search for SCC and then select the policy SAP_SCC_PSE_CERT. This is the standard policy delivered by SAP that performs configuration validation on Cloud Connector BTP Certificate expiry.

This policy monitors if a BTP certificate in cloud connector system is going to expire in less than 30 days or is already expired.

The first view upon selecting the policy is the Systems view. In this view all SAP Cloud Connector systems registered in Focused Run system are listed. A systems is marked as non compliant if it has any certificate which is going to expire in less than 30 days or has any expired certificates. This view also tells how many certificates are there in the system and how many are non compliant.

You can click on the row to see the details of the certificates which are expiring or expired.

To know more about Configuration and Security Analytics you can refer here.

Also you can find more detail at SAP documentation here.

<< This blog was originally posted on SAP Focused Run Guru by Manas Tripathy (Simac) and Frank Umans. Repost done with permission. >>

SAP Focused Run monitoring applications

This blog will explain specific items to keep in mind when monitoring certain applications.

Applications discussed and explained:

  • Adobe document servers (ADS)
  • BW systems
  • Cloud connectors
  • Content servers
  • ECC and S4HANA servers
  • EWM (enterprise warehouse management) servers
  • GTS (global trade system) servers
  • Netweaver gateway FIORI hub systems
  • SCM (supply chain management) servers
  • SLT servers
  • Web dispatchers

For each system we explain the monitoring of productive and non-productive system.

Adobe document server (ADS) application monitoring

Adobe Document Server (ADS) is used to generate PDF’s for output and/or interactive PDF forms.

Monitoring productive ADS systems

When monitoring a productive system, you will need to finetune the monitoring templates for:

  • SAP J2EE 7.20 – 7.50 Application template, for the JAVA application
  • SAP J2EE 7.20 – 7.50 Technical instance template, for the JAVA application servers
  • System host template
  • Database template

JAVA APPLICATION TEMPLATE for adobe document server monitoring

Make sure you cover in the JAVA application template the following items:

Availability:

  • JAVA HTTP availability
  • Expiring certificates
  • JAVA license expiry

From the JAVA instance template make sure to cover the following items:

  • J2EE application status
  • Instance HTTP availability and logon
  • JAVA server node status
  • GC (Garbage collection)

Fine tune the metrics so you are alerted on situation where the system is having issues.

ADobe document server template for monitoring

ADS has a specific Technical instance template.

Make sure you activate it:

Most important here is the first one: ADS availability. Please make sure you are alerted when this one is not available.

BW application monitoring

BW systems are at the often used as reporting systems within an SAP landscape.

The basic monitoring of a BW system is the same as for any ABAP based system.

For a BW system some numbers are typically higher than on an ECC or S4HANA system. Response times of 1.5 seconds would indicate horrible performance on ECC, but are normal on a BW system.

Process chain monitoring

BW uses process chains. To monitor process chains, read this dedicated blog.

Cloud connector application monitoring

The Cloud connector is used between on premise systems and Cloud solutions provided by SAP.

Monitoring of cloud connector focuses on availability and connectivity.

Monitoring productive cloud connector systems

The cloud connector template contains all the needed elements out of the box:

If your landscape has only one cloud connector that is also used for non-productive systems, you might find a lot of issues in the non-productive system. Like expired certificates, channels not working, many logfile entries. If the cloud connector is very important for your business, it is best to split off the productive cloud connector from the non-productive usage. This way you can apply sharp rule settings for production: even single issue will lead to alert. While on non-production the developers will be making a lot of issues as part of their developer process.

Monitoring non-productive cloud connector systems

In your landscape you might have a non-productive cloud connector that is used for testing purposes. In the non-productive cloud connectors you might apply a different template with less sensitive settings on certificates, logfiles and amount of tunnels that are failing.

Relevant OSS notes for cloud connector monitoring

3391143 – Cloud Connector system is not coming into SAP Focus Run LMDB

Content server application monitoring

Content servers are often used to store attachment and data archiving files. They are technical systems with usually no direct access for end user. End users normally fetch and store data form content server via an ABAP or JAVA application.

Technical setup for content server monitoring

The technical setup for monitoring content server in SAP Focused Run is described in detail in a PDF attached to OSS note 3151832 – SAP Content Server 6.40/6.50/7.53 Monitoring with SAP Focus Run. There is no need to repeat here.

The main part of content server monitoring is availability.

ABAP connection to content server monitoring

In some cases both your ABAP stack and content server are up and running, but communication between them is failing on application level. This leads to not working system for end users. Root causes can be firewall issues, certificate issues, or somebody altered settings.

To test the ABAP system connection to content server a custom ABAP program is needed. See this blog. You can schedule the program in batch and set up a new custom metric to capture the system log entry written by the program.

System host template for content server monitoring

For system host the regular CPU, memory, disc template is sufficient. Finetune the thresholds to your comfort level.

Database template for content server monitoring

Important items of the database template:

  • Database availability
  • Database health checks
  • Backup

In most installations it is chosen to install Content Server with the SAP MaxDB database (similar to LiveCache).

Relevant OSS notes for content server monitoring

ECC and S4HANA application monitoring

ECC and S4HANA systems are at the core of each SAP landscape, and most vital to the business.

When monitoring a productive system, you will need to finetune the monitoring templates for:

  • ABAP 7.10 and higher Application template, for the ABAP application
  • ABAP 7.10 and higher Technical instance template, for the ABAP application servers
  • System host template
  • Database template

ABAP application template

Make sure you cover in the ABAP application template the following items:

Availability:

  • Message server HTTP logon
  • System logon check
  • RFC logon check
  • License status
  • Certificates expiry
  • Update status

Performance and system health:

  • Critical number ranges
  • Enqueue lock % filled
  • SICK detection
  • Dumps last hour
  • Update errors last hour
  • Cancelled jobs last hour
  • Long running work processes and jobs (see blog)

Security:

  • Global changeability should be that the system is closed
  • Locking of critical users like SAP* and DDIC (see blog)

Fine tune the metrics so you are alerted on situation where the system is having issues.

ABAP application server template

Make sure you cover in the ABAP application server template the following items:

Availability:

  • Local RFC logon test
  • Local HTTP logon test
  • Local Logon test
  • Message server disconnects (see blog)

Application server performance and health:

  • Amount of critical SM21 messages
  • No more free work processes (see blog)
  • Update response times

You can consider to setup extra custom metrics for the application servers:

System host template

For system host the regular CPU, memory, disc template is sufficient. Finetune the thresholds to your comfort level.

Database template

Important items of the database template:

  • Database availability
  • Database health checks
  • Backup

Functions monitoring

Next to the availability and performance mentioned above, check also for monitoring certain functions:

EWM (enterprise warehouse management) application monitoring

EWM systems are at the often used as stand alone systems that make sure logistics and warehousing can keep running at high availability. If the connected ECC or S4HANA system is down, EWM can continue to support logistics operations.

EWM can be older version based on SCM/BI system core. Newer EWM systems are using S4HANA with EWM activated as standalone.

Extra in an EWM system are the use of qRFC and the CIF (Core interface). And many EWM systems have users that interact with the system via ITS GUI based handheld scanners.

EWM systems are at the often used as stand alone systems that make sure logistics and warehousing can keep running at high availability. If the connected ECC or S4HANA system is down, EWM can continue to support logistics operations.

EWM can be older version based on SCM/BI system core. Newer EWM systems are using S4HANA with EWM activated as standalone.

Extra in an EWM system are the use of qRFC and the CIF (Core interface). And many EWM systems have users that interact with the system via ITS GUI based handheld scanners.

CIF monitoring

The CIF is the core interface between SCM and ECC system. The interface typically uses RFC and qRFC. And it is working both ways.

Setup for the CIF specific RFC’s and qRFC’s the monitoring:

Handheld scanners

Many EWM systems are having interaction with scanners via the ITS server. Basically this is a small web page on a scanning device.

Make sure you monitor the availability of the URL’s that the scanners are using. More on URL monitoring can be found in this blog.

GTS (global trade system) application monitoring

GTS systems are at the not frequent in use. When in use they do play a vital role in import and export business scenario’s when good are crossing borders.

Since a GTS system is normally installed, and often no to little maintenance and software changes are performed on the system. Also basis teams tend not to look at it too often, since it normally runs stable.

In case of non-availability of GTS, ECC scenario’s linked to GTS might fail and can causes severe business disruptions.

For this reason it is important to set up monitoring in FRUN for your GTS system and also configure mail alerts in case of issues. They will not happen too often, but when they happen you can act fast. This will also save the basis team spending a lot of time on checking GTS system for log (most cases, the checks are good).

When monitoring a productive system, you will need to finetune the monitoring templates for:

  • ABAP 7.10 and higher Application template, for the ABAP application
  • ABAP 7.10 and higher Technical instance template, for the ABAP application servers
  • System host template
  • Database template

The next step is to set up interface monitoring for RFC from the ECC system towards the GTS system.

Netweaver Gateway Fiori hub system application monitoring

Netweaver Gateway systems are used to host Fiori applications.

Netweaver gateway template

For Netweaver gateway, also assign and fine tune the Gateway template:

The important custom check on URL availability is best to setup as well: read this blog for instructions.

Consider to setup interface monitoring for RFC‘s from End User to Gateway and Gateway to ECC backends.

SCM (supply chain management) application monitoring

SCM systems are at the often used logistics optimization systems. They are mainly used in combination with traditional ECC systems. They are less needed in combination with S4HANA systems (or you can use the embedded SCM of HANA).

The core of an SCM system is a BI system. Many data is using similar extractors and process chains as a BI system. Hence follow the tuning needed for a BI system.

Extra in an SCM system are the LiveCache and the CIF (Core interface).

The basic monitoring of an SCM system is the same as for any ABAP based system.

For an SCM system some numbers are typically higher than on an ECC or S4HANA system. Response times of 1.5 seconds would indicate horrible performance on ECC, but are normal on an SCM system.

LiveCache monitoring

LiveCache is normally running on a MaxDB database.

So it is important to activate, assign and finetune the metrics for the MaxDB database:

Focus on:

  • Availability
  • Backup
  • Performance

Next to the database, you also need to activate, assign and finetune the LiveCache specific application template:

This template contains the primary elements to monitor for the LiveCache functions like:

  • Availability of LiveCache as a function
  • Structure check for LiveCache
  • Memory issues for LiveCache specifically

Fine tune the metrics so you are alerted on situation where the system is having issues.

CIF monitoring

The CIF is the core interface between SCM and ECC system. The interface typically uses RFC and qRFC. And it is working both ways.

Setup for the CIF specific RFC’s and qRFC’s the monitoring:

Process chain monitoring

SCM uses process chains. To monitor process chains, read this dedicated blog.

SLT system application monitoring

SLT systems are mainly used to replicate data from source systems like ECC and S4HANA towards target systems like Enterprise HANA, HANA cloud and other data pool systems.

SLT DMIS template for SLT system

For SLT systems, apply the SLT DMIS template:

In the SLT system itself, make sure job /1LT/IUC_HEALTH_C with program R_DMC_HC_RUN_CHECKS runs. This will collect data that is needed for SLT itself, but which is also re-used by SAP Focused Run.

OSS notes to apply and check:

Anyhow you should make sure to regularly apply the notes for the DMIS component. See this blog.

SLT DMIS dummy template backend system

For SLT to work, the DMIS component is installed in both the SLT system and the backend system. For the backend system SLT component, Focused Run will pick up the template as well. But this will not make any sense in monitoring, since it is the source system and not the SLT system.

For this reason, set up a dummy empty template with every monitoring item disabled:

Assign this dummy template to your backend system.

SLT integration monitoring

Set up the SLT integration monitoring to monitoring communication.

Web dispatcher application monitoring

Standalone web dispatchers are used to load balance web traffic towards ABAP and/or JAVA systems. Common use case is to have web dispatcher for a large Netweaver Gateway FIORI installation.

Monitoring productive cloud web dispatchers

Monitoring of web dispatchers focuses on availability, connectivity and performance.

The web dispatcher template contains most needed elements out of the box:

Issues with performance are often caused by limitations set in the web dispatcher configuration. Keep these settings active.

You might want to add specific custom metric to monitor the most important URL for your web dispatcher. Read more in this specific blog.

Next to this setup the normal host monitoring to make sure the file system and CPU of the web dispatcher are not filling up and causing availability issues for the web dispatcher function.

Monitoring non-productive web dispatcher systems

For monitoring non-productive web dispatcher systems, it is normally sufficient to restrict to host and availability monitoring.

Relevant OSS notes for web dispatcher monitoring

3373764 – Issue with Content Server on Web dispatcher templates

LMDB OSS notes

3104059 – Troubleshooting ABAP Data supplier populating LMDB

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans. Repost done with permission. >>

SAP Focused Run workmode management

There will be planned maintenance on SAP systems, databases, operating systems and the below infrastructure. During this planned maintenance, you don’t want any alerts and mails from SAP Focused Run, since the application is not available for a planned time.

Using the work mode management function you plan these maintenance events.

Questions that will be answered in this blog are:

  • How can I set up planned maintenance for systems monitored by SAP Focused Run to temporarily suppress the alerts during maintenance?
  • How can I see the current maintenance and planned upcoming maintenance?
  • Can I make a public page for the IT calendar?

Planning maintenance

Goto the IT calendar and workmode management tile:

Select the system (or select multiple systems in one go) and create the work mode for maintenance, and immediately switch to Expert Mode:

With expert mode you can simply set start and stop date and time.

You also should check the scope (the system will make a proposal) and extend if needed:

After saving, you can send notification via mail on the planned maintenance:

You can now see the planned maintenance in the calendar mode.

Overview of upcoming events

If you are in the calendar mode there can be a lot of systems. The overview screen for upcoming events will filter on the systems which have maintenance mode planned:

Actual maintenance

Actual maintenance can be seen in the System Down monitor:

You will only see maintenance that is currently running in the system down monitor, not the upcoming events.

Alert suppression

Alert suppression is in details set in table ACWMGLOBALDIR (maintenance via transaction SM30). See also this OSS note 3142800 – Health Monitoring Alerts generated during an active workmode in Focused Run for all the details.

Tips and tricks

Can I select multiple systems for planned maintenance in one go? Yes you can, this will save you a lot of time to input.

Can I make a recurring maintenance? Yes you can, in the scheduling screen mark the maintenance as recurring.

IT Calendar access for Non-SAP Focused Run Users

During the maintenance the alerts are suppressed. The same information can also be made generally available to serve as a IT calendar for all interested persons. This can be people for the IT department and the business. These users are mainly interested in planned downtime of the IT systems.

Use of the IT calendar for non-SAP focused run users

After the setup each person can now use this URL:

https://<host>:<port>/sap/bc/ui5_ui5/sap/itcal_external/index.html? FILTER_VARIANT=<public_variant-name>&sap-client=<client>

End result looks like this:

Setup of IT calendar access for non-SAP focused run users

The basic steps are described in OSS note 2926433 – IT Calendar access to Non-SAP Focused Run Users.

Create a system user with copy of role SAP_FRN_APP_ITC role and update it with the UI5 application “itcal_external”(IWSG and IWSV).

Also add these 4 not documented authorizations:

  1. Display for service availability management:

2. Display for work modes:

3. Add generic services:

4. Add filter bar rights:

Activate and save this role.

Activate these 2 SICF services:

  • /default_host/sap/bc/bsp/sap/itcal_external
  • /default_host/sap/bc/ui5_ui5/sap/itcal_external

On the Logon tab of these 2 services set the user ID and password of the newly created system user.

Do the same for the external system alias /sap/itcalnonsolman (also in SICF transaction): also here set the user ID and password for the newly created system user with the assignment of the newly created role.

After settings are done, execute testing. Most issues are coming from lack of auhtorizations.

OSS note for IT calendar function

Relevant OSS notes:

  • 2926433 – IT Calendar access to Non-SAP Focused Run Users
  • 3150979 – Number of systems in scope is limited, while using External IT Calendar

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans. Repost done with permission. >>

SAP Focused Run creation of custom metrics for system monitoring

In most cases a fine tuning of an existing SAP template is sufficient for your needs.

In some cases you want to have your own metric defined to monitor a special part of the SAP system. This own created metrics are also called custom metric.

SAP, when you read this blog, please feel free to copy any of the custom metrics below into the standard SAP set. This will help everybody.

Questions that will be answered in this blog are:

  • How do I create a custom monitoring metric?
  • Do I need to re-create the custom metric per monitoring template?
  • What are examples of custom metrics?

Creating custom metric

In this example we create a custom metric to make sure that the background user WF-BATCH is not locked by accident.

There is already a metric in the ABAP template that is called User Lock Status. This can be used as a basis for our custom metric.

Goto your template into change mode and on top left choose Create (you need to be in Expert mode first):

And select Metric. Now the screen opens for a new metric creation:

Fill out the details, and create a custom description:

Now go to the tab Data Collection:

Copy the data from your reference metric here. Don’t forget to fill out the Parameter Value. In this case WF-BATCH. Also make sure you have a reasonable Collection Interval timing. Not everything is need to be collected every 5 minutes.

Now go to the tab Threshold:

Configure your threshold setting.

Now press the Next button and assign the metric to the correct group:

Now press Finish to save the metric.

The new custom metric is now available in the monitoring template:

You see that this one has the Custom created marked. Later you can use the filter on Custom created column to quickly find it again.

Deploying custom metric to other templates

If you have to deploy the custom metric to other templates: so far this is a manual action. Per template you have to re-create the same custom metric. I have not found a nice way of re-using custom metrics yet.

List of other custom metrics

See below:

  • Checking for certificates that are about to be expired and already expired certificates (in stead of only one out of the box metric that contains both)
  • Checking web dispatcher URL
  • Detecting errors in table locking of TBTCO
  • Detecting if file system is mounted
  • Detecting long running DIA processes
  • Detecting no more free work processes
  • Detecting OS signals
  • Detecting PRIV modes
  • Detecting message server disconnects
  • Detecting missing hardware ID
  • Detecting resource exhaustion in ABAP system
  • Detecting specific ABAP short dumps
  • User lock status of DDIC and SAP*
SAP, when you read this, please feel free to add these to standard SAP metrics.

Other interesting ones:

For system log message read OSS note 3391086 – Grey Metrics for ABAP Syslog.

Detecting errors in table locking of TBTCO

From availability perspective, you want to detect as quickly as possible if you are suffering from locking errors of table TBTCO. TBTCO table is used for printing. If the locking error situation occurs the printing function will fail, and even worse, it can impact the complete SAP ABAP system.

You can create a custom monitoring metric to measure and act on this.

Create technical name Z_METRIC_ERR_LOCK_TBTCO:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message text *TBTCO*. This captures severe errors for TBTCO like the locking error.

Define the threshold for alerting:

And assign the metric to the ABAP Instance not available alert group:

Detecting if file system is mounted

Some filesystems are critical to a business, such as those used in interfaces. This custom metric group will alert if a filesystem is not mounted.

Create the Bash Script to Check the Filesystem Status

Firstly, we need to create a bash script that takes the filesystem as its input argument and then checks its status. Create the following script called /sbin/checkfilesystemmounted.sh (owner is root, permissions 755). You may put this script somewhere else if you prefer, but be sure to refer to the correct location later on in this post.

#!/bin/bash
findmnt $1 >/dev/null && echo \{type:integer, name:FileSystemMounted, value:1\} || echo \{type:integer, name:FileSystemMounted, value:0\}

The findmnt command returns the mount details if the filesystem is mounted. The filesystem is passed as a script argument in variable $1. If the filesystem is mounted, the script returns integer 1. If the filesystem is not mounted, the script returns integer 0. For example, to check your desired filesystem, execute it like this as root:

/sbin/checkfilesystemmounted.sh /the/filesystem/you/want/to/check

The output will be in JSON format. If the filesystem is mounted, the value will be 1, as follows:

{type:integer, name:FileSystemMounted, value:1}

The name:FileSystemMounted is the name of the value to be picked up by saphostctrl, as described next.

Create the Custom Operation for saphostctrl

To load these values into Focused Run, we create a custom operation for saphostctrl. Create the following custom operations conf file:

/usr/sap/hostctrl/exe/operations.d/checkfilesystemmounted.conf

This contains:

Command: /sbin/checkfilesystemmounted.sh $[FILESYSTEM]
Workdir: /home/sapadm
Description: Check if filesystem is mounted
ResultConverter: flat
Platform: Unix

To test the custom operation, execute the following command:

/usr/sap/hostctrl/exe/saphostctrl -function ExecuteOperation -name checkfilesystemmounted FILESYSTEM=/the/filesystem/you/want/to/check

The result should be as per the following example:

Webmethod returned successfully
Operation ID: 0A02C69098121EDDA68C041B50FE858D

----- Response data ----
description=Check if filesystem is mounted
{type:integer, name:FileSystemMounted, value:1}
exitcode=0
Create the Custom Alert in SAP Focused Run

In Focused Run, we create an alert in a Linux host monitoring template. For example, the alert name is “Interface Filesystem not Mounted”. The Alert should be in Category “Exceptions” and the Severity is up to you. In this case it is 9.

Create the Custom Metric Group in SAP Focused Run

Next, we create the custom Metric Group . A Metric Group allows variants to be created, and each variant corresponds to a filesystem you wish to monitor.

Overview Tab:
  • Name: “Interface Filesystem not Mounted”
  • Category: Exceptions
  • Class: Metric Group
  • Data Type: Integer
  • Technical Name: INTERFACE_FILESYSTEM_NOT_MOUNTED

Data Collection Tab:
  • Data Collector Type: Diagnostic Agent (push)
  • Data Collector Name: OS: ExecuteOperation
  • Collection Interval: 5 Minutes (depending on the criticality)
  • CUSTOM_OPERATION_NAME: checkfileystemmounted – This corresponds to the custom operation for saphostctrl created earlier
  • METRIC_NAME: FileSystemMounted – This corresponds to the name of the metric in the JSON output by the bash script
  • RETURNFORMAT: JSON – This is the output format of the bash script

Usage Tab:

Threshold Tab:

As the script returns a numeric value 0 if the filesystem is not mounted, then the threshold will alert if the value is 0.

Assignment Tab

Assign to the custom alert created earlier.

Add Variants

The variable passed to the saphostctrl operation is “FILESYSTEM”. We can add the rest of the filesystems as individual variants. The format for the operation parameters is as follows:

FILESYSTEM:/the/filesystem/you/want/to/check

For example:

You can enter as many filesystems as you like as separate variants.

Activate Alert

Go to the “Metrics, Events, Alerts Hierarchy” tab, and activate System Monitoring.

Testing the Metric

In a non-Production environment, try to unmount a filesystem, and at most 5 minutes later, there should be an alert produced.

Detecting no more free work processes

From availability perspective, you want to detect as quickly as possible if you don’t have any free work processes left.

The template to be adjusted is the technical instance SAP ABAP 7.10 and higher template. Don’t forget to tick it on for monitoring otherwise it is not active.

In expert mode create a custom metric.

Create technical name Z_NO_FREE:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message number QoG with text *NOWP*.. For more information on system log messages, read this blog.

Define the threshold for alerting:

And assign the metric to the ABAP Resource Shortage alert group:

Detecting long running DIA process

In some exceptional cases you can have a DIA process that runs for a long time without action and still occupies the resources.

You can create a custom monitoring metric to measure and act on this. The template to be adjusted is the technical system SAP ABAP 7.10 and higher template. Don’t forget to tick it on for monitoring otherwise it is not active.

In expert mode create a custom metric. Create technical name Z_METRIC_LONGRUN_DIA_WP_36HRS:

Now setup the definition for the data collection:

It is using the Push.

And set the usage:

Last but not least: you need to set the alerting threshold:

The alert is raised if a single DIA work process is running longer than 36 hours.

Save the custom metric and make sure the template reassignment is done to activate the custom metric for your systems.

Detecting OS signals

In some cases the OS system will give critical signals to the SAP system that are visible in the ABAP system log. An example is the signal 11. When this happens, the system is in trouble and you as admin need to check fast to see what is going on to stop the system from full collapse, crash or very poor performance.

You can create a custom monitoring metric to measure and act on this.

The template to be adjusted is the technical instance SAP ABAP 7.10 and higher template.

Don’t forget to tick it on for monitoring otherwise it is not active.

In expert mode create a custom metric.

Create technical name Z_METRIC_OS_SIGNAL_RECEIVED:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on MSG_ID QoE. This captures severe errors for OS signals.

Define the threshold for alerting:

And assign the metric to the system message alert group:

Detecting PRIV modes

The template to be adjusted is the technical system SAP ABAP 7.10 and higher template. Don’t forget to tick it on for monitoring otherwise it is not active.

Create technical name Z_METRIC_DIA_WP_PRIV:

Now setup the definition for the data collection:

This will collect the PRIV dialog processes in percentage.

Mark the custom metric as relevant for monitoring:

And set the assignment:

Last but not least: you need to set the alerting threshold:

Save the custom metric and make sure the template reassignment is done to activate the custom metric for your systems.

Detecting message server disconnects

From availability perspective, you want to detect as quickly as possible if you are suffering from message server disconnects.

Creation of the custom metric for message server disconnects

Create technical name Z_MESSAGE_SERVER_DISCONNECT:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message number Q0L, Q0M and Q0N. Any of those indicate message server errors. For more information on system log messages, read this blog.

Define the threshold for alerting:

And assign the metric to the ABAP Instance not available alert group:

Detecting missing hardware ID

From availability perspective, you want to detect as quickly as possible if you are suffering from missing hardware ID.

The template to be adjusted is the technical instance SAP ABAP 7.10 and higher template. Don’t forget to tick it on for monitoring otherwise it is not active.

In expert mode create a custom metric.

Create technical name Z_METRIC_MSG_SRV_HW_ID_MISSING:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message number Q16. This indicates missing hardware ID. For more information on system log messages, read this blog.

Define the threshold for alerting:

And assign the metric to the ABAP Instance not available alert group, create a custom alert group:

detecting resource exhaustion in ABAP system

From availability perspective, you want to detect as quickly as possible if you are suffering from resource exhaustion.

You can create a custom monitoring metric to measure and act on this.

Creation of the custom metric for resource exhaustion

Create technical name Z_EXHAUST:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message number Q40. This is the message for resources exhausted. For more information on system log messages, read this blog.

Set the usage to monitoring:

Define the threshold for alerting:

And assign the metric to the ABAP Instance not available alert group:

User lock status of DDIC and SAP*

From security perspective, you want to validate that 2 important users are locked in the main system clients: SAP* and DDIC. For more background you can read this blog.

Create technical name ZUSER_LOCK_STATUS:

In the data collection:

Data to enter: RFC diagnostics agent (push). User Lock status Data collector. Enter as parameters the user ID (DDIC) and the COLLECTOR_CONTEXT_ID as TECHNICAL_SYSTEM.

Set the threshold as a text threshold:

Set the red rating in case the string contains the word ‘not locked’ and set to green in case it contains the word ‘locked’.

Now assign it to Alert group for locked users:

Save the metric.

Repeat the same for SAP*.

Checking web dispatcher URL

When you are using a web dispatcher, you want to check that the main URL is available. You can achieve this via URL monitoring in health monitoring (see blog).

In some cases you want to integrate this vital start URL into system monitoring, since that is your main central tool.

You can create a custom monitoring metric to measure and act on this.

In the use case below we will setup URL monitoring for web dispatcher for SAP Netweaver Gateway serving FIORI pages.

Creation of the custom metric for web dispatcher URL monitoring

The template to be adjusted is the technical system SAP Web Dispatcher template. Don’t forget to tick it on for monitoring otherwise it is not active.

In expert mode create a custom metric. Create technical name Z_WEBDISPATCHER_URL_AVAILABILITY:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select SRSM Ping Http Unsp. Select the HTTPS protocol and setup the URL to monitor: /sap/bc/ui2/flp?sap-client=xxx&sap-language=EN. This is the main FIORI start URL. The port number is taken from the LMDB settings of the web dispatcher: $SAP_WebDispatcherIPServicePort->SAP_IPServicePort.PortNumber$.

Define the threshold for alerting:

We take here three measurements. If we don’t do this, a single glitch in the network will cause an alert to be triggered.

And assign the metric to the system message alert group:

Monitoring specific ABAP dumps

In System Monitoring of ABAP systems in Focused Run, with the standard monitoring template, we can monitor the frequency of ABAP dumps.

For monitoring if there are dumps of a specific type we have to create a custom metric. In this blog we explain how.

For creating a custom metric in the monitoring template, open the Template Maintenance App in the Advanced System Management area of Focused Run Launch-pad.

Open your custom ABAP instance level template in edit mode.

Switch to expert mode for template maintenance.

Click on Create –> Metric to create a custom metric.

Provide a Name, a Technical Name and metric type in the Overview tab as shown below.

In the Data collection tab enter details as shown below. Here for the parameter Error_ID you can provide the ID of the specific ABAP dump that you want to monitor.

Navigate to Threshold tab to specify the Threshold type and Threshold. For instance in this example we have specify the threshold to rate the metric as reed if there is 1 or more of the specific dump.

Now you can click on Next and then Finish button to save the metric.

Next you need to create a custom alert to generate a specific alert for this specific dump.

In the expert mode maintenance go to the Create button and then from the drop down select Alert.

Enter the details as shown below and click on Next button.

Then in the Assignments tab you will see the custom metric you just created. Select the check box for this metric and click on Finish to save the alert.

Now your custom metric is ready together with alerting active.

To activate this template update on all managed systems navigate to Managed Objects tab for the template and click on Apply and Activate button.

Now you can see custom metric under instance level in system monitoring.

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans, Tony Swietochowski (DXC) and Manas Tripathy (Simac). Repost done with permission. >>

SAP Focused Run fine tuning of monitoring templates

This blog explains about the fine tuning of monitoring templates.

Questions that will be answered are:

  • How to update the SAP content for templates?
  • What is a good rule of thumb for the amount of templates to create and maintain?
  • Should I transport the templates or maintain them locally?
  • How to create your own template?
  • How to fine tune a single metric?
  • How to change the alerting settings of a metric?
  • How to assign the template to a system?
  • How to update the template of a system?
  • Which tools can I use to perform fine tuning of alert thresholds?
  • Can I perform a forecast based on the data?
  • Can I perform a sensitivity analysis?
  • Which installation activities are required to enable the forecast and sensitivity functions?
  • How do I define rule bassed template assignments?

SAP content updates

As a starting point you use the SAP pre-delivered content. Also the SAP content gets updated. OSS note 3275006 – Manual content update for FRUN-CONT 400 in SAP Focused Run. is keeping track of the updates. It also explains where to download the content files.

Use program RCSU_MANUAL_UPLOAD to upload the downloaded content. Then use the FIORI tile Content management to activate the new content:

And update the content or see it is already up-to-date:

Before you start fine tuning your own templates, make sure the standard SAP content is up-to-date.

Amount of templates to fine tune

In principle it is up to you to generate as much templates as needed. Initially it seems a good idea to have many different templates. The setback is that fine tuning a specific metric that is valid for all templates, you need to repeat this action. Also when you have fine tuned a template, you need to update the attached systems.

A good starting point for fine tuning is to have 2 templates to start with:

  1. Template for productive system
  2. Template for non-productive system

The template for productive system can have more metrics activated with sharper thresholds for generating alerts.

The main goal for a non-productive template can be focused on system availability only.

For productive system you want to manage all aspects of a system including performance and all content exceptions.

Local maintenance or transport

The template maintenance can be done on a productive Focused Run system directly. Or you can choose to maintain the templates on a trial/test Focused Run system, test it there, and then transport it to the productive Focused Run system. The transport is the best approach that gives the most control.

Who should fine tune a template?

This is an organisational question. If you let everybody maintain the templates and metric content of the templates, you will quickly loose control. Best to limit the amount of people to maintain the template settings. Be careful when handing out template control to a service provider. They tend to change the thresholds to very high levels, so they get less alerts. In stead of solving the alerts….

Creation of own template

Open the template maintenance Fiori tile:

Select the template you want to fine tune. In this example we will fine tune the Technical System template for ABAP 7.10 and higher:

Press the Edit button and then the button Create Custom Template:

Give the template a good name. The most descriptive text must be at the beginning.

Fine tuning the template

Case 1: include or exclude in monitoring

Goto the metrics tab:

In the system monitoring you can switch on or off metrics. Press save after each change to save your setting changes.

Case 2: fine tune data selection

In the standard SAP delivery there is an alert for Number of long running Dialog Work Processes. Goto the Expert mode (button top rights), then select the tab data collection:

Go into edit mode via the Change Settings button, and you can update the field value in parameter value for WP_MIN_RUNTIME to your needs:

This is just an example. You can fine tune a lot of metrics in this way.

Case 3: fine tune threshold and alert settings

If you want to change the thresholds, first click on the expert mode button on the top right corner. Then press the Change setting button to edit the Threshold tab settings:

In this example we changed the type from Numeric (green/red) to Numeric (green/yellow/red) and we changed the values. The modified column indicates that we have changed a metric and

that the definition is different from the standard SAP one.

On the Alerts tab you can make changes to the alert settings:

You can change the following:

  • If an alert is to be generated or not (Active means, alert is generated)
  • Severity of the alert
  • If an alert will be automatically confirmed when the system detects that the issue is solved
  • If an automatic notification will be send or not

In the last tab Managed Objects you can see there are no systems assigned yet to the newly created template:

How to fine tune an alert in practice

In our example we will look at the Dialog Response Time metric. The current threshold for red alert is set to 5000 ms (5 seconds). The alert is triggered too often. But the question to answer now: what is a good threshold to set based on the historical data?

First click the Open metric in new window icon to enlarge the screen:

The enlarged screen now opens:

As you can see 2 times the red threshold was hit. We want to fine tune now. First select the calendar icon and select last 7 days to get full week overview:

You can use the forecast button to let the system create a forecast:

The forecast will now show mean, mean low and mean high forecast:

In this specific use case the prediction is that the maximum is 3300 ms (3.3 seconds).

Now open the statistics button to see the statistics and the recommended threshold tool button:

By changing the Sensitivity slider, the system will calculate different proposal for the alert threshold. In our case when we move sensitivity to 4 the new recommended threshold value is recalculated:

In this case it is 7669 ms.

So we now have collected following facts:

  • Current threshold of 5 seconds is reached too often
  • Average forecast based on history has a mean value of 3.3 seconds
  • Performing the sensitivity analysis the threshold recommendation is about 7.7 seconds

Based on this data the red threshold is best to increase from 5 to 8 seconds to get a good alert function. It will not reach too soon, hence limiting false alerts, but it will still alert in time in case poor performance happens.

Enabling forecast and sensitivity analysis

The forecast and sensitivity analysis function use the Application Function Library (AFL) and SAP HANA Automated Predictive Library (APL). These must be installed separately. The installation details and post steps for granting permissions are described in the Focused Run master guide in the section “Predictive Analytics Setup – Metric Forecasting”.

After the installation you must activate and assign PFCG role SAP_FRN_APP_PAS_DISP to be able to see the buttons.

Assigning custom template to a system

To assign a new custom template to a system, goto the Individual maintenance Fiori tile:

Select the system and press the button Change assignment and assign the wanted new template:

Now press the button Reconfigure to effectuate the template change.

Automation of template assignments can be configured as well by using rules.

Template updates

If you have systems assigned to a template, and you have executed template changes, goto the tab Managed objects in the template maintenance screen:

Select the systems and press the Apply and Activate button. The system will apply the updated template now.

If you use transport mechanism for template updates: after transport import, you need to go to the updated templates, and still to the update assignment. This is not automatically done after the transport.

Compare templates

In the main screen of template maintenance you can select the button Compare to start the template comparison app. Select the templates to compare:

You now see the delta’s between the templates:

Creating custom metrics

Creation of custom metrics is possible to have metrics for your specific needs.

The setback of custom metric is that it needs to be created each time for each template. This is another reason to keep the amount of custom templates as low as possible.

Read all about custom metrics in this blog.

Rule Based Template Assignment

When we perform Simple System Integration (SSI) on a managed system , it automatically activates the SAP default monitoring template on the managed system. However, in most of the SAP Focused Run (FRUN) implementation scenarios, we create customer defined monitoring templates (Custom Templates), which we then manually assign/activate on the managed system.

Rule Based Template Assignment is a feature in FRUN by which we can define based on managed system category which custom monitoring template to be assigned and activated directly when we perform SSI on the managed system.

Defining Rule Based Template Assignment

For Rule Based Template Assignment navigate to the Fiori tile Individual Maintenance in the Advanced System Management section of Fiori launch-pad.

In the Individual Maintenance App navigate to the Rule Maintenance by clicking on the button as shown below.

In the Rule Rule Based Assignment Screen, on the left hand side panel, select the specific Managed Object type for which you want to define the Rule Based Template Assignment.

In this blog we take the example of defining a Rule Based Template Assignment for managed system of type SAP ABAP BASIS 7.10 and higher and specify the custom template for System Level monitoring template. So we select Technical Systems upon which the right side panel now gives a list of all product types. In the right side panel we scroll down and select SAP ABAP BASIS 7.10 and higher.

Now we need to define the Rule based on which the Custom Defined Template to be defined. In this blog we take the example that we have defined 2 custom templates one for Production Systems and one for Non Production Systems. So we will need to define rule to assign template based on filters System Type ABAP and IT Admin role defined in LMDB. For more information on this function read this blog.

In the subsequent screen select Maintain Rules.

In the Maintain Rule screen we select the following filters.

Name your Rule and Save.

Similarly you can create Rule ABAP Non production, just ensure to select the following IT Admin Roles.

Now back in the main screen select the Rule you created from the drop down.

And for Template select the custom template you want to select for the assignment.

Add the assignment.

Now click in “Continue with Next Step” button until you come to the Reconfiguration tab and then close. This will allow you to save your Rule Assignment.

Once you have assigned the ABAP Production and ABAP Non production rules in the main screen you will see the following assignments listed.

After the assignments done, the next time SSI performed on any ABAP system will take up the custom monitoring template as defined in these rules.

In Individual Maintenance system list you can also see whether current assignment is SAP default or Rule Based Template Assignment.

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans and Manas Tripathy (Simac). Repost done with permission. >>

SAP Focused Run system monitoring overview

This blog will give you and overview of the functional capabilities of the System Monitoring in SAP Focused Run.

Questions that will be answered in this blog are:

  • What are the main functions of System Monitoring?
  • How to zoom in on systems and specific metrics?
  • How to optimize the scope selection?
  • How to use the tabular view?
  • How to check a specific metric across multiple systems?
  • How can I quickly get an overview of all my systems that are down?
  • How to set IT admin role of systems in LMDB?
  • How to store specific metric data longer?
  • How to use the aggregation framework?

System monitoring top down approach

From the Advanced System monitoring group in Fiori launchpad, select the System Monitoring tile:

Now select the systems in the Scope Selection block, for which you want to see the monitoring data:

Select Go when you finished your filtering. You now reach the overview screen:

If you want to zoom in click on one of the numbers, or select the Systems button from the left hand toolbar:

The traffic lights indicate where the issue or issues are: availability, performance, configuration or exceptions. If you want to go directly to an alert click on the alert number. Alerts are explained in full in this blog.

Click on a single system in the left column to open the system monitoring view for a single system:

On the left hand side, you can see the application (in this case ABAP) on top. You can also see the database (HANA) and application server, CI and their hosts. On the right hand side in a tree structure you can see the diverse checkpoints and issues in the system. The checkpoints are called metrics and they are clubbed together into logical blocks (like system exceptions, performance, availability). In this case there is a system exception due to too many short dumps today.

You can open the graph for this metric to see the details in time:

By clicking on the start and to date, you can select the date/time range or use the Select Time Frame button for a predefined time range:

Optimizing scope selection

In the scope selection of systems, you can create a few variants to speed up your work.

In this example we will setup a variant to quickly select all productive systems. In the scope selection block select the IT Admin Role for Productive System:

Now select the down arrow next to Standard in the top left corner and select Save As:

You can choose to set this variant as default. Setting it as public will make the variant available for all users. Selecting the Apply Automatically tickbox will apply this specific variant immediately. This might be preferable, or annoying. Just try it.

Upon pressing Save you will get a request for transport popup or save it as local request.

You can also create a similar view for non-production systems.

In the end you can always press the Manage button to change the variants and texts:

Now you can easily switch between scopes for production and non-production:

How to set IT admin role of systems in LMDB

This chapter will show how you can set the IT Admin role of a system in LMDB. Goals is that you can use it easily as described above in the scope selection.

Go to the LMDB Object Maintenance Fiori tile:

Search for your system:

Select the system and press Display to open the detail screen:

Press Edit to change. Now change the IT Admin Role and press Save.

Using the tabular view

In stead of using the hierarchy view, you can also switch to the Tabular View:

In this view you can for example sort the items on a column like the traffic light:

Or you can apply a text filter to search for a specific metric:

Checking a metric across multiple systems

If you have an issue in one system, you might want to quickly validate if you have similar issue in different systems, or you simple want to compare with different systems. From the monitoring of a system select the metric.

For this example we selected Short Dumps:

Select the i button to get the explanation text:

This gives the exact name:

Now goto the metric tool:

If you don’t see the correct metric, use the metric selection filter on the top right of the screen:

Press Apply, and you get the overview of this specific metric across all systems in your selected scope:

Storing metric data longer

Focused Run stores the monitoring data 28 days. If you need the data for specific metrics and systems longer, you can make use of the aggregation framework.

Start the Fiori application for Advanced Configuration of System Monitoring:

On the left hand side choose the option Aggregation Framework:

Choose the button Create Variant to create a new variant:

Fill out the name and basic description and press the Continue with next step button:

The next screen is bit more complex:

In sequence: first search for the extended system ID and press go in the top left section. In the bottom left section, select the system you want. In the top right section now select Add filter from the left button. And press the Add selected objects for aggregation button on the bottom right part. Now press the Continue with next step button:

Select the metrics on the left hand side and add the filters on the right hand side. When done press the Continue with next step button:

Using the aggregation framework

For using the aggregation framework there are no special requirements. Whenever you use an aggregated metric in system monitoring, you can simply use the details with a long period.

Settings for the aggregation framework

In the aggregation framework configuration screen, you can click on the configuration wheel top right to set the retention period for Short/Medium/Long:

System down monitor

A special function is System Monitoring is the System Down Monitor. This overview directly gives an overview of the systems that are considered down by SAP Focused Run and the systems which are set to having maintenance.

In the system monitoring screen select the System Down monitoring icon on the left icon bar (here indicated with the arrow):

You can see systems that are down and which ones that are having planned maintenance. If you have set up the SLA management, it will also show that aspect.

If you want to zoom in on the issues, press the i icon right of the system. Then select Links to go to the respective tool for further investigation:

For systems down the best tools are usually the System Analysis and the Alert Event management.

Changing settings

You can change the layout settings with the glasses icon:

You can show/hide the SLA and charts section as per your need.

Definition of down

The definition of down is in Focused Run: any red alert in the availability metrics. This can be:

  • Complete system down
  • One of the application servers is down
  • A core function is down (for example ABAP stack is up and running, but the Https port is not available)
  • Important subfunctions are not working (for example in the SLT system 1 or more source systems can not be reached)

Relevant OSS notes

Summary

The overview above gives the top – down approach in full: from the total landscape, to single system, to group of metrics to single metric.

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans. Repost done with permission. >>