SAP Focused Run workmode management

There will be planned maintenance on SAP systems, databases, operating systems and the below infrastructure. During this planned maintenance, you don’t want any alerts and mails from SAP Focused Run, since the application is not available for a planned time.

Using the work mode management function you plan these maintenance events.

Questions that will be answered in this blog are:

  • How can I set up planned maintenance for systems monitored by SAP Focused Run to temporarily suppress the alerts during maintenance?
  • How can I see the current maintenance and planned upcoming maintenance?
  • Can I make a public page for the IT calendar?

Planning maintenance

Goto the IT calendar and workmode management tile:

Select the system (or select multiple systems in one go) and create the work mode for maintenance, and immediately switch to Expert Mode:

With expert mode you can simply set start and stop date and time.

You also should check the scope (the system will make a proposal) and extend if needed:

After saving, you can send notification via mail on the planned maintenance:

You can now see the planned maintenance in the calendar mode.

Overview of upcoming events

If you are in the calendar mode there can be a lot of systems. The overview screen for upcoming events will filter on the systems which have maintenance mode planned:

Actual maintenance

Actual maintenance can be seen in the System Down monitor:

You will only see maintenance that is currently running in the system down monitor, not the upcoming events.

Alert suppression

Alert suppression is in details set in table ACWMGLOBALDIR (maintenance via transaction SM30). See also this OSS note 3142800 – Health Monitoring Alerts generated during an active workmode in Focused Run for all the details.

Tips and tricks

Can I select multiple systems for planned maintenance in one go? Yes you can, this will save you a lot of time to input.

Can I make a recurring maintenance? Yes you can, in the scheduling screen mark the maintenance as recurring.

IT Calendar access for Non-SAP Focused Run Users

During the maintenance the alerts are suppressed. The same information can also be made generally available to serve as a IT calendar for all interested persons. This can be people for the IT department and the business. These users are mainly interested in planned downtime of the IT systems.

Use of the IT calendar for non-SAP focused run users

After the setup each person can now use this URL:

https://<host>:<port>/sap/bc/ui5_ui5/sap/itcal_external/index.html? FILTER_VARIANT=<public_variant-name>&sap-client=<client>

End result looks like this:

Setup of IT calendar access for non-SAP focused run users

The basic steps are described in OSS note 2926433 – IT Calendar access to Non-SAP Focused Run Users.

Create a system user with copy of role SAP_FRN_APP_ITC role and update it with the UI5 application “itcal_external”(IWSG and IWSV).

Also add these 4 not documented authorizations:

  1. Display for service availability management:

2. Display for work modes:

3. Add generic services:

4. Add filter bar rights:

Activate and save this role.

Activate these 2 SICF services:

  • /default_host/sap/bc/bsp/sap/itcal_external
  • /default_host/sap/bc/ui5_ui5/sap/itcal_external

On the Logon tab of these 2 services set the user ID and password of the newly created system user.

Do the same for the external system alias /sap/itcalnonsolman (also in SICF transaction): also here set the user ID and password for the newly created system user with the assignment of the newly created role.

After settings are done, execute testing. Most issues are coming from lack of auhtorizations.

OSS note for IT calendar function

Relevant OSS notes:

  • 2926433 – IT Calendar access to Non-SAP Focused Run Users
  • 3150979 – Number of systems in scope is limited, while using External IT Calendar

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans. Repost done with permission. >>

SAP Focused Run creation of custom metrics for system monitoring

In most cases a fine tuning of an existing SAP template is sufficient for your needs.

In some cases you want to have your own metric defined to monitor a special part of the SAP system. This own created metrics are also called custom metric.

SAP, when you read this blog, please feel free to copy any of the custom metrics below into the standard SAP set. This will help everybody.

Questions that will be answered in this blog are:

  • How do I create a custom monitoring metric?
  • Do I need to re-create the custom metric per monitoring template?
  • What are examples of custom metrics?

Examples of implementation of custom metrics that you can find in this blog below are:

  • Checking if specific background user ID is locked
  • Detecting PRIV modes

Creating custom metric

In this example we create a custom metric to make sure that the background user WF-BATCH is not locked by accident.

There is already a metric in the ABAP template that is called User Lock Status. This can be used as a basis for our custom metric.

Goto your template into change mode and on top left choose Create (you need to be in Expert mode first):

And select Metric. Now the screen opens for a new metric creation:

Fill out the details, and create a custom description:

Now go to the tab Data Collection:

Copy the data from your reference metric here. Don’t forget to fill out the Parameter Value. In this case WF-BATCH. Also make sure you have a reasonable Collection Interval timing. Not everything is need to be collected every 5 minutes.

Now go to the tab Threshold:

Configure your threshold setting.

Now press the Next button and assign the metric to the correct group:

Now press Finish to save the metric.

The new custom metric is now available in the monitoring template:

You see that this one has the Custom created marked. Later you can use the filter on Custom created column to quickly find it again.

Deploying custom metric to other templates

If you have to deploy the custom metric to other templates: so far this is a manual action. Per template you have to re-create the same custom metric. I have not found a nice way of re-using custom metrics yet.

List of other custom metrics

See below:

  • Detecting errors in table locking of TBTCO
  • Detecting PRIV modes
  • Detecting message server disconnects
  • Detecting resource exhaustion in ABAP system
  • User lock status of DDIC and SAP*

Detecting errors in table locking of TBTCO

From availability perspective, you want to detect as quickly as possible if you are suffering from locking errors of table TBTCO. TBTCO table is used for printing. If the locking error situation occurs the printing function will fail, and even worse, it can impact the complete SAP ABAP system.

You can create a custom monitoring metric to measure and act on this.

Create technical name Z_METRIC_ERR_LOCK_TBTCO:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message text *TBTCO*. This captures severe errors for TBTCO like the locking error.

Define the threshold for alerting:

And assign the metric to the ABAP Instance not available alert group:

Detecting PRIV modes

The template to be adjusted is the technical system SAP ABAP 7.10 and higher template. Don’t forget to tick it on for monitoring otherwise it is not active.

Create technical name Z_METRIC_DIA_WP_PRIV:

Now setup the definition for the data collection:

This will collect the PRIV dialog processes in percentage.

Mark the custom metric as relevant for monitoring:

And set the assignment:

Last but not least: you need to set the alerting threshold:

Save the custom metric and make sure the template reassignment is done to activate the custom metric for your systems.

Detecting message server disconnects

From availability perspective, you want to detect as quickly as possible if you are suffering from message server disconnects.

Creation of the custom metric for message server disconnects

Create technical name Z_MESSAGE_SERVER_DISCONNECT:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message number Q0L, Q0M and Q0N. Any of those indicate message server errors. For more information on system log messages, read this blog.

Define the threshold for alerting:

And assign the metric to the ABAP Instance not available alert group:

detecting resource exhaustion in ABAP system

From availability perspective, you want to detect as quickly as possible if you are suffering from resource exhaustion.

You can create a custom monitoring metric to measure and act on this.

Creation of the custom metric for resource exhaustion

Create technical name Z_EXHAUST:

In the data collection:

Data to enter: RFC on diagnostics agent (push). Select ABAP System Log Stats. Filter on message number Q40. This is the message for resources exhausted. For more information on system log messages, read this blog.

Set the usage to monitoring:

Define the threshold for alerting:

And assign the metric to the ABAP Instance not available alert group:

User lock status of DDIC and SAP*

From security perspective, you want to validate that 2 important users are locked in the main system clients: SAP* and DDIC. For more background you can read this blog.

Create technical name ZUSER_LOCK_STATUS:

In the data collection:

Data to enter: RFC diagnostics agent (push). User Lock status Data collector. Enter as parameters the user ID (DDIC) and the COLLECTOR_CONTEXT_ID as TECHNICAL_SYSTEM.

Set the threshold as a text threshold:

Set the red rating in case the string contains the word ‘not locked’ and set to green in case it contains the word ‘locked’.

Now assign it to Alert group for locked users:

Save the metric.

Repeat the same for SAP*.

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans. Repost done with permission. >>

SAP Focused Run fine tuning of monitoring templates

This blog explains about the fine tuning of monitoring templates.

Questions that will be answered are:

  • How to update the SAP content for templates?
  • What is a good rule of thumb for the amount of templates to create and maintain?
  • Should I transport the templates or maintain them locally?
  • How to create your own template?
  • How to fine tune a single metric?
  • How to change the alerting settings of a metric?
  • How to assign the template to a system?
  • How to update the template of a system?
  • Which tools can I use to perform fine tuning of alert thresholds?
  • Can I perform a forecast based on the data?
  • Can I perform a sensitivity analysis?
  • Which installation activities are required to enable the forecast and sensitivity functions?
  • How do I define rule bassed template assignments?

SAP content updates

As a starting point you use the SAP pre-delivered content. Also the SAP content gets updated. OSS note 3275006 – Manual content update for FRUN-CONT 400 in SAP Focused Run. is keeping track of the updates. It also explains where to download the content files.

Use program RCSU_MANUAL_UPLOAD to upload the downloaded content. Then use the FIORI tile Content management to activate the new content:

And update the content or see it is already up-to-date:

Before you start fine tuning your own templates, make sure the standard SAP content is up-to-date.

Amount of templates to fine tune

In principle it is up to you to generate as much templates as needed. Initially it seems a good idea to have many different templates. The setback is that fine tuning a specific metric that is valid for all templates, you need to repeat this action. Also when you have fine tuned a template, you need to update the attached systems.

A good starting point for fine tuning is to have 2 templates to start with:

  1. Template for productive system
  2. Template for non-productive system

The template for productive system can have more metrics activated with sharper thresholds for generating alerts.

The main goal for a non-productive template can be focused on system availability only.

For productive system you want to manage all aspects of a system including performance and all content exceptions.

Local maintenance or transport

The template maintenance can be done on a productive Focused Run system directly. Or you can choose to maintain the templates on a trial/test Focused Run system, test it there, and then transport it to the productive Focused Run system. The transport is the best approach that gives the most control.

Who should fine tune a template?

This is an organisational question. If you let everybody maintain the templates and metric content of the templates, you will quickly loose control. Best to limit the amount of people to maintain the template settings. Be careful when handing out template control to a service provider. They tend to change the thresholds to very high levels, so they get less alerts. In stead of solving the alerts….

Creation of own template

Open the template maintenance Fiori tile:

Select the template you want to fine tune. In this example we will fine tune the Technical System template for ABAP 7.10 and higher:

Press the Edit button and then the button Create Custom Template:

Give the template a good name. The most descriptive text must be at the beginning.

Fine tuning the template

Case 1: include or exclude in monitoring

Goto the metrics tab:

In the system monitoring you can switch on or off metrics. Press save after each change to save your setting changes.

Case 2: fine tune data selection

In the standard SAP delivery there is an alert for Number of long running Dialog Work Processes. Goto the Expert mode (button top rights), then select the tab data collection:

Go into edit mode via the Change Settings button, and you can update the field value in parameter value for WP_MIN_RUNTIME to your needs:

This is just an example. You can fine tune a lot of metrics in this way.

Case 3: fine tune threshold and alert settings

If you want to change the thresholds, first click on the expert mode button on the top right corner. Then press the Change setting button to edit the Threshold tab settings:

In this example we changed the type from Numeric (green/red) to Numeric (green/yellow/red) and we changed the values. The modified column indicates that we have changed a metric and

that the definition is different from the standard SAP one.

On the Alerts tab you can make changes to the alert settings:

You can change the following:

  • If an alert is to be generated or not (Active means, alert is generated)
  • Severity of the alert
  • If an alert will be automatically confirmed when the system detects that the issue is solved
  • If an automatic notification will be send or not

In the last tab Managed Objects you can see there are no systems assigned yet to the newly created template:

How to fine tune an alert in practice

In our example we will look at the Dialog Response Time metric. The current threshold for red alert is set to 5000 ms (5 seconds). The alert is triggered too often. But the question to answer now: what is a good threshold to set based on the historical data?

First click the Open metric in new window icon to enlarge the screen:

The enlarged screen now opens:

As you can see 2 times the red threshold was hit. We want to fine tune now. First select the calendar icon and select last 7 days to get full week overview:

You can use the forecast button to let the system create a forecast:

The forecast will now show mean, mean low and mean high forecast:

In this specific use case the prediction is that the maximum is 3300 ms (3.3 seconds).

Now open the statistics button to see the statistics and the recommended threshold tool button:

By changing the Sensitivity slider, the system will calculate different proposal for the alert threshold. In our case when we move sensitivity to 4 the new recommended threshold value is recalculated:

In this case it is 7669 ms.

So we now have collected following facts:

  • Current threshold of 5 seconds is reached too often
  • Average forecast based on history has a mean value of 3.3 seconds
  • Performing the sensitivity analysis the threshold recommendation is about 7.7 seconds

Based on this data the red threshold is best to increase from 5 to 8 seconds to get a good alert function. It will not reach too soon, hence limiting false alerts, but it will still alert in time in case poor performance happens.

Enabling forecast and sensitivity analysis

The forecast and sensitivity analysis function use the Application Function Library (AFL) and SAP HANA Automated Predictive Library (APL). These must be installed separately. The installation details and post steps for granting permissions are described in the Focused Run master guide in the section “Predictive Analytics Setup – Metric Forecasting”.

After the installation you must activate and assign PFCG role SAP_FRN_APP_PAS_DISP to be able to see the buttons.

Assigning custom template to a system

To assign a new custom template to a system, goto the Individual maintenance Fiori tile:

Select the system and press the button Change assignment and assign the wanted new template:

Now press the button Reconfigure to effectuate the template change.

Automation of template assignments can be configured as well by using rules.

Template updates

If you have systems assigned to a template, and you have executed template changes, goto the tab Managed objects in the template maintenance screen:

Select the systems and press the Apply and Activate button. The system will apply the updated template now.

If you use transport mechanism for template updates: after transport import, you need to go to the updated templates, and still to the update assignment. This is not automatically done after the transport.

Compare templates

In the main screen of template maintenance you can select the button Compare to start the template comparison app. Select the templates to compare:

You now see the delta’s between the templates:

Creating custom metrics

Creation of custom metrics is possible to have metrics for your specific needs.

The setback of custom metric is that it needs to be created each time for each template. This is another reason to keep the amount of custom templates as low as possible.

Read all about custom metrics in this blog.

Rule Based Template Assignment

When we perform Simple System Integration (SSI) on a managed system , it automatically activates the SAP default monitoring template on the managed system. However, in most of the SAP Focused Run (FRUN) implementation scenarios, we create customer defined monitoring templates (Custom Templates), which we then manually assign/activate on the managed system.

Rule Based Template Assignment is a feature in FRUN by which we can define based on managed system category which custom monitoring template to be assigned and activated directly when we perform SSI on the managed system.

Defining Rule Based Template Assignment

For Rule Based Template Assignment navigate to the Fiori tile Individual Maintenance in the Advanced System Management section of Fiori launch-pad.

In the Individual Maintenance App navigate to the Rule Maintenance by clicking on the button as shown below.

In the Rule Rule Based Assignment Screen, on the left hand side panel, select the specific Managed Object type for which you want to define the Rule Based Template Assignment.

In this blog we take the example of defining a Rule Based Template Assignment for managed system of type SAP ABAP BASIS 7.10 and higher and specify the custom template for System Level monitoring template. So we select Technical Systems upon which the right side panel now gives a list of all product types. In the right side panel we scroll down and select SAP ABAP BASIS 7.10 and higher.

Now we need to define the Rule based on which the Custom Defined Template to be defined. In this blog we take the example that we have defined 2 custom templates one for Production Systems and one for Non Production Systems. So we will need to define rule to assign template based on filters System Type ABAP and IT Admin role defined in LMDB. For more information on this function read this blog.

In the subsequent screen select Maintain Rules.

In the Maintain Rule screen we select the following filters.

Name your Rule and Save.

Similarly you can create Rule ABAP Non production, just ensure to select the following IT Admin Roles.

Now back in the main screen select the Rule you created from the drop down.

And for Template select the custom template you want to select for the assignment.

Add the assignment.

Now click in Continue with Next Step button till you come to the Reconfiguration tab and then close. This will allow you to save your Rule Assignment.

Once you have assigned the ABAP Production and ABAP Non production rules in the main screen you will see the following assignments listed.

After the assignments done, the next time SSI performed on any ABAP system will take up the custom monitoring template as defined in these rules.

In Individual Maintenance system list you can also see whether current assignment is SAP default or Rule Based Template Assignment.

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans and Manas Tripathy (Simac). Repost done with permission. >>

SAP Focused Run system monitoring overview

This blog will give you and overview of the functional capabilities of the System Monitoring in SAP Focused Run.

Questions that will be answered in this blog are:

  • What are the main functions of System Monitoring?
  • How to zoom in on systems and specific metrics?
  • How to optimize the scope selection?
  • How to use the tabular view?
  • How to check a specific metric across multiple systems?
  • How can I quickly get an overview of all my systems that are down?

System monitoring top down approach

From the Advanced System monitoring group in Fiori launchpad, select the System Monitoring tile:

Now select the systems in the Scope Selection block, for which you want to see the monitoring data:

Select Go when you finished your filtering. You now reach the overview screen:

If you want to zoom in click on one of the numbers, or select the Systems button from the left hand toolbar:

The traffic lights indicate where the issue or issues are: availability, performance, configuration or exceptions. If you want to go directly to an alert click on the alert number. Alerts are explained in full in this blog.

Click on a single system in the left column to open the system monitoring view for a single system:

On the left hand side, you can see the application (in this case ABAP) on top. You can also see the database (HANA) and application server, CI and their hosts. On the right hand side in a tree structure you can see the diverse checkpoints and issues in the system. The checkpoints are called metrics and they are clubbed together into logical blocks (like system exceptions, performance, availability). In this case there is a system exception due to too many short dumps today.

You can open the graph for this metric to see the details in time:

By clicking on the start and to date, you can select the date/time range or use the Select Time Frame button for a predefined time range:

Optimizing scope selection

In the scope selection of systems, you can create a few variants to speed up your work.

In this example we will setup a variant to quickly select all productive systems. In the scope selection block select the IT Admin Role for Productive System:

Now select the down arrow next to Standard in the top left corner and select Save As:

You can choose to set this variant as default. Setting it as public will make the variant available for all users. Selecting the Apply Automatically tickbox will apply this specific variant immediately. This might be preferable, or annoying. Just try it.

Upon pressing Save you will get a request for transport popup or save it as local request.

You can also create a similar view for non-production systems.

In the end you can always press the Manage button to change the variants and texts:

Now you can easily switch between scopes for production and non-production:

How to set IT admin role of systems in LMDB

This chapter will show how you can set the IT Admin role of a system in LMDB. Goals is that you can use it easily as described above in the scope selection.

Go to the LMDB Object Maintenance Fiori tile:

Search for your system:

Select the system and press Display to open the detail screen:

Press Edit to change. Now change the IT Admin Role and press Save.

Using the tabular view

In stead of using the hierarchy view, you can also switch to the Tabular View:

In this view you can for example sort the items on a column like the traffic light:

Or you can apply a text filter to search for a specific metric:

Checking a metric across multiple systems

If you have an issue in one system, you might want to quickly validate if you have similar issue in different systems, or you simple want to compare with different systems. From the monitoring of a system select the metric.

For this example we selected Short Dumps:

Select the i button to get the explanation text:

This gives the exact name:

Now goto the metric tool:

If you don’t see the correct metric, use the metric selection filter on the top right of the screen:

Press Apply, and you get the overview of this specific metric across all systems in your selected scope:

Storing metric data longer

Focused Run stores the monitoring data 28 days. If you need the data for specific metrics and systems longer, you can make use of the aggregation framework.

On the left hand side choose the option Aggregation Framework:

Choose the button Create Variant to create a new variant:

Fill out the name and basic description and press the Continue with next step button:

The next screen is bit more complex:

In sequence: first search for the extended system ID and press go in the top left section. In the bottom left section, select the system you want. In the top right section now select Add filter from the left button. And press the Add selected objects for aggregation button on the bottom right part. Now press the Continue with next step button:

Select the metrics on the left hand side and add the filters on the right hand side. When done press the Continue with next step button:

Using the aggregation framework

For using the aggregation framework there are no special requirements. Whenever you use an aggregated metric in system monitoring, you can simply use the details with a long period.

Settings for the aggregation framework

In the aggregation framework configuration screen, you can click on the configuration wheel top right to set the retention period for Short/Medium/Long:

System down monitor

A special function is System Monitoring is the System Down Monitor. This overview directly gives an overview of the systems that are considered down by SAP Focused Run and the systems which are set to having maintenance.

In the system monitoring screen select the System Down monitoring icon on the left icon bar (here indicated with the arrow):

You can see systems that are down and which ones that are having planned maintenance. If you have set up the SLA management, it will also show that aspect.

If you want to zoom in on the issues, press the i icon right of the system. Then select Links to go to the respective tool for further investigation:

For systems down the best tools are usually the System Analysis and the Alert Event management.

Changing settings

You can change the layout settings with the glasses icon:

You can show/hide the SLA and charts section as per your need.

Definition of down

The definition of down is in Focused Run: any red alert in the availability metrics. This can be:

  • Complete system down
  • One of the application servers is down
  • A core function is down (for example ABAP stack is up and running, but the Https port is not available)
  • Important subfunctions are not working (for example in the SLT system 1 or more source systems can not be reached)

Summary

The overview above gives the top – down approach in full: from the total landscape, to single system, to group of metrics to single metric.

<< This blog was originally posted on SAP Focused Run Guru by Frank Umans. Repost done with permission. >>