In this blog entry, I will discuss:
- A structured approach to SAP testing
- SAP current test automation paradigm and its challenges
- The need for a new solution for SAP test automation
- How Arsin Packaged Test Automation for SAP integrated with IBM Rational Functional Tester helps address these challenges
A Structured Approach to SAP Testing
SAP implementations pose some of the most intriguing and difficult challenges in the QA universe. The thickly netted system is extremely integrated and is typically linked to every business process in the enterprise. To tackle such an immense system, QA engineers must approach SAP applications with care.
With more than a decade of experience in testing SAP systems for a large client base in myriad industry verticals, we have developed a test maturity model assessment and improvement framework to bring about an organized and a structured approach to SAP testing. This framework has a three pronged approach, which offers process improvement, knowledge management, and test automation, as follows:
1. Process improvement. Process improvement deals with the assessment of the current Test Maturity Model and developing a plan to improve the Test Maturity Model to the next level and then implement it. A mature test process that has standardized templates, well-defined processes, clear protocols, and no bottlenecks provides for a complete and comprehensively tested SAP system. By comparing the current test maturity model with industry standards and identifying the gaps and focusing on them, test maturity can be improved.
2. Knowledge management. Knowledge management deals with institutionalizing QA knowledge collected over time. Traditional testing for SAP systems relies on the functional and technical consultants of the SAP system for subject matter expertise to deal with various instances. In this phase, test artifact libraries are built for critical business process for regression. The following test artifacts are documented:
- Test Requirements
- Test Cases
- Test Procedures
The remainder of this discussion focuses on the test automation aspect of the Structured SAP Testing Approach. Our belief is that RFT, in conjunction with Arsin's Effecta Validation Engine, makes SAP testing thorough, comprehensive, easy, and cost effective.
Importance of Test Automation in SAP Implementations
The SAP landscape is continuously changing, as a result of changes to SAP modules from SAP, business process changes within the client’s company, changes to the system environment, changes to applications interfacing with SAP, and a multitude of regulatory compliance mandates.
In order to keep up with these changes, SAP systems must be thoroughly tested. With every change, there is a regression library of test cases that needs to be executed to ensure stability. Each test requires time and effort when executed manually; by comparison, automated test take a very small fraction of the time and effort to execute. Automation also helps makes most of the test assets reusable.
Current SAP Testing Solutions and their Limitations
The existing SAP testing model on the market today makes a very rudimentary use of automation, in terms of:
Validation: In most cases, user interface (UI) tools that are available are used to automate test execution, which is only about 25% of the total testing effort. Validation represents more than 75% of this effort, and scrubbing the data using UI test automation tools is difficult. A certain level of validation is possible through UI based test automation tools, however it takes a long time to script this validation and any change requires a lot of coding following the first implementation.
Data management: Traditionally, data used for testing is captured and maintained in spreadsheets. Searching and sorting through this data is difficult, as is maintaining the consistency of data across users and locations. This difficulty is compounded by ever increasing volumes of test data to be maintained. In addition, there is no intelligent association between SAP metadata and its corresponding test data.
Managing change: Changes to SAP implementations occur during reconfiguration or the addition of custom-built components (programs). In these situations, the scripts for automated test execution need to be changed regularly, which is difficult. Moreover, when using UI tools for automation, 75% of the effort needs to be constantly re-worked to keep up with the changes to the SAP system.
Addressing these Limitations
The limitations above described call for a new solution that can address these issues. We offer a complete and scalable testing solution that combines Arsin Effecta Validation Engine with IBM Rational Functional Tester.
By automating the validation of data, business processes, custom development and integrations across SAP applications, you can increase the quality of implementation, support multiple changes in their environment and mitigate business risks. Also, by eliminating manual testing you can avoid greater difficulties in production that ultimately impact the quality and performance of the business. Arsin’s Effecta Test Suite provides the benefits of a complete testing solution by automating impact analyses of changes, test data maintenance, test execution and validation.
Figure 1: Arsin Effecta Solution Architecture for testing SAP applications
Test Data Manager
Stores test data along with criteria in the Effecta database. Before executing an automated test, validity of test data is checked on the target system and system is automatically updated. If the test data no longer exists in the target system or cannot be reused, the data set update feature will help to refresh with new valid data.
Script Manager
Automatically enhances recorded scripts and eliminates need for customization. Script manager enables script less automation of IBM Rational Functional Tester.
Change Impact Manager
When changes occur in a system, Change Impact Manager automatically extracts affected objects and identifies test cases to be executed for regression testing. It also identifies objects being changed that don’t have test cases in the library.
Report Manager
Report Manager provides out-of-the-box reports for tracking test artifacts, development and test execution. Detailed test results pinpoint failed events in test case.
Test Manager
Effecta promotes reusability and repeatability with the following features:
- Ability to create Test Requirements and link them to Test Cases and development objects
- Ability to create Test Cases and link them to Test Requirements for coverage analysis
- Ability to create separate Test execution steps in the form of Test Procedures and link them to Test cases
- Defect management
- Dashboard for reporting and metrics
Validation Manager for Middleware
Simulates inbound messages at various data interchange points and validates outbound messages.
Automatically validates translations and mappings
Validation Manager for Transactional Systems
Validation Manager for Transactional Systems is a completely configurable, customizable and readily deployable validation library of components for various business processes. It significantly accelerates validation by automatically extracting the actual data created by transactions and comparing it with expected results. The Validation Manager is specifically designed to support SAP systems.
Validation Manager for BI
Tests Business Intelligence systems during initial implementation and during maintenance and support pack deployments. It also automates the validation of data loaded from multiple ERP and other systems. Provides sophisticated reporting including detailed results.
Conclusion
The benefits of using automation in SAP testing are abundant. Test automation, deployed with minimal effort, enables increased test coverage, which in turn reduces cycle time and enables efficient bug detection early in the development cycle. Since test automation is designed for reusability, routine tasks are eliminated and total cost of ownership is reduced. Test automation is far more precise and consistent, and features standardized reporting, enabling clear test analysis across the QA environment.
Sarat Addanki is the Vice President, ERP Practice. He has 18 years of experience in the ERP arena including design, development and testing of ERP implementations. He was part of a SAP Quality professionals team contributing to the design of SAP Test Accelerator TAO. He founded the ERP Division at Arsin, which focuses on developing frameworks and accelerators to ensure delivery excellence, reduce the overall cost of ownership and increase productivity in ERP implementations. The Test Accelerators he designed significantly improve the testing process, knowledge management and test automation. His division focuses on providing quality services for SAP, Oracle, PeopleSoft, Sterling, Retek and Middleware applications. His domain expertise ranges from Pharmaceutical Distribution, Hi-Tech, Manufacturing to Retail industries. He is a PMI (Project Management Institute) certified Project Management Professional (PMP). Sarat holds a bachelor's degree in Computer Science and Engineering from Osmania University, Hyderabad, India.
