## Tuesday, August 26, 2014

### Testing Philosophy in ODM: Feasibility of Complete Rule Testing in Decision Validation Services (DVS)

Software testing is an important step in the software development life cycle. IBM Operational decision Management (ODM) is not an exception. Testing in ODM is done through Decision Validation Services (DVS). It automatically generates an excel sheet with specified input fields from the Execution Object Model (XOM).  To run a test you need to fill out the excel sheet with test cases and expected outputs. Each row would represent a test case. In this article I would like to discuss the feasibility of running a complete (or close to complete) test in DVS based on the number of fields and the complexity of the decision (rules).

Let us first consider the feasibility of complete test regardless of the technology/software choice. For simplicity let us consider rules around only three fields (Field1, Field2, Field3), which carry binary values. The maximum number of test cases needed to run a complete test for the ruleset is 2^3=8.

 # Field1 Field2 Field3 Ruleset Output 1 T T T 2 T T F 3 T F T 4 T F F 5 F T T 6 F T F 7 F F T 8 F F F

This is a simple example (n=3), but it can help us visualize and understand how to handle the test cases with a lot more fields/elements. Note:  2^3=8 is also the maximum number of unambiguous rules that can be implemented with 3 fields. This can also be visualized as a binary tree. The height represents the number of fields and the leaves of the tree represent the rules. E.g. the left most leaf R1 corresponds to the rule#1 in the above table (F1=true and F2=true and F3=true).

In general for N fields, each of which can take K values (group of values), there can be at most K^N unambiguous rules implemented. In the above example N=3 and K=2 (binary fields). We can generalize it even further. There can be fields with different number of accepted values (group of values).  If there are n fields, each of which can take k values and there are m fields each of which can take p values, there can be at most k^n* p^m unambiguous rules implemented.

Another philosophical question we should ask is how much testing is sufficient. E.g. if we have implemented R1,R2,R3, do we have to implement test cases for the tracks R4, R5, R6, R7,R8 , i.e. do we have to prove that unimplemented rules did not fire(unintentionally)?!?!  The answer is - it depends!! We may have to do 1 or 2 test cases for unimplemented rules, depending on the complexity of the rules. Example: loan approval application that has a rule: “if the age of the applicant is greater than 40 and the credit score is “good” then approve”. If we were to write test cases for that single rule, we might have to do the following:

 # Age is greater than 40 Credit score is “good” Expected output 1 T T Approve 2 T F The default approval status
Theoretically we did not “have to” do the second test case, but it is good to make sure the approval is set by the implemented rules and not by some bug that sets it to  “approve”  regardless of rule conditions.

What if there were 100 fields instead and let us keep them binary for simplicity (would not change the problem if they could accept 3, 4, 5 ... n values). The max number of rules constructed and the max number of test cases need to do a complete case is 2^100.

 # Field1 Field2 … Field100 Ruleset Output 1 T T T 2 T T F … … … … F F F

Often times we implement rules with more than 100 XOM elements and to run a complete test we would need 2^100~1.27*〖10〗^30test cases. Theoretically this is an unfeasible problem if we consider the worst case scenario.

The number of fields is relevant only in theoretical discussion, considering the worst cases (2^n  , where n is the number of fields). In practice though the number of test cases is needed for DVS is equal or comparable to the number of rules.  That means that if it was feasible to create N number of rules, then it would be feasible to implement sufficient test cases for complete testing in DVS.

 Artur Sahakyan is an Associate Consultant at Prolifics specializing in IBM WebSphere Operational Decision Management (v5.xx - v8.xx). Artur has a strong background in mathematics and probability/statistics. He also has profound knowledge of IBM Business Process Manager, IBM Integration Bus (IIB v9), IBM WebSphere MQ (v7), IBM SPSS Modeler, IBM SPSS Statistics, Java, C++, C. 