The Business Intelligence systems collect data that present a coherent picture of the enterprise at a point in time, integrate and process it in an analytical mode. The resulted information, presented in a flexible and easy to use format, supports managers in the decision-making process. The BI systems testing role is to ensure high quality of the information that supports managers to make decisions with regard to the future of the business.
To measure the business state and performance, the Decisional Support System analyst defines Key Performance Indicators - also called business measures - and the perspective from which they look at these - dimensions.
Starting BI testing process, we pay attention to the business it is dedicated for, as it is described by the DSS analyst. While testing BI systems, we stay focused on: Data Integration (ETL) process, Data Quality (objective) and Information Quality (subjective).
Data Quality (objective)
Information, the main asset of the organization, is stored in operational systems (OS) and Data Warehouses (DWH). To ensure a valuable basis for the analytical informational processing, we check that the data coming from the OSs is properly transferred into DWH. Checking data quality, we take into account quality dimensions as follows: data completeness and accuracy and time-related dimensions.
To perform it, we split data stored into DWH into categories, with regard to business areas, each of which corresponding to an OS (source system - Booking, Sales, HR etc.). After that, we compare data stored into DWH corresponding to a category (business area), with data stored into OS that corresponds to the same category.
For each data category, we establish the items that will be checked during testing process, prioritizing them by their importance for the business process (Booked amount, Invoiced amount, Employees holidays, etc.).
The next step is to establish the aggregation level we will check data for, in terms of the dimensions defined by DSS analyst (year by year, month by month, department by department, etc.). For easier comparison of large amounts of data, we apply mathematical and statistical functions on the items chosen for test, taking into account established aggregation level (e.g. add invoiced amount from sales year by year).
Information Quality (subjective)
The input data of the BI application comes from different OSs, representing the state of the business at a point in time. The change of the business state generates a new set of input data. The business state is modified when at least a business event occurs at the OSs level (new entry, change, delete, etc.).
To test the information quality, we design a logical flow of business events raised at the OSs level that should produce input data desired for various test situations (designed in the test design phase of testing). This business scenario is described in a document.
If the business ran according to the above mentioned scenario, it would produce a set of input data for each business state. What we do is to simulate it by creating these sets of data - synthetic data. Creating synthetic data, we take into account the source systems data model and the way used to send the data to BI system (flat files, tables, etc.).
To obtain expected result, we calculate the KPIs values that should be displayed, looking to them from various perspectives (using various dimension combinations). We calculate it based on the business rules described by DSS analyst.
Finally, we compare the expected result with the KPIs values produced by the BI system, when the application runs using synthetic data as input data.