Get Started

Introduction

The Blu Age Compare Tool gives you the ability to efficiently compare your data in an incremental and fix-oriented perspective. It enables you to get your differences, decide if some are normal and some are not, fix the abnormal ones and setup the tool to ignore the normal ones, then repeat, until the result is acceptable.

Key Benefits

Use case

Tables will be compared in two given databases, files will be compared in two different folders. A convention in the tool is that one of the compared database/folder is on the left and the other is on the right. At the end of the comparison, an HTML report is generated, which will display if the data match, or, if they don’t, what is only in the element on the left, what is only in the element on the right, and what appears in both but with differences.    

When there are several elements (files/tables) to compare, comparison will be performed in parallel if possible, depending on the capabilities of the machine.    
 

Installation

The compare tool is available in Blu Age Toolbox.

Prerequisites

• You need to have access to the Compare Tool S3 bucket. If you don’t have access yet, you can request it via Blu Insights Toolbox.
• You need to have minimal IAM policies on your account to download the Compare tool from S3 bucket. In case you don't have policies, then create one with the below information.

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Effect": "Allow",
          "Action": [
              "s3:ListBucket"
          ],
          "Resource": [
              "arn:aws:s3:::toolbox-compare-tool"
          ]
      },
      {
          "Effect": "Allow",
          "Action": [
              "s3:GetObject"
          ],
          "Resource": [
              "arn:aws:s3:::toolbox-compare-tool/*"
          ]
      }
  ]
}

• Java 8 needs to be installed.

Step-by-step guide

Toolbox buckets are replicated on the us-east-1 and us-east-2 regions. To use these replicated buckets, append the region to the bucket name: e.g. s3://toolbox-compare-tool-us-east-1. Make sure to adapt your user or role policy accordingly.

1. Check that you configure the AWS credentials with the AWS account used in your request made in Blu Insights Toolbox.
2.  Download Compare Tools last version archive using the command aws s3 cp --recursive s3://toolbox-compare-tool/latest LOCAL_PATH
3. Unpack the archive present in LOCAL_PATH.
4. The ‘drivers’ folders are empty. If you intend to use the database comparison feature, you must fill the relevant folder with the appropriate driver.
5. Run the command: <Compare-exe-path> -root <Compare-project-path> -configuration <Configuration-file-path>
6. Compare Tool will launch the comparison and generate the HTML reports at <Compare-project-path>/report

Running a comparison

Typical project structure

The ‘configurations’ folder contains one or several configuration files to launch comparison. Those have a ‘bacmp’ extension and a json structure.   

The ‘db’, ‘flatfile’ and ‘text’ folders are used depending on the comparison type.   
 

• For a database comparison, the ‘queries’ folder will contain the sql files for the data to be compared.
• For a flat file comparison, the tool will compare files found in the ‘left’ and ‘right’ folder.
• For a textual comparison, the tool will compare files found in the ‘left’ and ‘right’ folder.

If several comparison types must be performed, folders and configuration files for different types can be mixed in a single project (but a given configuration file is for one mode only, so there must be several configurations in the ’configurations’ folder)   

Please note that ‘configurations’ is the default folder name used throughout this document, but you can in fact use any name you want. It is not enforced in any way. The other folder names are default values used if not further specified in the configuration, as explained in Additional properties (for database, flat files and text files)    
 

Return code, log and HTML report

The compare tool can use three ways to communicate back to the user: it yields a return code, it writes information in a log, and it generates a report in HTML.    

The HTML report is covered in details here.    

The log is displayed in the console, as well as in the log file. The log level can be setup using ‘logLevel’ option. (see Additional properties)    

The return code of the process can have five values: 

• 0 – the comparison ran correctly and did not find any mismatch
• 1 – a technical unexpected error occurred during comparison
• 2 – the configuration file is invalid
• 3 – the comparison ran correctly and found mismatches
• 4 – a functional error occurred during comparison (for instance, a missing file)

In case of a code 1, 2 or 4, the log contains exceptions and/or detailed error messages.

Overriding configurations

One can use several configuration files in the command line, separated by a semicolon. The first one will be read, then the second one will be read and possibly override some options, and so on.    

This feature is useful to patch a given configuration, by amending some options (in this case, the “main” configuration file comes first), for example:   
java -jar CompareTool.jar -root C:\Users\c.kent\workspace\MyCompareProject -configuration compare\configurations\my.bacmp;compare\configurations\patch.bacmp   

Conversely, it can be used to have some common configuration options as a template (in this case, the “main” configuration file comes in second). for example:    
java -jar CompareTool.jar -root C:\Users\c.kent\workspace\MyCompareProject -configuration compare\configurations\template.bacmp;compare\configurations\my.bacmp