AWS Blu Insights - Releases

The team is excited to see how hundreds of users are adopting the solution to modernize legacy applications and move them to the Cloud. We have an ambitious plan for this year and the first outcomes are already here in this release full of improvements and new features. You may also noticed that AWS Blu Insights moved from bluinsights.io to bluinsights.aws!

New features

Codebase

• Users are now notified by email when their project is created and ready (no need to wait in front of the screen, although we have a nice collection of funnies).
• You can download the dependencies result as a JSON file if the graph’s generation fails, but the dependencies’ computation succeeds (we just wanted to make sure it does not block you but believe us, you never see that).
• If for any reason you want to force the encoding of your files (expert mode), you will be able to.
• We alert users if the dependency analysis becomes outdated and needs to be relaunched (e.g. new files added using Catch-up.

Capture & Replay

• You can now pause and resume the recording. This may be useful if you need to do some intermediate activities and thus will not have a long video with a long inactivity period.
• Support enabled for workstation id for TN5250 to specify how the server will configure sessions.
• Host Code Page support for TN5250 to specify the Code Page used by the emulator
• Recording duration is now displayed next to the recording buttons to let you monitor a reasonable test scenario duration.

Dashboards

• Permissions on dashboards are now managed through users profile.

Improvements

Transformation Center

We have made most of the improvements to speed up Runs launching and results display.

• Only Velocity output files are displayed in the Runs view. We removed the includes files to reduce loading time and improve readability of the results (no more ignored status). A new button in the toolbar allows to show Velocity Includes used during a specific Run.
• We ignore dependencies extraction when all the selected files are supported by Velocity. This allows to reduce unnecessary computation.
• Parallel Runs are now proposed to users only when AWS Blu Insights require them (i.e. +50M loc in the Run).
• We added a maximal Run limit to avoid performance flaws dealing with Runs page. From now, you cannot keep over 50 Runs (which is sufficient for all projects).
• We removed “Select artifacts” action from Issues since “Load issues” is easier to use.
• We automatically disable the Refactor action (and skip it during Run execution) if we find no Refactor zip.
• We added the columns “Effective lines of code”, “Contractual lines of code”, “Number of Issues” and “Total lines of code” to the Runs.
• Users can now delete multiple Runs.
• Users can delete all the Patch configurations.
• We added a filter to the form Metadata/Property set to facilitate the search for the properties. Velocity properties can now be filtered in the Metadata and Property set configuration based on their name or description.
• We added advanced filtering options in Configuration.
• Update existing project configurations when non-editable properties are edited, added, or removed.

Codebase

• We removed the project size calculation (this information is not useful, and it takes time to compute on large project creations).
• We redesigned the UX of “Show” and “Select” menus in dependencies graph.
• We improved the performance in tree tables (Statistics, Files, …) for large projects.
• We explicitly tell the users of project creation fails cause (e.g. archive protected with a password, too much files, corrupted archive…).
• The “Manage type” option is now disabled after adding new unknown files with Catch-up.
• CL comments highlighting improved in file visualization.
• In Catch-up, users are informed when the module is not available for the demo projects and during ongoing analysis.

Dependencies

• We refactored the missing labels to refer to a category of source files instead of a list of wanted file types, for example “Missing Program” instead of “Missing[COB, PL1, RPG]”
• We optimized the dependencies analysis performance, especially for very long statements and files.
• In JCL, irrelevant “CICS FILE FILE DEFINITION” missing node are now deleted.
• In Cobol, a program without “PROCEDURE DIVISION” is now analyzed correctly.

Classification

• We added the support for IMS (Information Management System), PSB (Program Specification Block), DBD (Database Description) generation and MFS (Message Format Service)
• We improved detection of ASM, Easytrieve, JCL, PL1, CSD and LISTCAT language.

Shared Spaces

• You can now invite multiple users at once (comma separated emails).
• Searching results in Shared Spaces includes their related files.

Capture & Replay

• Removed empty fields verification from json and side files.
• Several layouts improvements have been added (e.g. states, underlying, colors)

System Utilities

• Technologies and label columns are sorted in alphabetical order.
• When system utilities are selected, the count number in the selection pop-up is updated.

Misc

• Do not allow creating projects with special characters in the name to avoid errors when downloading files.
• Support the preview of .webm and .mov videos

Website & Documentation

• We updated the website with new logos, fonts, illustration images, icons and the integration elements to improve the visuals.
• New sections added for Versions Manager and Transformation Center.
• We updated the documentation.

Bug fixes

Codebase

• Delete files with labels can’t be done.
• Update an event in Schedule will create a new event.
• The save button of subgraphs is not working.

Transformation Center

• In the table of Runs, when selecting multiple actions and clicking on the load issues button, the issues are not loaded correctly.

Capture & Replay

• Actions in the last position of a field or in screens without fields are not captured in TN5250.

Misc

• Pictures are not saved in the description fields of To-Do cards.

Behind the scene

• We added +40 more test cases (for front-end, back end and security).
• We performed a security update of all AWS Blu Insights (infrastructure, dependencies, etc.).
• We participated to an AWS Blu Age training sessions (+80 participants).

New features

Transformation Center

• Configuration can be cloned from existing projects for a faster setup.
• A Refactor skeleton zip can be downloaded directly from the Configuration.
• File types column is available in the Velocity Runs tab (and thus can be used in the filters).

Capture & Replay

• Connection parameters (host, port,…) used will persist as default configuration for future visits.
• Capture simplification by regrouping video and interactions recordings in a single action.
• The recording is automatically disabled for non supported browsers.
• All actions are time-stamped in the Json file (based on video start time).
• ESC key is now supported for TN5250.

Dependencies

• PL1: Alias of type Entry are no more handled as missing.
• SQL: Call from SQL files to CL programs is added.
• CL: In CALLPGM statements, use paths to identify the file to link with.
• LNK files are handled. Those files specify the version of a program to be executed at runtime (see documentation).

Misc

• The Web page title is the name of the project and its type.

Improvements

Capture & Replay

• The value of the fields just written to is no longer checked.
• Irrelevant key press for the modernized application are now ignored. For instance, instead of capturing each individual modification (Delete, Backspace…) in an input field, we create a single “Write” Selenium command corresponding to the whole value of the input.
• The value of all fields and text values are now checked using a Verify Selenium command.
• If interacting with a target field is impossible, function keys will now interact with the whole page.
• All verify actions for text values are now performed on trimmed values.

Transformation Center

• Dependencies extraction is ignored when all files or patched inputs are selected.
• Only charts with data are now displayed in the newly created Transformation Center dashboard.
• Names of the Runs are extracted from filters names (if applied).
• Numbers are easier to read, in the Velocity page, as they have commas after every third digit.
• When launching Velocity, the transform step can’t be toggled off.
• Improve error management when a run fails.
• Perform security checks for Transformation Center.
• Prevent to add a new member if they are not invited in Codebase.

Dependencies

• RPG: Handle external datastructure file involving dependencies on a DDS file.
• JCL:
  • Variables are replaced by their value. For example, if a variable “VAR” has a value “A”, an usage like “PROG&VAR” will be replaced by “PROGA”.
  • Dependency analysis will be processed on control cards (CTL) called by a JCL program, like it is done with CPY file for Cobol.

Misc

• Allow users to export data to Excel or CSV files only if they are allowed to download them.
• Improve To-Dos performances dealing with large boards and many cards.

Bug fixes

Capture & Replay

• Press the space bar stops the record.
• Time bar is not displayed in recorded videos.
• “Tab” action not recorded properly.
• Enter action in the SIDE file is not working.
• Write actions captured on protected fields (labels).
• Video recorded twice when we relaunch it after leaving the page.
• Crash after an erase command from the server in TN5250.

Codebase

• When creating a large subgraph, sometimes the main graph is displayed instead of the subgraph.
• The size of the dependency graph’s nodes and text is too tiny on Mac.

Transformation Center

• Generate Patch configuration is not considered.
• The patched inputs were not displayed under the three Categories (Velocity Includes, Velocity Outputs, Others).
• The error icon is only displayed if the transformation center project is not supported by velocity.
• The values shown in the version column of the Velocity tab were not correct.
• The paths shown when versioning a file were not correct.
• We could toggle the Transform and Refactor steps off while having the Generate step toggled on.
• In the Inputs tab, clicking on the path of a file didn’t open a view of the file for all files.
• The versioning feature did not work for all files in the Velocity Issues view.
• In issues view, all columns can be hidden.

Dependencies

• JCL: In-stream procedures were handled as missing.
• Cobol: Cobol sub programs were handled as missing.
• CSD: CICS File definitions were not linked to related CICS dataset.
• CL: CALLPGM dependencies now are supported as a variation of CALL PGM.

Misc

• Email update with invalid email breaks the Profile.
• Sort by “Last Modified” not working properly in Shared spaces.
• Added, deleted, and modified files can’t be displayed in Versions Manager.
• Last modified information on the folders does not have the expected value.

This is one of the major versions of AWS Blu Insights! Most of the work is behind the scene but it brings more stability, better performances, improved User eXperience and much more. Enjoy!

New Features

Dependencies

• Engine
  • CSD parsing to extract CICS object definitions (i.e. transactions, files, datasets)
  • CICS statements are now linked to the CICS object (File and Dataset)
  • “EXEC CICS START TRANSID” and “EXEC CICS LOAD” statements are now detected
  • RPG menu files are now considered “CMD CALL …”
  • Detection of PL/I procedures declared as “generic”
  • Include of include with PL/I .INC files
  • PL/I procedures declared in an included file are now detected
  • Cobol – Handle aliases declared with “SELECT … ASSIGN TO …” in FILE-CONTROL section
  • Names resolution handles special characters
• Visualization
  • Show and select all levels of children, parents, and neighbors’ nodes within one click

Improvements

Performances

This new version of AWS Blu Insights brings multiple performance improvements including:

• Loading projects is almost x6 times faster than before
• Downloading multiple legacy files or Run outputs is almost x10 times faster than before
• Versioning inputs is almost instantaneous
• Deleting files and Runs is almost instantaneous
• Displaying dependencies graphs is almost x12 times faster than before
• Run splits algorithm has been improved for better Velocity outputs management
• Input files copy is parallelized to let Run launch up to x2 times faster
• Loading of inputs page in large projects is up to x3 times faster
• Dependencies graph processing before running a Velocity run is up to x10 faster on medium and large projects, and no longer freezes on large projects

Codebase

• SAVF extraction with specific CCSID (Coded Character Set IDentifier)
• Impacts of the catch-up on cyclomatic complexity, dependency analysis or Zip Extraction are detected and the user is alerted
• Identical homonyms can be filtered

Terminals

• Added support for F13 to F24 function keys, “+”, “-”, and special characters
• When writing in a field, we now write the result of the field instead of all the individual key pressed
• Function keys press are now assigned to a field in the Selenium script
• Copy/Paste is handled
• Hidden fields are underlined
• Control on field types
• When editing the fields’ values, the colors are now set properly
• Error messages are now displayed in the right position
• Browser scrolling turned off when the spacebar is pressed

Transformation Center

• Project name is automatically populated with details from the reference Codebase project
• The Velocity version is saved for future Runs
• Run names are suggested in multiple scenarios
• Export Velocity Issues to Excel
• Preview (with syntax highlighting) for configuration files
• Runs view contains total lines of code, contractual lines of code and effective lines of code columns
• Removed double scrollbars
• Launch of splits using only patched inputs is possible
• Run outputs zip file name is more explicit
• Configuration values are validated against the expected types
• Patches can be deleted by any team member invited to the project
• The welcome message has been updated to match the various scenarios (Customers, Trainings…)

Misc

• The registration email details the required actions to activate the account
• Inactivity emails are sent 5 days prior to disable the account
• Natural system utilities has been added
• Download permissions inside Workspace are also controlled
• Improved activity emails for documents, labels and To-Dos modifications
• Inviting an existing user to a project is not more allowed

Bug fixes

Codebase

• Workspace search results are now consisted when using filters.

TN5250

• Backspaces and delete keys are not working after some server commands
• The cursor is sometimes not being correctly initialized after a screen change
• The terminal sometimes freezes after sending a message to the server with no response

Transformation Center

• Issues view was not showing all issues if they were on the same line of a file
• Runs may fail due to file with one character name long
• Refactor zip configuration is not considered
• Documents Visualization is not working

Misc

• Email update with invalid email breaks the Profile
• Needs to refresh the To-Dos after modificaitons
• Document-based resources (Library/Training) cannot be deleted

Features

• Dependencies – Mainframe:

• • Handle File Definition and Dataset dependencies in COBOL and JCL files.
  • Handle SQL queries in RPG files.
  • Handle another format of LISTCAT files to detect Dataset properly.

• Transformation Center:

• • A new module called configuration has been added to add custom settings.

• • • Transform: Custom properties retrieve from the Back Office to make them match your needs. After this generate a metadata.ini file used by the transformation center with all the custom properties.
    • Refactor: Import an archive used to refactor the code.
    • Generate: Import a SQL model file used to generate the code.

• • Run Transform Velocity action on a selection of files and get updated status at the end.

Improvements

• Transformation Center:

• • Run Velocity on a selection of files
  • A new module called Velocity has been added to view and delete the runs
  • Advanced filtering options on Velocity

• UI/UX:

• • Simplify navigation in the subgraph menu
  • Add new default filter for Mainframe analysis
  • The Status, Labels and Workpackages lists are now sorted alphabetically.
  • Dependencies: Adding a column called “Velocity” based on System Utilities inputs in dependencies tables view. For each System Utility, identified by the dependencies analysis, this column indicates whether it is supported, not required or not supported.

• Performance:

• • Improve project creation
  • Improve Classification performance

Bug Fixes

• Dependencies – Mainframe:

• • Fix detection of ANSI/TANDEM format in Cobol files
  • Fix commented line and string handling resulting false missing in Cobol
  • Fix false missing CTL in JCL
  • Improve performance for RPG project
  • Handle call to sqlcbl as a SQL procedure in SQL and SPZ file
  • Handle call to a file or a SQL table from a select query with an assignment

• Dependencies – SQL

• • Fix commented line and string handling resulting false missing in SQL queries
  • Handle table creation with a system name property
• Dependencies – Java: Fix the java dependencies analysis on the demo project
• Assets
  • Prevent the launch of several dependency analysis by double click
  • Keep the selection between the table view and the graph view
  • Fix the focus in the subgraph and group properties edit pop-up

• Versions Manager: Make the menu visible to all users
• Estimate Console: fix the avatar editing bug.
• Dashboard Settings: disable the Jira Issues card toggle if the project is not linked to a Jira booster.
• Statistics: disable the Manage Types button if classification is needed.
• Labels: do not allow the creation of duplicate label names which are now case-sensitive.
• System Utilities: badge value with filter
• Shared Spaces: Fixed download error on Shared Spaces and Files above 1GB

We are pleased to announce a new version of AWS Blu Insights (Lilac-Throated Roller). This release brings multiple improvements, bug fixes and new features as depicted below.

Dependencies

• • RPG: QDDS/QDDSI and PF/LF files are respectively now linked based on their respective names.
  • SQL: Comment, Delete and Drop statements are now handled.
  • Link types:
    • edges can have multiple types if relevant.
    • link types can be filtered to look for specific dependency types.
  • UX:
    • More details about the graphs (creator, data, size, etc.) are displayed.
    • The zoom is automatically reset after applying global layouts.

Transformation Center

• Mass patching is now possible (versions and comments are automatically generated).
• Revert to the source code file (all the patched inputs of the specific file are automatically deleted).
• Compare two files in order to track modification.
• Automatic synchronization between Codebase projects and their related Transformation Center projects have been improved.

Filters

New keywords have been added:

• “in all” to ensure that all elements of the given array have the filtered element.
• “is empty”
• “uncategorized” is applicable on workpackages, status and labels.

The column “issue” can now be filtered (issue = id of the issue)

Assets

• Loading performance has been improved especially for projects using SAVF Extraction and Classification.
• Homonyms are pre-filtered to compare only files with the same type.

Test Scenarios

• A daily Test Scenario report is now sent to summarize the latest results and highlights those who failed.
• The name and date of result updates are visible.

Most significant UX improvements

• Home – It is possible to clear the list of recently visited projects.
• Login/out – If your session expires, AWS Blu Insights will restore the last open page.
• Activities – Images are correctly displayed in the properties dialog, activities, and emails.
• To-Dos boards can now be filtered and sorted on creation date or name.
• GitLab – activities related to the GitLab booster are logged.

Behind the scene

We have also been working on some features that are used behind the scene to improve the whole product. We list here the most relevant:

• Dependencies debugger: we are building a debugger to let us (probably you in the future) to have, in addition to the dependencies graph, a suggestion of potentially missing links (especially for languages that are not supported yet) that can be inserted in the results and allow in a few clicks get more consistent dependencies.
• Security: we collaborated with security experts in order to review and scan all AWS Blu Insights security (application, infrastructure, tools, dependencies, etc.) in order to make sure our customer data is always protected and securely stored.
• Stability: we substantially increased the number of automated test scenarios of all our features to make sure we are always aligned with our promised SLA.

In addition to our regular activities, i.e. delivering new features, improvements, and bug fixes, the last AWS Blu Insights dev cycle focus has been dedicated to security and reliability. We depict below the main achievements.

Security and Reliability

Protecting customers data has always been a non-stop activity for the AWS Blu Insights team. In each batch (4 weeks dev cycle), we plan different topics to test and improve the security of the application, the infrastructure, the data workflow, the APIs/Frameworks versions, etc. During the last batch, we decided to check everything from scratch as it has never been done before! In fact, we checked all permissions, endpoints, security annotations, tokens validity, etc. We had also checked all our infrastructure configurations to apply all the recommended options and make sure nothing is set by default. We are also carrying out a deep security audit and pentest in collaboration with security experts.

PS: AWS Blu Insights does not use Apache log4j and thus has not been impacted by the security vulnerabilities detected a few days ago.

New features

• GitLab Booster: It is now possible to connect AWS Blu Insights to your own GitLab and link your artifacts (files, test scenarios, and workpackages) to your GitLab CI Jobs and/or Issues. Specific charts have also been added for this booster.
• Transformation Center: AWS Blu Age modernization experts can now create a specific project type to manage the transformation of their legacy assets. This new project type brings a new set of capabilities to accelerate, simplify and automate Velocity transformation projects:

• Versioning of source code files
• Tracking of files modification (patches) using specific attributes such as descriptions, versions, created by, etc.
• Download and delete patched files
• Advanced filtering options on patched file
• …
• Timesheets: Team members can report their activities directly into AWS Blu Insights (dedicated Time Tracker projects). Based on customized configuration (tasks, profiles, etc.), managers can track their project duration/budget.

Improvements

• Classification: Mainframe SAS (Statistical Analysis System files) and EZT (Easytrieve) detection enhancement.
• Cyclomatic Complexity:  DotNet (C#) enhancements.
• Dependencies:
  • Java: Performance improvement on large projects.
  • Mainframe & iSeries
    • Statements EXEC CICS MAPSET with INTO or SET term are now handled
    • Delete false missing due to statements in string value
    • JCL, INCLUDE MEMBER= can now address JCL or PROC files
    • Multiple minor fixes for Cobol
  • Natural
    • Delete false missing and dependencies due to inner subroutine call
  • Powerbuilder
    • Delete missing links due to indirect dependencies
• UX/UI: Multiple small improvements have been implemented to enhance the UI and the UX.

Bug fixes: Multiple minor bugs have been fixed.
Updated documentation/Website: The documentation and the Website have been updated with new details/content.