Assured CX powers Socotra’s InsureTech disruption

Summary

Socotra, an enterprise grade end-to-end insurance platform undertook to upgrade its platform that would offer an integrated front end and simplify the experience for both online customers and their agents. This was planned to be rolled out in 6 months. The tight timelines required extensive functional testing and automated checks across all stages of the development lifecycle.

Background

Socotra is the first true-cloud, enterprise grade SaaS Platform for Insurance that enables insurers to rapidly build products and experiences for its millennial customers. They can configure and create any type of insurance product, for any region, and distribute it any way that they choose.The platform handles the end-to-end process of insurance product development, pricing, underwriting, risk assessment, policy issuance, premium payment, customer onboarding, claim management and payments.

To take advantage of the latest technology capabilities, Socotra undertook to upgrade its platform that would offer an integrated front end that dramatically simplified the experience for both online customers and their agents.

The new front end interface needed to:

  • Speed up, by upto 3x, the ability for a new customer to go-live on their platform
  • Reduce the costs of customer implementation by upto 50%
  • Provide a frictionless customer experience for online consumers
  • Ease navigation across the process with improved menus
  • Switch from traditional linear flow and be able preset multiple policy states
  • Improve view and drill down into data across functions
  • Improve cross site search ability

To maintain its disruptive leadership position in the market, Socotra planned to roll out the upgrade in 6 months. Challenging, as it included design, development, deployment and migration of all its customers to the new system with zero business disruptions.

Quality Strategy for Socotra’s Digital First Experience

To achieve the desired Customer Experience, Socotra switched from using a very old version of React, Typescript, Gulp and Bower to the latest version of React and Typescript, supported by Webpack and npm.

As the new customer experience was designed to provide customers greater flexibility and choice, it also exponentially increased the business rules and data requirements that needed to be validated.

Socotra partnered with Trigent to augment and extend their QA capabilities, undertake functional testing and build the automated checks across the new platform.

As a seamless extension of Socotra, the Testing team got involved in defining the test strategy, the core elements of which were:

  • Shift left the tests early by identifying errors or missing business rules early in the DIY workflows
  • Build rapid failfast cycles of tests to ensure errors in weekly builds were trapped as early in the sprint cycle
  • Ensure a strong focus on consistency of the UX across all stages of the insurance process

Test automation was integral to the strategy and would help:

  • Eliminate the time-consuming and repetitive manual regression testing
  • Slash test cycle time from the expected 3 days that it took to run a suite of tests
  • Integrate tests into the CI/CD pipeline to be executed at all development stages
  • Reuse test scripts to check workflows across multiple customer configurations

Automation Strategy Implementation

The automation team focused on the component integration and the end to end business functional tests parts of the test pyramid. Scenarios to automate were chosen to maximize coverage in areas of risk and business criticality.

The API integration test cases were to be run at every pull request in the local development branch, periodically, in the main development branch and whenever a release was undertaken (usually weekly). API tests were to be integrated into the CI/CD pipeline and controlled through Jenkins jobs.

UI tests, that covered critical business functionality, were to be run twice daily to check for regressions on the development branch and against release candidates on different test environments.

A standardized test framework was to be setup that would:

  • Allow dynamic selection of tests, that could be run for various checks (Smoke/Regression/Release Verification etc.)
  • Link the results of one test run to a subsequent run (e.g UI test runs were to be linked to API test results)
  • Anonymize, reusable test data sets to use across the different test environments that were to be set up and referenced during the test runs
  • Run the tests in parallel across multiple test environments for faster feedback

AutoMATE, Trigent’s Test Automation Framework

Trigent’s AutoMATE framework, formed the basis and jump-started the framework development process, thus saving time, in an already time crunched project.

The framework accelerates script development, reduces dependency on tool-specific resources, and the overall automation maintenance costs. In addition to supporting multi-browser, multi-device testing, integrating with standard CI/CD tools, AutoMATE’s AI/ML utilities also enabled effective Test Impact analysis and test selection.

The enhanced framework, enabled

  • Implementation of page object models having separate classes for each page
  • Parallel execution of tests with no dependency on any pre-conditions
  • Creation of data through APIs (stored in JSONs) and passed on to UI tests for validations
  • Avoidance of code duplication
  • Handling dynamic waits thus reducing the flakiness of tests
  • Implementation of utilities like Date Utils to format date, Faker utils to generate fake data
  • Generation of Mochawesome reports showing detailed representation of test cases and messages
  • Static code analysis using ESLint

Choice Of Tools - horses for courses

Cypress was chosen because of its ease of set up, and the ability to generate automated tests quickly. The possibility to extend the tool to cover API testing, was also key. The ease of integration with the requirements & test management tools like Test Rail and JIRA were other factors.

TestRail as the test case management tool provided the ability to link manual and automated tests, thus providing a single reporting view of all test executions for every release.

Jira was used to track user stories, create defects and track the sprint related activities.

Github was used for maintaining code.

ESLINT for static code analysis offered a wide set of default rules that could be enabled or disabled as well as ensure consistent code formats.

Mochawesome for eporting as it supported attractive reports with screenshot views, and the ability to drill down into contextual information related to the tests.

Postman for API testing allowed easy setup and ability to create collections and parametrizations for tests and debug test results.

Continuous integration and testing in different test environments was undertaken using Jenkins as the orchestrator.

Automation Coverage Achieved

Automation of 90% of the test scenarios across high risk and business critical scenarios, was achieved. This included 600+ API tests and 150+ End To End business functional scenarios.

The entire CI/CD pipeline was set up so that the checks covered:

  • Validating code correctness using ESLint
  • Building test data (encapsulation of test data in JSON into the test case)
  • Creating Json data files for each entity with all the required test data; handling different data combinations to validate for edge and corner cases
  • Randomizing the functional scenario execution to simulate a real-time environment. For example:
    • Multiple policy workflows with random policy actions
    • According to business workflow from policy creation to lapse
    • Covering edge scenarios and negative scenarios
  • Deploying correct Configuration through API and executing tests on that environment
  • Execution of tests on different customer configurations
  • Adding sufficient assertions and verifying all possible validations
  • Verifying audit logs are generated correctly and verifying correct date,time stamp etc.
  • Trapping any console errors

Continuous Improvements in Execution

All the requirements were converted into detailed test cases and all critical test cases (automation candidates) were automated at either the API level or UI level to provide the right levels of confidence. Achieved a 90% automation of all regression scenarios.

Effective testing - Reported a good number of defects in the application which in turn reduced the production defects - Reduced the number of post production issues, in customer configurations by 75%.

With code checks + 600+ API tests run at every pull request, daily (and on every build) runs of 150+ UX tests - which represented 90% of all manual regression tests - helped identify regression issues early and reduced the testing cycle time by as much as 70%.

The ability of the framework to integrate seamlessly into the CI/CD process helped improve the sprint velocity by 25%.

Increased confidence in the product, such that the overall test execution time for each intermediate release was reduced from 3 weeks to 2 days.

Best Practices Followed

  • Simple and transparent communication using various channels like Slack, workflows in Jira + TestRail and regular daily, weekly meetings and retrospectives
  • Collaboration between product, development and implementation teams
  • Constantly review the various automated suites to eliminate duplication of effort and speed up feedback to the development teams
  • Begin automation early in the life cycle despite incomplete behaviour/code and constantly refactor automation scripts
  • Leveraging tags to group automated checks to be able to quickly identify, execute and report on particular impact areas
  • Judicious use of End To End automation and emphasize on the creation of API tests that are faster
  • Use APIs to create data and then pass to UI Tests
  • Created tests to be independent and should work in any environment without any dependency
  • Spin up instances for test environments to be able to run tests in parallel
  • Treat scripts like code - ensure good documentation, standardized coding practices and coding styles