Pushing the frontier of development tooling and environment for Scala and JVM ecosystems

The challenge

Our client, one of the Tier-1 financial institutions in the world, was developing a large project. It was written in the Scala programming language and had a few million lines of code. The client was developing the project 24/7, as their operations were stretched between multiple time zones.

Due to the size and complexity of the project, our client experienced issues with the performance of their tooling, which contributed to delays in delivery. The client needed new tooling to improve their productivity and meet the project's deadline. With most of their engineers and resources focused on current development works, they lacked the manpower to do so themselves. 

The client decided to turn to VirtusLab for help, impressed by our reputation in the industry as experts in the Scala and JVM ecosystem.

The solution

Our engineers developed a solution that covers every area of the typical Software Development Life Cycle (SDLC), such as development, testing, and deployment. Here’s what we did in each of these areas:

Development

To enhance development works, and speed up compilation time, our engineers:

• Ensured a smooth transition to Jenkins pipelines.
• Implemented unit testing frameworks that work seamlessly with Jenkins.
• Created an extensive PR review/testing rules system that comprises about a thousand rules.
• Built a command-line tool customized for the project's specific requirements, complemented by a set of IntelliJ IDEA plugins.
• Created support for Git-sparse checkout to help developers manage large repositories more efficiently.
• Made improvements to the existing Scala compiler to enhance performance and reliability.

Testing

Our engineers optimized the client’s testing process. They built a system that had test granularity beyond any other solution currently available on the market. It uses the approach called “scoping”, which smartly pinpoints and runs only necessary tests related to the latest code updates. It ensures that testing is both focused and efficient. The new system also supports:

• Running tests in a cloud environment
• Automatic test distribution
• Monitoring
• Analysis

Deployment

A deployment process was improved with a custom system tailored to meet the requirements of the client's Scala project and integrate with their existing systems.

The results

VirtusLab optimized the delivery time of the client’s project. The average build time went from 1 hour to 10 minutes or less. The project now benefits from automatic deployment and faster merging of pull requests. Our improvements also:

• Enhanced the development environment. 
• Ensured code quality and consistency using the new rules system. 
• Made the integration process more efficient with unit testing frameworks.
• Enabled a smoother transition to Jenkins pipelines, which allows developers to write their build scripts in a Groovy syntax. 

A smoother transition to Jenkins pipelines made the deployment process more flexible and user-friendly. It simplified the testing of code modifications and made developers more keen on making changes to the build process, speeding up the development process.

The new testing environment is a single fire-and-forget solution, which allows engineers to work on other things while it sets up. It also allows the client to run up to 70% fewer tests.

The client now automatically picks relevant reviewers, tests that need to be run, and conditions that need to be met for the Pull Request to be mergeable. That lets the client save time on builds, tests, and reviews.

The tech stack

Languages: Scala, Java, Groovy, Python

Automation: Jenkins

Repository: BitBucket

IDE: IntelliJ

Tooling: Akka

Development platform: Docker