A leading investment bank operates a large-scale Scala monorepo that has evolved over the past decade. As their codebase grew, the client noticed that IntelliJ IDEA’s indexing process increasingly disrupted development workflows, creating a major productivity bottleneck. To ensure seamless operations across the organization, our client aimed to reduce these delays. They turned to VirtusLab for support, encouraged by our expertise in IntelliJ.
The challenge
IntelliJ indexes project files to enable features such as code navigation, search, and refactoring. However, in large repositories, this process can be slow. The IDE detects significant source file changes—such as switching branches or pulling upstream updates—and initiates indexing. During this time, most IDE features are unavailable, forcing developers to wait before resuming their work.
At scale, these delays had a substantial impact on the investment bank. Developers frequently waited for indexing to complete, resulting in 1,270 hours of lost productivity per month across the organization. This figure came from both full and incremental indexing.
Full Indexing
Full indexing occurs in scenarios such as:
- Setting up a new workspace
- Regenerating/cleaning existing workspace for troubleshooting
- Cleaning indexes during troubleshooting
Before optimization, full indexing was a major pain point:
- 25% of users performed a full reindex 1–2 times per week
- Each operation took up to 30 minutes
- Even after upgrading to IntelliJ 2023.3, full indexing still required almost 14 minutes per refresh
This led to an average productivity loss of 20 minutes per developer each week.
Incremental Indexing
Incremental indexing occurs during routine tasks such as Git pulls or checkouts.
- ~50,000 file changes: 1 minute, 10 seconds
- ~500,000 file changes: Increased delays, scaling with file volume
As the monorepo grew, indexing time became a growing challenge.
Diverse JDK requirements and lack of detailed metrics
On top of the delays, the project required seven different JDK versions, which added indexing overhead. Our client also lacked telemetry on indexing performance. Without comprehensive data, it was difficult for them to diagnose delays and prioritize improvements.
The solution
Our engineers took a two-step approach to the problem:
- Optimized IntelliJ’s existing indexing process to reduce unnecessary workload.
- Implemented shared indexes to minimize local processing.
Step 1: Indexing optimizations without shared indexes
Before implementing new methods, our engineers identified bottlenecks and optimized IntelliJ’s indexing behavior.
Enhanced telemetry and monitoring
- Extended IntelliJ’s logging to capture indexing performance data.
- Integrated performance insights into a centralized dashboard.
- Analyzed reports to identify and resolve bottlenecks.
Selective file filtering
Our engineers introduced a configurable filtering mechanism to reduce redundant indexing in three categories:
- By file size – Excluded large, rarely used files.
- By file location – Skipped unnecessary files inside JARs.
- By file type – Omitted non-essential formats (e.g., large XMLs, map files, test data).
Also, they provided two configurations:
- Balanced configuration – Optimized indexing without disrupting workflows.
- Extreme configuration – A more aggressive option for maximum performance. This setting can improve indexing speeds further but may limit certain functionalities, such as project-wide text search in potential XML configuration.
This gives our client’s developers the ability to fine-tune settings based on their team’s requirements.
Step 2: Implementing shared indexes
To further accelerate indexing, our engineers developed an automated system to pre-build and distribute indexes for JDKs and the project codebase.
Comprehensive index generation
- JDK Indexes – Pre-built for all seven JDK versions.
- Project source code and dependencies – Pre-indexing libraries, sources, and other assets.
Optimized Index Distribution
- Pre-built indexes are stored in a geo-distributed storage system with synchronized data across multiple regions. Users are directed to the appropriate regional server based on their location to ensure efficient access.
- A high-performance proxy server handles index retrieval, optimizing transfer speeds and reducing latency for developers.
Automated CI Integration & Maintenance
- Indexes are generated periodically on Continuous Integration (CI) infrastructure and automatically cleaned up when outdated.
- Developers do not need to manually configure or manage these indexes, making adoption seamless.
This solution is fully integrated with IntelliJ Idea Ultimate, ensuring developers benefit from dramatically faster indexing times without additional overhead.
The results
Our optimizations significantly reduced indexing times, minimizing disruption for our client.
Improvements in full indexing
- Reduced indexing time from 14 minutes to under 6 minutes, which depending on the conditions, translates to 50–65% reduction.
Improvements in incremental indexing
- In scenarios where ~50,000 files change, incremental indexing now takes approximately 45 seconds, a 35% improvement over previous times.
- The larger the incremental indexing operation, the greater the speedup—reaching up to 50% or more for high-volume file changes (e.g., 500,000+ files).
Real-time monitoring provides ongoing insights to track and fine-tune indexing performance.
Business impact
Based on time saved alone we estimate that by reducing indexing delays, our client saved about $420,000 annually. This excludes additional benefits such as:
- Reduced cost of opportunity.
- Faster time to market.
