Feasibility of Multi-tenant JDT LS & Memory Optimization Strategies (Bachelor Thesis @TUM)

[CodeByNikolas]

Feasibility of Multi-tenant JDT LS & Memory Optimization Strategies (Bachelor Thesis)

Hi everyone,

I am currently working on my Bachelor’s Thesis at the Technical University of Munich. My research focuses on optimizing resource efficiency for cloud-based development environments (Eclipse Theia) in large-scale educational settings.

The Context
We deploy Theia on Kubernetes for programming courses where hundreds of students often work on the exact same "starter project" simultaneously. Currently, we provision a dedicated container with a separate JDT LS process for every user. Scaling this to thousands of concurrent students results in massive memory consumption due to the overhead of thousands of JVMs running in parallel.

My Architectural Goal
My initial thesis proposal involved designing a "Shared JVM" architecture, where a single centralized JVM hosts multiple, logically isolated JDT LS sessions for different users to reduce the JVM memory overhead.

Question 1: Feasibility of a Shared JVM
Based on my analysis of the Eclipse/OSGi architecture, I suspect this approach is technically impossible because:

1. The Eclipse Workspace appears to be a strict singleton per process.
2. OSGi architecture and static states in core components likely prevent running multiple isolated instances in one JVM.

Could you briefly confirm if this assessment is correct? Is strictly one JVM process required per JDT LS session?

Question 2: Shared Indexes for Identical Projects
Since many of our users work on identical starter projects with the same dependencies (JDK + Libraries), every single JDT LS instance currently computes and stores the exact same index data.

• Is there an existing mechanism or configuration to create a shared, read-only index (e.g., for the JDK and standard libraries) that multiple separate JDT LS processes can attach to?
• This would allow us to avoid redundant computation and storage across the cluster.

Question 3: Application Class Data Sharing (AppCDS)
To mitigate the overhead of running many separate JVMs, I am considering using Java AppCDS.

• Do you have any experience or benchmarks regarding the use of AppCDS with JDT LS?
• Would this be a recommended strategy to significantly reduce the memory footprint and startup time in a scenario with high pod density?

General Recommendations
Beyond the specific points above, I would greatly appreciate any other suggestions or "low-hanging fruits" regarding configuration flags, JVM tuning, or architectural approaches to further reduce the memory footprint and file size of the language server in a containerized environment.

Any insights, architectural constraints, or pointers to relevant documentation would be incredibly helpful for my research.

Best regards,

Nikolas Hack
Technical University of Munich

[mickaelistria]

The Eclipse Workspace appears to be a strict singleton per process.

Yes, it is.
It could be improved, one could imagine allowing to instantiate and access multiple Workspace instances, and access them accoridng to some session ids; but brining that to life would be a lot of work, and making sure it's well used by its consumers would be true.
Note that AFAIK, another important object, the JavaModel is also a singleton (which happens to rely on the Workspace singleton too).

OSGi architecture and static states in core components likely prevent running multiple isolated instances in one JVM.

Yes, although it's less important (if the workspace and JDT model were multi-tenant, multiple used could use the same OSGi instance and just access the information that's visible to them).
You may want to read about how Eclipse RAP and Dirigible achieve some multi-users applications served by the same OSGi instance.

Shared Indexes for Identical Projects

I think it's theorically already possible to provision a project and/or a workspace with pre-build indexes for jars so there is no need to read the jars (the pre-built index is directly pushed to the local index). You'd like to read https://www.eclipse.org/lists/jdt-dev/msg01688.html and look at ClasspathEntry.getIndexLibraryLocation() to get a sense of how to use it.
But still, each instance of the JavaModel will have in memory a copy of the index. Sharing index as files would allow mostly saving CPU and disk space (same file could be used by multiple readers), but not so much saving memory.

[CodeByNikolas]

Hi Mickael,

Thank you so much for this detailed insight! This is incredibly valuable validation for my thesis.

Regarding Shared JVM:
I understand. Having official confirmation that the Singleton Workspace/JavaModel is a hard architectural constraint saves me from chasing a dead end. I will pivot my implementation goals accordingly to focus on external optimization rather than internal architectural changes.

Regarding Shared Indexes:
The pointer to ClasspathEntry.getIndexLibraryLocation() is perfect. I will investigate how to inject a pre-built index.

You mentioned that this mostly saves CPU/Disk and not so much Memory because each instance loads a copy into the JavaModel.

• Follow-up Question: Does the JDT LS use Memory-Mapped Files (mmap) to access these index files? If so, the OS Page Cache might technically allow sharing the physical memory pages between processes, even if the JVM logical view is separate. Or does the JavaModel deserialize the index completely into Java Heap Objects (which would indeed prevent sharing)?

Regarding AppCDS (Question 3):
Since Shared Indexes might not solve the RAM bottleneck entirely:

• Do you have any experience or gut feeling about using AppCDS (Application Class Data Sharing) with JDT LS?
• Given the heavy use of OSGi/Equinox, is AppCDS even viable to reduce the memory footprint of the loaded classes across multiple containers?

Thanks again for your help!

[mickaelistria]

Does the JDT LS use Memory-Mapped Files (mmap) to access these index files?

Not as far as I know. The index is IIRC loaded fully in memory; and backed up for reuse upon restarts on filesystem. I don't think further optimizations are made; but that's worth investigating, a good topic to address in a bachelor thesis IMO.
Note that it is not really JDT LS, but just plain JDT here. The index, the Java Model and so on are reused from upstream https://github.com/eclipse-platform/eclipse.platform and https://github.com/eclipse-jdt/eclipse.jdt.core by JDT-LS without any modification. So you can address those topics without having to care about the LS part, potentially making analysis simpler as less technologies are involved.

An alternative to the explicit shared index, if everyone starts the same project, it could also be possible to create your container of JDT-LS so that the project is already loaded in it and processed at leaast once: jdt(-ls?) already started with this project and some basic operation already run ahead of actual usage, when creating the container. This choice should allow the Eclipse IDE to bootstrap many models (the workspace, the index, the build model, the java model...) and ensure they are stored on the filesystem to optimize next startup.
If you create a container where those models where already loaded, the startup should be significantly faster.

[fbricon]

Since you're using Theia, you can enable App CDS in vscode-java with "java.jdt.ls.appcds.enabled": "on"

see https://github.com/redhat-developer/vscode-java/blob/114980f83ef34f7085491a2c1350cfc86463836a/package.json#L425-L436