The LLVM development team has presented a proposal called Multi-Thread Parallel Compilation (MTPC) to improve ThinLTO. Currently, this technique parallelizes entire modules, but large modules are processed serially, creating bottlenecks. MTPC aims to break that barrier by dividing work at the function level.
How intra-module parallelism works in ThinLTO 🚀
MTPC introduces the ability to split a large module into smaller units based on functions. Each function can be compiled and optimized in an independent thread, as long as data dependencies and global references are properly managed. This allows multi-core processors to work simultaneously within the same module, reducing compilation time in projects with very large source files.
The drama of modules that thought they were untouchable 😅
Until now, large modules enjoyed a quiet life: all the code was compiled serially, like a line at the supermarket. With MTPC, those lazy functions will have to start working in parallel, as if a boss told them the coffee was gone. Developers will be able to say goodbye to those breaks while waiting for the compiler to finish, although surely some module will go on strike due to lack of synchronization.