feat(compilerlib): give each compilation a working directory of its own

.github/workflows/build.yml

@@ -186,6 +186,44 @@ jobs:
           ${{ runner.temp }}/ct_autofree_tests/*.log
           ${{ runner.temp }}/ct_runtime_tests/*.log
 
+  # compilerlib and its unit tests built with ThreadSanitizer: compilations from several threads
+  # at once, each in a working directory of its own, race on none of compilerlib's state. LLVM
+  # and Clang come from packages built without ThreadSanitizer: a race between two
+  # compilations inside their code goes unseen.
+  tsan-compilerlib:
+    name: compilerlib with ThreadSanitizer (ubuntu-24.04, LLVM 20)
+    runs-on: ubuntu-24.04
+    env:
+      TSAN_OPTIONS: halt_on_error=1:exitcode=66
+
+    steps:
+    - uses: actions/checkout@v4
+
+    - name: Install dependencies
+      run: |
+        sudo apt-get update
+        sudo apt-get install -y build-essential cmake ninja-build
+        sudo bash scripts/ci/install-llvm-apt.sh 20 llvm-20 llvm-20-dev clang-20 libclang-20-dev \
+          libclang-rt-20-dev
+        # The kernel randomizes mmap addresses over more bits than ThreadSanitizer supports.
+        sudo sysctl vm.mmap_rnd_bits=28
+
+    - name: Build
+      run: |
+        cmake -S . -B build -G Ninja \
+          -DCMAKE_C_COMPILER=clang-20 -DCMAKE_CXX_COMPILER=clang++-20 \
+          "-DCMAKE_C_FLAGS=-fsanitize=thread -g" "-DCMAKE_CXX_FLAGS=-fsanitize=thread -g" \
+          -DCMAKE_EXE_LINKER_FLAGS=-fsanitize=thread \
+          -DLLVM_DIR=/usr/lib/llvm-20/lib/cmake/llvm \
+          -DClang_DIR=/usr/lib/llvm-20/lib/cmake/clang \
+          -DUSE_SHARED_LIB=OFF
+        cmake --build build --target compilerlib_unit_tests
+
+    - name: Run the compilations with a working directory, concurrent ones included
+      run: |
+        build/test/unit/compilerlib_unit_tests \
+          --gtest_filter='*WorkingDirectory*:*ConcurrentCompilationTest*'
+
   # Line coverage of compilerlib, cc and the runtime (ENABLE_COVERAGE), from the unit tests,
   # the smoke tests and the runtime fixture suites. Informational: no threshold is enforced.
   # The suites build their programs with -fprofile-instr-generate, so that the runtime they

CHANGELOG.md

@@ -14,6 +14,23 @@ pull request.
 
 ## Unreleased
 
+### Features
+
+- **Compilations from several threads.** `compilerlib::compile` may run from several threads at
+  once: each compilation reads and writes files through a file system of its own, whose working
+  directory `-working-directory <dir>` moves while the process's stays. A host compiles each
+  command of a compilation database in its directory that way, instead of moving the process
+  under a lock. A compilation with a job that runs as a program, such as a link, and one that
+  passes `-mllvm`, `-mdebug-pass`, `-mlimit-float-precision` or `-ftime-report` must not run
+  while another one does (#262, #261).
+
+### Behaviour changes
+
+- **`-working-directory` leaves the process where it is** when every job of the compilation runs
+  in the process: a compilation to a file or to memory, instrumented or not. The process was
+  left in `<dir>`. A compilation that links or assembles still moves the process to `<dir>`, as
+  clang does (#262, #261).
+
 ### Fixes
 
 - **A child of `fork()` and the vtable diagnostics, on Linux.** With `--ct-vtable-diag`, a
@@ -119,6 +136,12 @@ pull request.
 - **`CT_DEBUG_AUTOFREE` read from several threads.** The allocation pass read the variable into
   two plain statics on its first use, on which two threads instrumenting modules at once raced.
   It is now read once, into a constant (#251).
+- **A relative `-working-directory`** names a directory relative to the process's working
+  directory. The compilation failed: the `-cc1` job resolved it again from the directory the
+  driver had moved to (#262, #261).
+- **Objects linked alone with `-working-directory`** are read in that directory, to choose
+  between the C and C++ drivers. They were looked up in the process's working directory, and
+  the link failed with `failed to inspect object` (#262, #261).
 
 ### Performance
 
@@ -177,6 +200,10 @@ pull request.
   Ubuntu 26.04, where clang-format 19 ran and apt.llvm.org has no LLVM 20. `scripts/format.sh`
   and `scripts/format-check.sh` run the binary that `CLANG_FORMAT` names and print its version
   (#238, #237).
+- A job of the Build workflow builds compilerlib and its unit tests with ThreadSanitizer, and
+  runs the tests of `-working-directory`, eight threads compiling at once included. LLVM and
+  Clang come from packages built without ThreadSanitizer, so a race inside their code goes
+  unseen (#262, #261).
 - When `sbrk()` fails in the sbrk test of the full allocation table, the test reports `errno`,
   the program break and the mapping above it, `RLIMIT_DATA`, and the memory of the process and
   of the system (#239, #233).
@@ -211,6 +238,8 @@ pull request.
 - `NOTICE` dates the copyright from 2025, when the code began, credits the CoreTrace
   contributors alongside Hugo Payet and links the repository. It keeps only these
   attribution notices, without the license header meant for source files (#248).
+- The README and `compilerlib/compiler.h` describe compiling from several threads, with
+  `-working-directory`, and its limits (#262, #261).
 
 ## v0.12.0 (2026-10-22)
 

README.md

@@ -498,3 +498,22 @@ expects, and waits for it: a host can compile from a thread with a small stack,
 secondary thread on macOS, whose stack is 512 KiB (#204). The time-trace profiler a host runs on
 its own thread, started with `llvm::timeTraceProfilerInitialize`, therefore does not record the
 compilation. With `-ftime-trace`, each job writes its own trace file, as clang does.
+
+### Compiling from several threads
+
+`compilerlib::compile` may run from several threads at once, as for a host that compiles each
+command of a compilation database. Each compilation reads and writes files through a file system
+of its own, whose working directory `-working-directory <dir>` sets without moving the process:
+the sources, the include directories and the files the compilation writes, its output, its
+dependency file and the others, are relative to `<dir>`, and a relative `<dir>` is relative to the
+process's working directory. A host passes each command's directory that way, instead of changing
+the process's working directory under a lock.
+
+Two kinds of compilations must not run while another one does:
+
+- **Jobs that run as programs.** The linker, the assembler and, with `-fno-integrated-cc1`, every
+  job run as programs of their own, which start in the process's working directory. With
+  `-working-directory`, such a compilation moves the whole process to `<dir>`, as clang does, and
+  leaves it there.
+- **Options of LLVM's own.** Clang applies `-mllvm`, `-mdebug-pass`, `-mlimit-float-precision`
+  and `-ftime-report` to LLVM's state, which the whole process shares.

include/compilerlib/compiler.h

@@ -30,6 +30,18 @@ namespace compilerlib
     };
 
     // std::pair<bool, std::string> compile(const std::vector<std::string>& args);
+    /// Compiles as clang does with `args`, in this process.
+    ///
+    /// `-working-directory <dir>` gives the compilation a working directory of its own: the
+    /// sources, the include directories and the files it writes are relative to `<dir>`, and a
+    /// relative `<dir>` to the process's working directory, which stays where it is. A
+    /// compilation with a job that runs as a program of its own, as linking and assembling do,
+    /// moves the whole process to `<dir>` instead, as clang does, and leaves it there.
+    ///
+    /// Compilations may run from several threads at once. A compilation that moves the process
+    /// must not run while another compilation of the process does, nor one that passes an
+    /// option that Clang applies to LLVM's state of the whole process: `-mllvm`, `-mdebug-pass`,
+    /// `-mlimit-float-precision` or `-ftime-report`.
     CompileResult compile(const std::vector<std::string>& args,
                           OutputMode mode = OutputMode::ToFile, bool instrument = false);
 

src/compilerlib/args.cpp

@@ -2,10 +2,16 @@
 #include "args_internal.hpp"
 #include "clang_driver_compat.hpp"
 
+#include <llvm/ADT/STLExtras.h>
+#include <llvm/ADT/SmallString.h>
 #include <llvm/Config/llvm-config.h>
 #include <llvm/Option/ArgList.h>
+#include <llvm/Support/Path.h>
+#include <llvm/Support/VirtualFileSystem.h>
 
 #include <cstring>
+#include <map>
+#include <utility>
 
 namespace compilerlib
 {
@@ -107,6 +113,157 @@ namespace compilerlib
         args.swap(out);
     }
 
+    std::string normalizeWorkingDirectoryArgs(std::vector<std::string>& args,
+                                              llvm::vfs::FileSystem& fs)
+    {
+        std::vector<const char*> argv;
+        argv.reserve(args.size());
+        for (const auto& arg : args)
+            argv.push_back(arg.c_str());
+        unsigned missingIndex = 0;
+        unsigned missingCount = 0;
+        // The driver's own parse: the value of another option, or an input after "--", is
+        // not an option.
+        const llvm::opt::InputArgList parsed =
+            clang_compat::driverOptTable().ParseArgs(argv, missingIndex, missingCount);
+
+        // Index of each -working-directory, either spelling, and its absolute directory.
+        std::map<unsigned, std::string> directories;
+        std::string last;
+        for (const llvm::opt::Arg* arg :
+             parsed.filtered(clang_compat::options::OPT_working_directory))
+        {
+            llvm::SmallString<256> directory(arg->getValue());
+            (void)fs.makeAbsolute(directory);
+            last = directory.str().str();
+            directories.emplace(arg->getIndex(), last);
+        }
+        if (directories.empty())
+            return {};
+
+        std::vector<std::string> out;
+        out.reserve(args.size() + directories.size());
+        for (unsigned i = 0; i < args.size(); ++i)
+        {
+            auto found = directories.find(i);
+            if (found == directories.end())
+            {
+                out.push_back(std::move(args[i]));
+                continue;
+            }
+            // The separate spelling has its directory in the next argument.
+            if (args[i] == "-working-directory")
+                ++i;
+            out.push_back("-working-directory");
+            out.push_back(std::move(found->second));
+        }
+        args.swap(out);
+        return last;
+    }
+
+    llvm::opt::ArgStringList
+    resolveCc1OutputPaths(const llvm::opt::ArgStringList& args, llvm::StringRef directory,
+                          llvm::function_ref<const char*(llvm::StringRef)> save)
+    {
+        // The -cc1 spellings the driver gives these options, LLVM 16 to 23.
+        static constexpr llvm::StringLiteral kSeparate[] = {
+            "-o",
+            "-dependency-file",
+            "-header-include-file",
+            "-opt-record-file",
+            "-split-dwarf-output",
+            "-stack-usage-file",
+            "-serialize-diagnostic-file",
+            "-dependency-dot",
+        };
+        static constexpr llvm::StringLiteral kJoined[] = {
+            "-ftime-trace=", "-stats-file=", "-fmodules-cache-path="};
+
+        // nullptr when `path` needs no change: absolute, or "-" for a standard stream. A
+        // relative path is appended to the directory, as clang's FileManager does.
+        auto absolute = [&](llvm::StringRef prefix, llvm::StringRef path) -> const char*
+        {
+            if (path.empty() || path == "-" || llvm::sys::path::is_absolute(path))
+                return nullptr;
+            llvm::SmallString<256> resolved(directory);
+            llvm::sys::path::append(resolved, path);
+            std::string argument = prefix.str();
+            argument += resolved.str();
+            return save(argument);
+        };
+
+        llvm::opt::ArgStringList out;
+        out.reserve(args.size());
+        for (size_t i = 0; i < args.size(); ++i)
+        {
+            const llvm::StringRef arg = args[i];
+            out.push_back(args[i]);
+            if (llvm::is_contained(kSeparate, arg) && i + 1 < args.size())
+            {
+                ++i;
+                const char* resolved = absolute("", args[i]);
+                out.push_back(resolved ? resolved : args[i]);
+                continue;
+            }
+            for (llvm::StringRef prefix : kJoined)
+            {
+                if (!arg.starts_with(prefix))
+                    continue;
+                if (const char* resolved = absolute(prefix, arg.drop_front(prefix.size())))
+                    out.back() = resolved;
+                break;
+            }
+        }
+        return out;
+    }
+
+    void resolveDriverOutputPaths(std::vector<std::string>& args, llvm::StringRef directory)
+    {
+        std::vector<const char*> argv;
+        argv.reserve(args.size());
+        for (const auto& arg : args)
+            argv.push_back(arg.c_str());
+        unsigned missingIndex = 0;
+        unsigned missingCount = 0;
+        // The driver's own parse, as for -working-directory.
+        const llvm::opt::InputArgList parsed =
+            clang_compat::driverOptTable().ParseArgs(argv, missingIndex, missingCount);
+
+        // Index of each option, either spelling, with its spelling and its absolute path.
+        std::map<unsigned, std::pair<std::string, std::string>> resolved;
+        for (const llvm::opt::Arg* arg : parsed.filtered(
+                 clang_compat::options::OPT_MJ, clang_compat::options::OPT_gen_cdb_fragment_path))
+        {
+            const llvm::StringRef value = arg->getValue();
+            if (value.empty() || llvm::sys::path::is_absolute(value))
+                continue;
+            llvm::SmallString<256> path(directory);
+            llvm::sys::path::append(path, value);
+            resolved.emplace(arg->getIndex(),
+                             std::make_pair(arg->getSpelling().str(), path.str().str()));
+        }
+        if (resolved.empty())
+            return;
+
+        std::vector<std::string> out;
+        out.reserve(args.size() + resolved.size());
+        for (unsigned i = 0; i < args.size(); ++i)
+        {
+            auto found = resolved.find(i);
+            if (found == resolved.end())
+            {
+                out.push_back(std::move(args[i]));
+                continue;
+            }
+            // The separate spelling has its path in the next argument.
+            if (args[i] == found->second.first)
+                ++i;
+            out.push_back(std::move(found->second.first));
+            out.push_back(std::move(found->second.second));
+        }
+        args.swap(out);
+    }
+
     void appendDiagnostics(std::string& out, const std::string& extra)
     {
         if (extra.empty())

src/compilerlib/args_internal.hpp

@@ -7,13 +7,19 @@
 
 #include "compilerlib/attributes.hpp"
 
+#include <llvm/ADT/STLFunctionalExtras.h>
 #include <llvm/ADT/StringRef.h>
 #include <llvm/Option/Option.h>
 #include <llvm/TargetParser/Triple.h>
 
 #include <string>
 #include <vector>
 
+namespace llvm::vfs
+{
+    class FileSystem;
+} // namespace llvm::vfs
+
 namespace compilerlib
 {
     // Value of `opt` given either as a separate argument (`-o out`) or joined with
@@ -36,6 +42,29 @@ namespace compilerlib
     // Rewrites the `-o=path` and `-x=lang` spellings into the two-argument forms.
     void normalizeEqualsArgs(std::vector<std::string>& args);
 
+    // Rewrites each -working-directory of a driver command line, `-working-directory <dir>` or
+    // `-working-directory=<dir>`, into the two-argument form, with <dir> made absolute by `fs`.
+    // Returns the directory of the last one, which the driver uses; empty without one.
+    CT_NODISCARD std::string normalizeWorkingDirectoryArgs(std::vector<std::string>& args,
+                                                           llvm::vfs::FileSystem& fs);
+
+    // The arguments of a -cc1 job, with the relative path of each file it writes made absolute
+    // against `directory`: its output, dependency file and graph, header list, optimization
+    // record, split DWARF, stack usage, serialized diagnostics, time trace, statistics and
+    // module cache. A job resolves the files it reads, and its output, against its
+    // -working-directory, but the other files it writes against the process's working
+    // directory, which clang itself moves to that directory first. `save` keeps a new argument
+    // alive as long as the job.
+    CT_NODISCARD llvm::opt::ArgStringList
+    resolveCc1OutputPaths(const llvm::opt::ArgStringList& args, llvm::StringRef directory,
+                          llvm::function_ref<const char*(llvm::StringRef)> save);
+
+    // Rewrites each option of a driver command line that names what the driver itself writes,
+    // `-MJ <file>` and `-gen-cdb-fragment-path <directory>`, into the two-argument form, with a
+    // relative path made absolute against `directory`. The driver writes them relative to the
+    // process's working directory, which clang itself moves to that directory first.
+    void resolveDriverOutputPaths(std::vector<std::string>& args, llvm::StringRef directory);
+
     // Appends `extra` to `out` with exactly one newline between them; no-op when empty.
     void appendDiagnostics(std::string& out, const std::string& extra);