Fix missing folly iobuf

Summary:
Change our library loading strategy for multiprocessing children to fix the missing `folly.iobuf` library.

This is a challenging library loading problem that behaves differently when using the buck binary and the standalone PAR file.
A first fix for this missing library was introduced in D91807939, but only resolved library loading for the buck target and smoke tests. This fix, D95444857, resolves the standalone PAR loading.

1) Avoid fully re-importing `__main__`
2) Trigger this fix before every multiprocess child `Start` through `get_context`

Reviewed By: giorgidze, kavehahmadi60

Differential Revision: D95444857

fbshipit-source-id: 14eb06084d6704e6e641386cb338e3d6e51171eb

Author: SBones

eden/fs/cli/mp.py

@@ -9,10 +9,26 @@
 """
 Multiprocessing helpers for EdenFS CLI.
 
-Re-derives native library directories from sys.path (which still contains
-Buck2 link-tree paths even after the bootstrapper cleans LD_LIBRARY_PATH)
-and sets them as LD_LIBRARY_PATH / DYLD_LIBRARY_PATH in os.environ so that
-spawned child processes can find native .so/.dylib files like folly.iobuf.
+Fixes two problems that prevent multiprocessing 'spawn' children from
+working correctly in standalone PAR builds on macOS/Windows:
+
+1. __main__ re-import: multiprocessing.spawn re-imports the __main__
+   module in the child process. In a PAR, this triggers the full import
+   chain (main.py -> config.py -> thrift_clients -> folly.iobuf), which
+   fails because the child doesn't have the PAR bootstrapper's custom
+   import hooks and the PAR zip shadows the filesystem unpack directory
+   for packages like folly. We prevent this by clearing __main__.__spec__
+   and __file__ inside get_context() so the child skips the re-import
+   entirely — it only needs the target function (e.g. lstat_process),
+   not the full main module. This must be done inside get_context()
+   rather than at module level because enable_lazy_imports defers
+   module-level side effects.
+
+2. Native library paths: Re-derives native library directories from
+   sys.path (which still contains Buck2 link-tree paths even after the
+   bootstrapper cleans LD_LIBRARY_PATH) and sets them as
+   LD_LIBRARY_PATH / DYLD_LIBRARY_PATH in os.environ so that spawned
+   child processes can find native .so/.dylib files like folly.iobuf.
 """
 
 import multiprocessing
@@ -48,23 +64,61 @@ def _setup_library_paths() -> None:
     if not dirs:
         return
 
+    combined = os.pathsep.join(dirs)
+
     if sys.platform == "win32":
         add_dll_directory = getattr(os, "add_dll_directory", None)
         if add_dll_directory is not None:
             for d in dirs:
                 add_dll_directory(d)
     elif sys.platform == "darwin":
         existing = os.environ.get("DYLD_LIBRARY_PATH", "")
-        combined = os.pathsep.join(dirs)
         if existing:
-            combined = combined + os.pathsep + existing
-        os.environ["DYLD_LIBRARY_PATH"] = combined
+            os.environ["DYLD_LIBRARY_PATH"] = combined + os.pathsep + existing
+        else:
+            os.environ["DYLD_LIBRARY_PATH"] = combined
     else:
         existing = os.environ.get("LD_LIBRARY_PATH", "")
-        combined = os.pathsep.join(dirs)
         if existing:
-            combined = combined + os.pathsep + existing
-        os.environ["LD_LIBRARY_PATH"] = combined
+            os.environ["LD_LIBRARY_PATH"] = combined + os.pathsep + existing
+        else:
+            os.environ["LD_LIBRARY_PATH"] = combined
+
+
+def _prevent_main_reimport() -> None:
+    """
+    Prevent multiprocessing.spawn from re-importing __main__ in children.
+
+    In standalone PAR builds, the child process is a bare Python interpreter
+    without the PAR bootstrapper's custom import hooks. When the child tries
+    to re-import __main__ (eden.fs.cli.main), it triggers the full import
+    chain including native extensions like folly.iobuf. These fail because
+    the PAR zip claims packages like 'folly' via zipimport, but zipimport
+    cannot load .so extension modules — so folly.iobuf is never found
+    despite existing in the PAR's unpack directory.
+
+    By clearing __main__.__spec__ and __file__, multiprocessing.spawn's
+    get_preparation_data() won't include 'init_main_from_name' or
+    'init_main_from_path', and the child will skip the __main__ re-import
+    entirely. The child can still import the target function's module
+    (e.g. eden.fs.cli.mtab for lstat_process) without issues since those
+    modules don't trigger the problematic import chain.
+
+    We access __main__ via sys.modules to bypass the lazy import proxy
+    that enable_lazy_imports creates.
+    """
+    main_mod = sys.modules.get("__main__")
+    if main_mod is None:
+        return
+
+    if getattr(main_mod, "__spec__", None) is not None:
+        main_mod.__spec__ = None  # type: ignore[assignment]
+
+    if hasattr(main_mod, "__file__"):
+        try:
+            del main_mod.__file__
+        except AttributeError:
+            pass
 
 
 _setup_library_paths()
@@ -74,13 +128,16 @@ def get_context() -> multiprocessing.context.DefaultContext:
     """
     Return the platform-default multiprocessing context.
 
-    With _setup_library_paths() having propagated native library dirs into
-    the environment, spawned children can find native modules without
-    needing to force fork.
+    Clears __main__.__spec__ on each call to prevent multiprocessing.spawn
+    from re-importing __main__ in the child process. This must be done here
+    (not at module level) because enable_lazy_imports defers module-level
+    side effects, and we need the fix applied before Process.start()
+    captures __main__.__spec__ for the child.
 
     Returns:
         The default multiprocessing context for the current platform.
     """
+    _prevent_main_reimport()
     # pyre-ignore[7]: multiprocessing.get_context() is typed as BaseContext
     # but actually returns DefaultContext at runtime.
     return multiprocessing.get_context()