remove_non_existing_dirs tries to access system dir

[chris-aeviator]

When using bitsandbytes for huggingface (model = AutoModelForCausalLM.from_pretrained("bloom", device_map="auto", load_in_8bit=True)),

bitsandbytes tries to remove (as it seems) a system directory (!!)

  File "/home/myUser/myproject/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 166, in remove_non_existent_dirs
    if path.exists():
  File "/usr/lib/python3.10/pathlib.py", line 1290, in exists
    self.stat()
  File "/usr/lib/python3.10/pathlib.py", line 1097, in stat
    return self._accessor.stat(self, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied: '/var/lib/lightdm-data/myUser'

python 3.10
bitsandbytes 0.37.0
arch linux

EDIT: chmod 777 /var/lib/lightdm-data/myUser (an empty dir) will not resolve the issue - bitsandbytes still crashes my program

[chris-aeviator]

to reproduce, set an environment variable to a system dir (see #35 for more info)

SOME_ARBITRARY_ENVIRONMENT_VAR=/etc python myAppUsingBitsandbytes.py

[svgsponer]

Same Problem here. However not dependent on anything with hugging face. A simple import in the REPL is enough to reproduce the error.

Traceback (most recent call last):
  File "myApp.py", line 363, in inner_loop
    import bitsandbytes as bnb
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/__init__.py", line 6, in <module>
    from .autograd._functions import (
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/autograd/_functions.py", line 5, in <module>
    import bitsandbytes.functional as F
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/functional.py", line 13, in <module>
    from .cextension import COMPILED_WITH_CUDA, lib
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 113, in <module>
    lib = CUDASetup.get_instance().lib
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 109, in get_instance
    cls._instance.initialize()
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 59, in initialize
    binary_name, cudart_path, cuda, cc, cuda_version_string = evaluate_cuda_setup()
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 122, in evaluate_cuda_setup
    cudart_path = determine_cuda_runtime_lib_path()
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/paths.py", line 110, in determine_cuda_runtime_lib_path
    cuda_runtime_libs.update(find_cuda_lib_in(value))
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/paths.py", line 51, in find_cuda_lib_in
    resolve_paths_list(paths_list_candidate)
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/paths.py", line 46, in resolve_paths_list
    return remove_non_existent_dirs(extract_candidate_paths(paths_list_candidate))
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/paths.py", line 23, in remove_non_existent_dirs
    raise exc
  File "/myPath/venv/lib/python3.10/site-packages/bitsandbytes/cuda_setup/paths.py", line 19, in remove_non_existent_dirs
    if path.exists():
  File "/usr/local/lib/python3.10/pathlib.py", line 1288, in exists
    self.stat()
  File "/usr/local/lib/python3.10/pathlib.py", line 1095, in stat
    return self._accessor.stat(self, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied: '/var/lib/lightdm-data/myUser'

Python: 3.10.0
Bitsandbytes: 0.35.4
System: Arch

[muratcankurtulus]

this is still relevant for RHEL when load_in_4bit set to True using transformers.AutoModelForCausalLM.from_pretrained
this occurs while using JupyterLab.

I got the following:
PermissionError: [Errno 13] Permission denied: '/fs/applications/jupyterhub...

bitsandbytes==0.39.1
transformers==4.31.0.dev0

update:
I was able to solve this by hacking get_potentially_lib_path_containing_env_vars() in bitsandbytes/cuda_setup/env_vars.py and making sure that non of the values of the dictionary having "/fs" in it. but I do not think this is the way. I would like to add the relevant environment variable to ignore list but I couldn't figure out which one it is or if there is only one of them. I'll update if I can figure out a more robust solution to this.

[PierreColombo]

same here :/ cannot use it. due to permission error.

File "/gpfslocalsup/pub/anaconda-py3/2022.05/envs/pytorch-gpu-1.13.0+py3.10.8/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/gpfslocalsup/pub/anaconda-py3/2022.05/envs/pytorch-gpu-1.13.0+py3.10.8/lib/python3.10/runpy.py", line 146, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/gpfslocalsup/pub/anaconda-py3/2022.05/envs/pytorch-gpu-1.13.0+py3.10.8/lib/python3.10/runpy.py", line 110, in _get_module_details
import(pkg_name)
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/init.py", line 6, in
from . import cuda_setup, utils, research
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/research/init.py", line 1, in
from . import nn
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/research/nn/init.py", line 1, in
from .modules import LinearFP8Mixed, LinearFP8Global
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in
from bitsandbytes.optim import GlobalOptimManager
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/optim/init.py", line 6, in
from bitsandbytes.cextension import COMPILED_WITH_CUDA
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cextension.py", line 13, in
setup.run_cuda_setup()
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 120, in run_cuda_setup
binary_name, cudart_path, cc, cuda_version_string = evaluate_cuda_setup()
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 337, in evaluate_cuda_setup
cudart_path = determine_cuda_runtime_lib_path()
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 295, in determine_cuda_runtime_lib_path
cuda_runtime_libs.update(find_cuda_lib_in(value))
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 232, in find_cuda_lib_in
resolve_paths_list(paths_list_candidate)
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 227, in resolve_paths_list
return remove_non_existent_dirs(extract_candidate_paths(paths_list_candidate))
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 201, in remove_non_existent_dirs
raise exc
File "/linkhome/rech/genltc01/unm25jp/.local/lib/python3.10/site-packages/bitsandbytes/cuda_setup/main.py", line 197, in remove_non_existent_dirs
if path.exists():
File "/gpfslocalsup/pub/anaconda-py3/2022.05/envs/pytorch-gpu-1.13.0+py3.10.8/lib/python3.10/pathlib.py", line 1290, in exists
self.stat()
File "/gpfslocalsup/pub/anaconda-py3/2022.05/envs/pytorch-gpu-1.13.0+py3.10.8/lib/python3.10/pathlib.py", line 1097, in stat
return self._accessor.stat(self, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied: '/gpfslocalsys/idrzap/current/bin'

[AashayBhupendraD0shi]

I have the same issue. I've read at a few places that changing your file permissions (like role "storage admin" in GCP) will resolve the issue. The above commit resolves it only for arch linux. Is there any other way to resolve the issue on ubuntu?