I’m trying to run the docker image of Huggingface inference using apptainer. The issue is that i could mount my local directory and the models can be downloaded from huggingface but when the server starts, there is a permission denied error.

Apptainer command:

apptainer run --nv --env HF_TOKEN=$HF_TOKEN --env RUST_BACKTRACE=full --bind $volume:/data text-generation-inference_2.2.0.sif --model-id $model

Error:

thread 'main' panicked at /usr/src/router/src/server.rs:1910:67:
called `Result::unwrap()` on an `Err` value: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }
stack backtrace:
   0:     0x55eda601389f - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h1e1a1972118942ad
   1:     0x55eda5b5b71b - core::fmt::write::hc090a2ffd6b28c4a
   2:     0x55eda5fde1c2 - std::io::Write::write_fmt::h8898bac6ff039a23
   3:     0x55eda6015389 - std::sys_common::backtrace::print::ha96650907276675e
   4:     0x55eda6014bae - std::panicking::default_hook::{{closure}}::h215c2a0a8346e0e0
   5:     0x55eda6015dd4 - std::panicking::rust_panic_with_hook::hac8bdceee1e4fe2c
   6:     0x55eda60156f2 - std::panicking::begin_panic_handler::{{closure}}::h00d785e82757ce3c
   7:     0x55eda6015649 - std::sys_common::backtrace::__rust_end_short_backtrace::h1628d957bcd06996
   8:     0x55eda6015636 - rust_begin_unwind
   9:     0x55eda5730bf2 - core::panicking::panic_fmt::hdc63834ffaaefae5
  10:     0x55eda5731045 - core::result::unwrap_failed::h82b551e0ff2b2176
  11:     0x55eda589475c - text_generation_router::server::run::{{closure}}::h1de6c6135029d307
  12:     0x55eda5aca7d6 - text_generation_router::main::{{closure}}::hb9f6c1fd1a653473
  13:     0x55eda5abf8ab - text_generation_router::main::h48ab4f33024a7b20
  14:     0x55eda5919c63 - std::sys_common::backtrace::__rust_begin_short_backtrace::h43b47338ee716213
  15:     0x55eda591a5ce - std::rt::lang_start::{{closure}}::h2a82aaad38d0ecd2
  16:     0x55eda5ad995f - main
  17:     0x1511480b5d90 - <unknown>
  18:     0x1511480b5e40 - __libc_start_main
  19:     0x55eda5771931 - _start
  20:                0x0 - <unknown>
2024-08-14T12:54:07.064654Z ERROR text_generation_launcher: Webserver Crashed

I tried using --writable-tmpfs and --fakeroot but that doesn’t seem to help.