When I execute the code out of the singularity (/rigel/opt/tshark/tshark_python3.sif), everything is correct.
However, when I execute it in singularity, it always shows me that libcublas.so.10.0: cannot open shared object file in singularity, even though it exists in the directory (/cm/shared/apps/cuda10.0/toolkit/10.0.130/targets/x86_64-linux/lib/libcublas.so.10.0).
$singularity exec --nv /rigel/opt/tshark/tshark_python3.sif python
>import tensorflow as tf
>tf.test.is_gpu_available()
Even I add "the directory path", the problem still exists.
$env SINGULARITYENV_APPEND_PATH=/cm/shared/apps/cuda10.0/toolkit/10.0.130/targets/x86_64-linux/lib/ SINGULARITY_ENV_LD_LIBRARY_PATH=$LD_LIBRARY_PATH singularity exec --nv /rigel/opt/tshark/tshark_python3.sif python
After several days attempting, I found the reason was that: in the singularity, the directory (/cm/shared/apps/) contains ‘libcublas.so.10.0‘ cann‘t be seen in the singularity.
Knowing the reason, it‘s easy to solve.
Solution: just using --bind.
> singularity exec --nv --bind /cm/shared/apps /rigel/opt/tshark/tshark_python3.sif python
>import tensorflow as tf
>tf.test.is_gpu_available()
Error: pip install
ERROR: Could not install packages due to an EnvironmentError: [Errno 28] No space left on device
TMPDIR=./pip_cache pip install --upgrade tensorflow_gpu==2.0.0 --user
http://naoko.github.io/pip-install-no-space-left/
原文：https://www.cnblogs.com/quinn-yann/p/12250977.html