您当前的位置: 首页 >  ubuntu
  • 2浏览

    0关注

    483博文

    0收益

  • 0浏览

    0点赞

    0打赏

    0留言

私信
关注
热门博文

Pytorch 在 Ubuntu18.04 上的编译、安装、与问题

高精度计算机视觉 发布时间:2019-01-16 16:49:44 ,浏览量:2

这里主要是develop开发版的编译与安装。如果不是开发版,安装是非常容易的,anaconda已经做得非常到位了,可参考该帖的后面部分,https://blog.csdn.net/tanmx219/article/details/82831964

在自己编译的情况下,和官网相比,我每次重新编译总是缺些什么,所以决定记录一下,

https://github.com/pytorch/pytorchhttps://m.oldpan.me/archives/pytorch-build-simple-instruction

目前是pytorch v1.0 ------------------------------------------------------------------------------------

conda install numpy pyyaml mkl mkl-include setuptools cmake cffi typing pybind11
conda install cudnn (cudatoolkit: 9.2-0+cudnn:7.2.1-cuda9.2_0)
conda install ninja (optional, warning: C++ will not be compiled incrementally)
conda install -c pytorch magma-cuda92 (optional, Add LAPACK support for the GPU if needed)
git clone --recursive https://github.com/pytorch/pytorch
cd pytorch

export CMAKE_PREFIX_PATH=${CONDA_PREFIX:-"$(dirname $(which conda))/../"}
如果不需要使用cuda的话,这里还要加上一句:export NO_CUDA=1

python setup.py install 
对于开发者模式,可以使用
python setup.py build develop

------------------------------------------------------------------------------------

源码安装的Pytorch,卸载需要执行:

pip uninstall torch
python setup.py clean

------------------------------------------------------------------------------------

注意git clone之前确保已经安装git,如果没有安装,sudo apt-get install git-all安装一下。

 

碰到的问题

Linux的问题总是千奇百怪,每次都不一样,也记录一下

=========================================================================

Problem 1:

---------------------------------------------------------------------------------------------------------------------------------

Cuda的安装,网上讲的安装方法非常多,其中包括一种比较有意思的,https://askubuntu.com/questions/1028830/how-do-i-install-cuda-on-ubuntu-18-04

-----how to install-----
sudo add-apt-repository ppa:graphics-drivers/ppa
sudo apt update
sudo ubuntu-drivers autoinstall
reboot
sudo apt install nvidia-cuda-toolkit gcc-6
nvcc --version

-----replace------
sudo sh cuda_9.1.85_387.26_linux.run (--override)
to uninstall the NVIDIA Driver, run nvidia-uninstall

-----how to remove-----
sudo apt-get remove --purge nvidia-*
sudo apt-get remove --purge libnvidia-*

这个方法是通过PPA proprietary graphics driver team 开发的包安装,可参考:https://launchpad.net/~graphics-drivers/+archive/ubuntu/ppa 这里有其团队的信息,https://launchpad.net/~graphics-drivers 不过写这东东时我也是第一次用这种方式安装,检查了一下,貌似通过这种形式安装的都是最新版本,还没看到过详细文档。 官网上提到: Current long-lived branch release: `nvidia-410` (410.66) Dropped support for Fermi series (https://nvidia.custhelp.com/app/answers/detail/a_id/4656) Old long-lived branch release: `nvidia-390` (390.87) 默认是nvidia-410, 总之,通过这种方式会安装很多包,这里装了多少多余的东西俺就不知道了。

--------------------

那假设我就是想安装nvidia-390怎么办呢?还是老办法保险: sudo apt-get install nvidia-driver-390 sudo apt-get install nvidia-driver-390-dev

=========================================================================

Problem 2:

---------------------------------------------------------------------------------------------------------------------------------

CMake Error at cmake/public/cuda.cmake:318 (message):   CUDA 9.1 is not compatible with std::tuple from GCC version >= 6.  Please   upgrade to CUDA 9.2 or use the following option to use another version (for   example):     -DCUDA_HOST_COMPILER=/usr/bin/gcc-5 ..... -- Configuring incomplete, errors occurred! See also "/home/matthew/dev/pytorch/build/CMakeFiles/CMakeOutput.log". See also "/home/matthew/dev/pytorch/build/CMakeFiles/CMakeError.log". Failed to run 'bash ../tools/build_pytorch_libs.sh --use-cuda --use-fbgemm --use-nnpack --use-mkldnn --use-qnnpack caffe2'

网上有不少解决办法,参考:https://github.com/pytorch/pytorch/issues/14152

Solution 1:

--------------------------------------------------------------------------------------------------------------------------------

版本不对,大概是先卸载再安装,参考:

step1: remove the previous cuda
sudo apt-get --purge remove nvidia-cuda-toolkit
conda remove cudnn (if has installed a different version needed)

step2: install cudnn  (this will install cudnn7_cuda9.2)
conda install cudnn

https://blog.csdn.net/tanmx219/article/details/86210023https://github.com/pytorch/pytorch/issues/14152

=========================================================================

Problem 3:

---------------------------------------------------------------------------------------------------------------------------------

$ python setup.py build develop Building wheel torch-1.1.0a0+964732f running build running build_deps setup.py::build_deps::run() + SYNC_COMMAND=cp ...... --     USE_GLOO_IBVERBS    : OFF --   Public Dependencies  : Threads::Threads;caffe2::mkl;caffe2::mkldnn --   Private Dependencies : qnnpack;nnpack;cpuinfo;fbgemm;fp16;gloo;aten_op_header_gen;onnxifi_loader;rt;gcc_s;gcc;dl -- Configuring done CMake Error in caffe2/CMakeLists.txt:   Imported target "caffe2::mkl" includes non-existent path     "/home/matthew/anaconda3/envs/torchdev/include"   in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:   * The path was deleted, renamed, or moved to another location.   * An install or uninstall procedure did not complete successfully.   * The installation package was faulty and references files it does not   provide.

-- Generating done -- Build files have been written to: /home/matthew/dev/pytorch/build Failed to run 'bash ../tools/build_pytorch_libs.sh --use-fbgemm --use-nnpack --use-mkldnn --use-qnnpack caffe2'

解决办法 ---------------------------------------------------------------------------------------------------------------------------------

这个问题是因为上一次编译没编译完,这次接着编译,但命令路径变了,所以必须清空重来 python setup.py clean python setup.py build develop

=========================================================================

Problem 3:

--------------------------------------------------------------------------------------------------------------------------------

The Nouveau kernel driver is currently in use by your system.  This driver is incompatible with the NVIDIA driver,

参考:https://tutorials.technology/tutorials/85-How-to-remove-Nouveau-kernel-driver-Nvidia-install-error.html How to remove Nouveau kernel driver (fix Nvidia install error)

sudo apt-get remove nvidia*
sudo apt autoremove
sudo apt-get install dkms build-essential linux-headers-generic
sudo vim /etc/modprobe.d/blacklist.conf

blacklist nouveau
blacklist lbm-nouveau
options nouveau modeset=0
alias nouveau off
alias lbm-nouveau off

echo options nouveau modeset=0 | sudo tee -a /etc/modprobe.d/nouveau-kms.conf
sudo update-initramfs -u
reboot

sudo apt-get --purge remove xserver-xorg-video-nouveau

=========================================================================

Problem 4:

--------------------------------------------------------------------------------------------------------------------------------

-- Caffe2: Cannot find glog automatically. Using legacy find. -- Could NOT find glog (missing: GLOG_INCLUDE_DIR GLOG_LIBRARY)  CMake Warning at cmake/public/glog.cmake:66 (message):   Caffe2: glog cannot be found.  Depending on whether you are building Caffe2   or a Caffe2 dependent library, the next warning / error will give you more   info. Call Stack (most recent call first):   cmake/Dependencies.cmake:291 (include)   CMakeLists.txt:219 (include)

===>>>

$ sudo apt-get install libgflags-dev    libgoogle-glog-dev

=========================================================================

Problem 5:

--------------------------------------------------------------------------------------------------------------------------------

-- Found CUDA: /usr/local/cuda (found version "10.0")  -- Caffe2: CUDA detected: 10.0 -- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc -- Caffe2: CUDA toolkit directory: /usr/local/cuda -- Caffe2: Header version is: 10.0 -- Could NOT find CUDNN (missing: CUDNN_INCLUDE_DIR CUDNN_LIBRARY)  CMake Warning at cmake/public/cuda.cmake:101 (message):   Caffe2: Cannot find cuDNN library.  Turning the option off Call Stack (most recent call first):   cmake/Dependencies.cmake:685 (include)   CMakeLists.txt:219 (include)

===>>> 

安装了cuda+cudnn后这个就会消失

 

 

 

 

 

关注
打赏
1661664439
查看更多评论
立即登录/注册

微信扫码登录

0.0443s