NVIDIAのGPUとドライバが入ったLinux (Debian) マシン上でrvis (ROS kinetic) をDockerを介して実行する
[注: 未解決]
環境
- Debian Stretch (GNU/Linux 4.9.0-8-amd64)
- 01:00.0 VGA compatible controller: NVIDIA Corporation GK208 [GeForce GT 710B] (rev a1)
問題
Dockerを介してrviz (おそらくopenGLを使うアプリケーション全て) をウィンドウを使いつつ表示させるにはいろいろ必要条件がある
やったこと
nvidia-dockerを使うようにした (1) (3)
基本的には (3) の通り。ROS用にDockerfileを書き換えた。
FROM ros:kinetic-ros-base
ENV NVIDIA_VISIBLE_DEVICES ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics
RUN apt update && apt install -y \
x11-apps \
mesa-utils \
gdb \
ros-kinetic-rviz
CMD ["rosrun", "rviz", "rviz"]
docker run --runtime=nvidia --rm -it --cap-add=SYS_PTRACE --security-opt="seccomp=unconfined" --net host -e QT_X11_NO_MITSHM=1 -e DISPLAY=$DISPLAY -v $HOME/.Xauthority:/root/.Xauthority -v /tmp/.X11-unix/:/tmp/.X11-unix/ ros-rviz /bin/bash
--cap-add=SYS_PTRACE --security-opt="seccomp=unconfined" はGDBによるデバッグに必要らしい（プログラムアドレスがセキュリティの関係でいじられるのを防ぎtrace出来るようにするらしい）。
-e QT_X11_NO_MITSHM=1 はX Error: BadAccess [...]が出るのでつけた (4)。
しかしnvidia-docker2を使い始めた途端コンテナがinitialization error: driver error: failed to process request\\\\n\\\"\"": unknown.で起動しない。
NVIDIAのドライバを入れ直した (2)
(2)を参考に、sudo nvidia-container-cli -k -d /dev/tty infoの結果が良くなかったのでホストのドライバが良くないと判断。以前はNVIDIAのページから得られる.runからインストールした。aptでnvidia-driversをinstall（推奨されません）。xserver-xorg-video-nvidiaがdependenciesを崩すというので手動で依存解決（aptするだけ）。いろいろ消えると言われるがこの頃は完全にしんどくなっていたのでとりあえずY。rebootしてXを起動すると入力機器が何も受け付けなくなった。(5)を参考にxserver-xorg-input-allのインストールで解決した。
コンテナは起動した。しかし
# ROS_MASTER_URI=http://xxx:11311 rviz [ INFO] [1549507151.920337704]: rviz version 1.12.16 [ INFO] [1549507151.920376105]: compiled against Qt version 5.5.1 [ INFO] [1549507151.920386034]: compiled against OGRE version 1.9.0 (Ghadamon) libGL error: No matching fbConfigs or visuals found libGL error: failed to load driver: swrast Could not initialize OpenGL for RasterGLSurface, reverting to RasterSurface. libGL error: No matching fbConfigs or visuals found libGL error: failed to load driver: swrast Segmentation fault (core dumped)
# ROS_MASTER_URI=http://163.143.94.197:11311 gdb rviz GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from rviz...(no debugging symbols found)...done. (gdb) r Starting program: /opt/ros/kinetic/bin/rviz [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffdda6f700 (LWP 85)] [ INFO] [1549507206.675844716]: rviz version 1.12.16 [ INFO] [1549507206.675877843]: compiled against Qt version 5.5.1 [ INFO] [1549507206.675886056]: compiled against OGRE version 1.9.0 (Ghadamon) [New Thread 0x7fffdce4e700 (LWP 86)] [New Thread 0x7fffd7fff700 (LWP 87)] [New Thread 0x7fffd77fe700 (LWP 88)] [New Thread 0x7fffd6ffd700 (LWP 89)] [New Thread 0x7fffc3fff700 (LWP 91)] [New Thread 0x7fffc37fe700 (LWP 92)] [New Thread 0x7fffc2ffd700 (LWP 93)] [New Thread 0x7fffc27fc700 (LWP 94)] libGL error: No matching fbConfigs or visuals found [Thread 0x7fffc27fc700 (LWP 94) exited] [Thread 0x7fffc2ffd700 (LWP 93) exited] [Thread 0x7fffc37fe700 (LWP 92) exited] [Thread 0x7fffc3fff700 (LWP 91) exited] libGL error: failed to load driver: swrast Could not initialize OpenGL for RasterGLSurface, reverting to RasterSurface. [New Thread 0x7fffc27fc700 (LWP 95)] [New Thread 0x7fffc2ffd700 (LWP 96)] [New Thread 0x7fffc37fe700 (LWP 97)] [New Thread 0x7fffc3fff700 (LWP 98)] libGL error: No matching fbConfigs or visuals found [Thread 0x7fffc3fff700 (LWP 98) exited] [Thread 0x7fffc37fe700 (LWP 97) exited] [Thread 0x7fffc2ffd700 (LWP 96) exited] [Thread 0x7fffc27fc700 (LWP 95) exited] libGL error: failed to load driver: swrast Thread 1 "rviz" received signal SIGSEGV, Segmentation fault. 0x00007ffff354a6b4 in glXCreateContext () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1 (gdb) bt #0 0x00007ffff354a6b4 in glXCreateContext () from /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1 #1 0x00007ffff7ad5600 in rviz::RenderSystem::setupDummyWindowId() () from /opt/ros/kinetic/lib/librviz.so #2 0x00007ffff7ad9712 in rviz::RenderSystem::RenderSystem() () from /opt/ros/kinetic/lib/librviz.so #3 0x00007ffff7ad9a05 in rviz::RenderSystem::get() () from /opt/ros/kinetic/lib/librviz.so #4 0x00007ffff7ad4dc1 in rviz::QtOgreRenderWindow::QtOgreRenderWindow(QWidget*) () from /opt/ros/kinetic/lib/librviz.so #5 0x00007ffff7afb949 in rviz::RenderPanel::RenderPanel(QWidget*) () from /opt/ros/kinetic/lib/librviz.so #6 0x00007ffff7b47160 in rviz::VisualizationFrame::initialize(QString const&) () from /opt/ros/kinetic/lib/librviz.so #7 0x00007ffff7b52509 in rviz::VisualizerApp::init(int, char**) () from /opt/ros/kinetic/lib/librviz.so #8 0x0000000000400c90 in main ()
NVIDIA DriverをDocker Imageにもインストール
ここでindigoのイメージだとちゃんとrvizがうごくことを確認する。
kineticの方で、GDBの結果とldconfig -p | grep -i gl.soの結果から、mesaのlibGL.soが良くないと思い綺麗なubuntuイメージにnvidia drivers nvidia-384 をインストール (mesaとかOpenGLとかNvidiaとかの関係は(6)を読むとわかりそうだがまだちゃんと読んでない）。
Dockerfile
FROM ubuntu:16.04
ENV NVIDIA_VISIBLE_DEVICES ${NVIDIA_VISIBLE_DEVICES:-all}
ENV NVIDIA_DRIVER_CAPABILITIES ${NVIDIA_DRIVER_CAPABILITIES:+$NVIDIA_DRIVER_CAPABILITIES,}graphics
RUN apt update && apt install -y expect
COPY nvidia-expect.exp /
RUN expect /nvidia-expect.exp
RUN sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list'
RUN apt-key adv --keyserver hkp://ha.pool.sks-keyservers.net:80 --recv-key 421C365BD9FF1F717815A3895523BAEEB01FA116
RUN apt update
RUN apt-get install ros-kinetic-desktop-full -y
CMD ["rosrun", "rviz", "rviz"]
nvidia-expect.exp
#!/usr/bin/env expect
set timeout -1
spawn apt-get install -y nvidia-384
expect {
"\\\[More\\\]" { send "\n"; exp_continue; }
"Country of origin for the keyboard: " { send "29\n"; }
}
expect {
"\\\[More\\\]" { send "\n"; exp_continue; }
"Keyboard layout: " { send "1\n" }
}
expect eof
これで、 docker runしrvizを実行する。
# ROS_MASTER_URI=http://xxx:11311 rviz [ INFO] [1549507833.199991200]: rviz version 1.12.16 [ INFO] [1549507833.200033889]: compiled against Qt version 5.5.1 [ INFO] [1549507833.200045286]: compiled against OGRE version 1.9.0 (Ghadamon) [...] [ WARN] [1549507834.409556933]: OGRE EXCEPTION(2:InvalidParametersException): Window with name 'OgreWindow(1)' already exists in GLRenderSystem::_createRenderWindow at /build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/RenderSystems/GL/src/OgreGLRenderSystem.cpp (line 1057) rviz::RenderSystem: error creating render window: OGRE EXCEPTION(2:InvalidParametersException): Window with name 'OgreWindow(1)' already exists in GLRenderSystem::_createRenderWindow at /build/ogre-1.9-mqY1wq/ogre-1.9-1.9.0+dfsg1/RenderSystems/GL/src/OgreGLRenderSystem.cpp (line1057) [ERROR] [1549507834.409586881]: Unable to create the rendering window after 100 tries. Segmentation fault (core dumped)
(gdb) bt #0 0x00007ffff7ad4e85 in rviz::QtOgreRenderWindow::QtOgreRenderWindow(QWidget*) () from /opt/ros/kinetic/lib/librviz.so #1 0x00007ffff7afb949 in rviz::RenderPanel::RenderPanel(QWidget*) () from /opt/ros/kinetic/lib/librviz.so #2 0x00007ffff7b47160 in rviz::VisualizationFrame::initialize(QString const&) () from /opt/ros/kinetic/lib/librviz.so #3 0x00007ffff7b52509 in rviz::VisualizerApp::init(int, char**) () from /opt/ros/kinetic/lib/librviz.so #4 0x0000000000400c90 in main ()
ちなみにここでglxgearsすると正常に動いているっぽい（fpsがstdoutに出てる）が描画が正常じゃない（真っ黒）。
TODO
docker/Tutorials/GUI - ROS Wiki のSimple Way以外を試す
感想
わからん
References
(2):
(1):
(3):
(4): github.com
(5): askubuntu.com
(6):
