文章目录前言一、环境搭建1.1 CUDA11.7安装1.2 虚拟环境构建二、数据集下载及预处理2.1 数据集下载2.1.1 TUM RGB-D数据集2.1.2 BONN动态RGB-D数据集2.2 数据预处理2.2.1 Semantic motion mask生成三、运行前言主要用于自己搭建环境。我采用CUDA11.71.13.1来构建虚拟环境代码地址DG-SLAM最后结果非常差不知是不是我的操作问题欢迎各位留言讨论一、环境搭建下载项目文件并进入文件夹gitclone https://github.com/fudan-zvg/DG-SLAM.gitcdDG-SLAM1.1 CUDA11.7安装如何安装CUDA可参考Unbuntu22.04中安装多版本的CUDA可任意切换版本1.2 虚拟环境构建DG-SLAM依赖安装将environment.yaml改为name: dg-slam channels: - nvidia - pytorch - conda-forge - defaults dependencies: -_libgcc_mutex0.1main -_openmp_mutex5.11_gnu -blas1.0mkl -bzip21.0.8h7b6447c_0 - ca-certificates2022.10.11h06a4308_0 - charset-normalizer2.0.4pyhd3eb1b0_0 -cudatoolkit11.7-ffmpeg4.3hf484d3e_0 -fftw3.3.9h27cfd23_1 -freetype2.12.1h4a9f257_0 -giflib5.2.1h7b6447c_0 -gmp6.2.1h295c915_3 -gnutls3.6.15he1e5248_0 - intel-openmp2021.4.0h06a4308_3561 -jpeg9eh7f8727e_0 -lame3.100h7b6447c_0 -lcms22.12h3be6417_0 - ld_impl_linux-642.38h1181459_1 -lerc3.0h295c915_0 -libdeflate1.8h7f8727e_5 -libffi3.3he6710b0_2 - libgcc-ng11.2.0h1234567_1 - libgfortran-ng11.2.0h00389a5_1 -libgfortran511.2.0h1234567_1 -libgomp11.2.0h1234567_1 -libiconv1.16h7f8727e_2 -libidn22.3.2h7f8727e_0 -libpng1.6.37hbc83047_0 - libstdcxx-ng11.2.0h1234567_1 -libtasn14.16.0h27cfd23_0 -libtiff4.4.0hecacb30_0 -libunistring0.9.10h27cfd23_0 -libuuid1.0.3h7f8727e_2 -libwebp1.2.4h11a3e52_0 - libwebp-base1.2.4h5eee18b_0 - lz4-c1.9.3h295c915_1 -mkl2021.4.0h06a4308_640 -mkl_fft1.3.1py310hd6ae3a3_0 -mkl_random1.2.2py310h00e6091_0 -ncurses6.3h5eee18b_3 -nettle3.7.3hbbd107a_1 - numpy-base1.23.3py310h8e6c178_1 -openh2642.1.1h4ff587b_0 -openssl1.1.1qh7f8727e_0 -pycparser2.21pyhd3eb1b0_0 -pyopenssl22.0.0pyhd3eb1b0_0 -python3.10.6haa1d7c7_1 - pytorch-mutex1.0cuda -readline8.2h5eee18b_0 -six1.16.0pyhd3eb1b0_1 -sqlite3.39.3h5082296_0 -tk8.6.12h1ccaba5_0 -typing_extensions4.3.0py310h06a4308_0 -wheel0.37.1pyhd3eb1b0_0 -xz5.2.6h5eee18b_0 -zlib1.2.13h5eee18b_0 -zstd1.5.2ha4553b6_0 - pip - pip: - --extra-index-url https://download.pytorch.org/whl/cu117 -torch1.13.0cu117 -torchvision0.14.0cu117 -torchaudio0.13.0 - torch-fidelity0.3.0 -torchmetrics0.11.1 -addict2.4.0 - antlr4-python3-runtime4.9.3 -appdirs1.4.4 -asttokens2.4.1 -attrs22.1.0 -autopep82.0.0 -brotli1.0.9 -brotlipy0.7.0 -certifi2022.9.24 -cffi1.15.1 -click8.1.3 -colorama0.4.6 -comm0.2.1 -configargparse1.5.3 -contourpy1.0.6 -cryptography38.0.1 -cycler0.11.0 -dash2.6.2 - dash-core-components2.0.0 - dash-html-components2.0.0 - dash-table5.0.0 -decorator5.1.1 - docker-pycreds0.4.0 -einops0.5.0 -exceptiongroup1.2.0 -executing2.0.1 - faiss-gpu1.7.2 -fastjsonschema2.16.2 -flask2.2.2 - flask-compress1.13-fonttools4.38.0 -gitdb4.0.9 -gitpython3.1.29 - hydra-core1.3.2 -idna3.4-imageio2.22.2 -ipython8.19.0 -ipywidgets8.1.1 -itsdangerous2.1.2 -jedi0.19.1 -jinja23.1.2 -joblib1.2.0 -jsonschema4.16.0 - jupyter-core4.11.2 - jupyterlab-widgets3.0.9 -kiwisolver1.4.4 -lpips0.1.4 -markupsafe2.1.1 -mathutils3.3.0 -matplotlib3.4.3 - matplotlib-inline0.1.6 - mkl-service2.4.0 -nbformat5.5.0 -networkx2.8.7 -nibabel5.2.1 -numpy1.24.1 -omegaconf2.3.0 -open3d0.16.0 - opencv-python4.6.0.66 -packaging21.3-pandas2.1.4 -parso0.8.3 -pathtools0.1.2 -pexpect4.9.0 -pillow9.3.0 -plotly5.11.0 -promise2.3- prompt-toolkit3.0.43 -protobuf4.21.9 -psutil5.9.4 -ptyprocess0.7.0 - pure-eval0.2.2 -pycodestyle2.9.1 -pygments2.17.2 -pyparsing3.0.9 -pyquaternion0.9.9 -pyrsistent0.18.1 -pysocks1.7.1 - python-dateutil2.8.2 - pytorch-msssim0.2.1 -pytz2023.3.post1 -pywavelets1.4.1 -pyyaml6.0-requests2.28.1 - scikit-image0.19.3 - scikit-learn1.1.3 -scipy1.9.1 - sentry-sdk1.10.1 -setproctitle1.3.2 -setuptools65.4.0 -shortuuid1.0.10 -smmap5.0.0 -snakeviz2.1.1 - stack-data0.6.3 -tenacity8.1.0 -threadpoolctl3.1.0 -tifffile2022.10.10 -tomli2.0.1 -tornado6.2-tqdm4.64.1 -traitlets5.5.0 -trimesh3.15.8 - typing-extensions4.3.0 -tzdata2023.4-urllib31.26.12 -wandb0.13.9 -wcwidth0.2.13 -werkzeug2.2.2 -widgetsnbextension4.0.9构建环境这里会很慢亲耐心等待condaenvcreate-fenvironment.yaml conda activate dg-slamOneFormer依赖安装后面Semantic motion mask生成需要用到OneFormer所以将它的依赖也装在dg-slam中。进入submodules下载OneFormer项目文件cdsubmodulesgitclone https://github.com/SHI-Labs/OneFormer.git无法下载可以前往OneFormer下载cdOneFormer更改requirements.txt中的内容cython shapelyh5py3.7.0submitit1.4.2timm0.4.12icecream2.1.2ftfy6.1.1regex2022.6.2inflect5.6.0diffdist0.1pytorch_lightning1.8.6mmcv1.7.0然后在终端键入pipinstall--upgradepip pipinstall-rrequirements.txt安装natten下载对应Natten将下载的文件放入OneFormer文件夹下wget--no-check-certificatehttps://shi-labs.com/natten/wheels/cu117/torch1.13/natten-0.14.5%2Btorch1130cu117-cp310-cp310-linux_x86_64.whlpipinstallnatten-0.14.5torch1130cu117-cp310-cp310-linux_x86_64.whl要是在上面网站下载不了可以尝试下面的方法pipinstallnatten0.14.5 --no-build-isolation-ihttps://pypi.tuna.tsinghua.edu.cn/simple --trusted-host pypi.tuna.tsinghua.edu.cn --no-cache-dir--timeout100测试依赖pythonimportnattenimporttorchimportpytorch_lightning as pl print(natten.__version__)print(torch.__version__)print(pl.__version__)输出类似exit()安装detectron2需要保证g为10以下sudoaptinstallg-9 gcc-9sudoupdate-alternatives--install/usr/bin/gcc gcc /usr/bin/gcc-990sudoupdate-alternatives--install/usr/bin/g g /usr/bin/g-990sudoupdate-alternatives--configgccsudoupdate-alternatives--configg gcc--versiong--version出现类似上面的输出时就键入g±9前面的序号我这是2没有就不用。gitclone https://github.com/facebookresearch/detectron2.gitcddetectron2 pipinstall--upgradepip pipinstall-e.--no-build-isolation安装其他依赖pip3installgithttps://github.com/cocodataset/panopticapi.git pip3installgithttps://github.com/mcordts/cityscapesScripts.git要是不行试试下面的方法挂个梯子然后访问浏览器panopticapi然后将下载的源码放在OneFormer文件夹下然后解压将解压文件命名为panopticapi然后再终端键入cd..cdpanopticapi pipinstall.访问浏览器cityscapesScripts然后将下载的源码放在OneFormer文件夹下然后解压将解压文件命名为cityscapesScripts然后再终端键入cd..cdcityscapesScripts pipinstall.然后把panopticapi和cityscapesScripts文件夹删了构建MSDeformAttncd..cdoneformer/modeling/pixel_decoder/opsshmake.sh二、数据集下载及预处理2.1 数据集下载回到DG-SLAM文件夹下:cd ~/VIO/SLAM/GS/DG-SLAM2.1.1 TUM RGB-D数据集bashscripts/download_tum.sh2.1.2 BONN动态RGB-D数据集bashscripts/download_bonn.sh在data文件夹下可以找到对应的数据集2.2 数据预处理由于DG-SLAM依赖 OneFormer 语义分割来生成运动掩码所以还需要部署OneFormer生成运动掩码。2.2.1 Semantic motion mask生成由于demo只能对单张图片进行处理我们需要对代码进行一些改动在./submodules/OneFormer/demo/demo.py头部加入import glob修改demo.py中ifname “main”:下面的内容为if__name____main__:seed0random.seed(seed)np.random.seed(seed)torch.manual_seed(seed)torch.cuda.manual_seed_all(seed)torch.backends.cudnn.deterministicTrue torch.backends.cudnn.benchmarkFalse mp.set_start_method(spawn,forceTrue)argsget_parser().parse_args()setup_logger(namefvcore)loggersetup_logger()logger.info(Arguments: str(args))cfgsetup_cfg(args)demoVisualizationDemo(cfg)ifargs.input:# 假设 args.input 是一个字符串单个目录如果不是请取 args.input[0]input_dirargs.input[0]ifisinstance(args.input, list)elseargs.input# 获取该目录下所有 .png 文件png_pathssorted(glob.glob(os.path.join(input_dir,*.png)))ifnot png_paths: raise ValueError(fNo .png images found in {input_dir})# 输出目录必须存在否则创建ifargs.output: os.makedirs(args.output,exist_okTrue)else: raise ValueError(Please specify an output path!)forpathintqdm.tqdm(png_paths,disablenot args.output): imgread_image(path,formatBGR)start_timetime.time()predictions, visualized_outputdemo.run_on_image(img, args.task)logger.info({}: {} in {:.2f}s.format(path,detected {} instances.format(len(predictions[instances]))ifinstancesinpredictionselsefinished, time.time()- start_time,))# 保存输出forkinvisualized_output.keys():# 每种输出类型一个子目录如 semantic / panoptic / instanceout_diros.path.join(args.output, k)os.makedirs(out_dir,exist_okTrue)filenameos.path.basename(path)out_pathos.path.join(out_dir, filename)visualized_output[k].save(out_path)else: raise ValueError(No Input Given)在OneFormer下面下载checkpoint如下图将下载好的文件放在DG-SLAM/submodules/OneFormer/configs/ade20k/swin下。运行demomkdir./data/seg_mask# 创建一个输出文件夹python ./submodules/OneFormer/demo/demo.py --config-file ./submodules/OneFormer/configs/ade20k/swin/oneformer_swin_large_bs16_160k.yaml--input./data/TUM/rgbd_dataset_freiburg3_walking_rpy/rgb--tasksemantic--output./data/seg_mask--optsMODEL.IS_TRAIN False MODEL.IS_DEMO True MODEL.WEIGHTS ./submodules/OneFormer/configs/ade20k/swin/250_16_swin_l_oneformer_ade20k_160k.pth然后可以得到把semantic_inference中的图片放到seg_mask下。然后将seg_mask放到DG-SLAM/data/TUM/rgbd_dataset_freiburg3_walking_rpy下。得到的图如下三、运行python run_tum.py--plot_curve--datapath./data/TUM--config./configs/TUM_RGBD/freiburg3_walking_rpy.yaml报错1ModuleNotFoundError缺少lietorch进入submodules中安装cdsubmodulesgitclone--recursivehttps://github.com/princeton-vl/lietorch.gitcdlietorch python setup.pyinstall安装好后cd ../..报错2attempted relative import beyond top-level package错误来自 Python 把导入的 “modules” 解析为你系统环境中已安装的第三方包MultiScaleDeformableAttention通过OneFormer的方法安装的而不是你仓库里的局部包我们需要的是dg_slam下的 “modules”这会导致很多错误所以需要将绝对导入改为相对导入需要改的地方太多了所以我直接将改好的dg_slam文件夹放到王盘中如果需要可自行下载dg-slam报错3ModuleFoundError: No module named torch scatter需要装torch-scatter,进入网站torch-scatter找到下图下载在改文件的目录中激活虚拟环境conda activate dg-slam,然后pip install ./torch_scatter-2.1.0pt113cu117-cp310-cp310-linux_x86_64.whl报错4No module named diff gaussian rasterization depth pose这是因为没有安装diff-gaussian-rasterization_pose扩展模块cd/home/nightcat/VIO/SLAM/GS/DG-SLAM/submodules/diff-gaussian-rasterization_pose python setup.pyinstall报错找不到 glm 库头文件glm/glm.hpp安装 GLM 头文件库sudo apt install libglm-dev安装好后重新python setup.py install报错5No module named plyfile需要安装该依赖库 plyfile 包pip install plyfile报错6No module named simple_knn需要安装该依赖库 simple_knn 包需要用到3D Gaussian Splatting的simple-knn,所以去gaussian-splatting下载其中的submodules/simple-knn放到DG-SLAM/submodules中然后cdsubmodules/simple-knn python setup.pyinstall报错7No module named droid_backends查看DROID-SLAM发现需要通过setup.py安装droid_backends模块但是DG-SLAM中没有所以需要自行构建。先克隆DROID-SLAMgit clone https://github.com/princeton-vl/DROID-SLAM.git然后将里面的setup.py和src文件夹移动到DG-SLAM下面修改setup.py里面内容为from setuptoolsimportsetup from torch.utils.cpp_extensionimportBuildExtension, CUDAExtensionimportos.path as osp ROOTosp.dirname(osp.abspath(__file__))EIGEN_DIRosp.realpath(osp.join(ROOT,./submodules/lietorch/eigen))setup(namedroid_backends,ext_modules[CUDAExtension(droid_backends,include_dirs[EIGEN_DIR],sources[src/droid.cpp,src/droid_kernels.cu,src/correlation_kernels.cu,src/altcorr_kernel.cu,],extra_compile_args{cxx:[-O3],nvcc:[-O3],}),],cmdclass{build_ext:BuildExtension})终端键入exportTORCH_CUDA_ARCH_LIST8.6pipinstall--no-build-isolation.运行后如下效果非常差。