点击上方“3D视觉工坊”，选择“星标”
干货第一时间送达
作者丨宇宙爆肝锦标赛冠
来源丨古月居
项目介绍
论文：《The Perfect Match: 3D Point Cloud Matching with Smoothed Densities》
论文链接：
https://arxiv.org/abs/1811.06879
Github 链接：
https://github.com/zgojcic/3DSmoothNet
系统环境
-
在 AutoDL 上租用的服务器的环境：
-
TensorFlow 1.15.5
-
Python 3.8(ubuntu18.04)
-
Cuda 11.4
-
GTX 1080 Ti
配置步骤
Step 1: 克隆项目代码
git clone https://github.com/zgojcic/3DSmoothNet.git
Step 2: 安装 PCL
./install_pcl.sh
这里我对官方的 Shell 脚本做了一些修改，因为我是在服务器上的 root 权限下操作，所以我删掉了所有的 sudo 命令，并且在开头设置了默认同意安装，且对 PCL 代码下载路径作了修改，具体内容如下：
################################### # 在服务器容器上安装，已经是 root 权限 # ################################### # 安装时默认 yes 同意安装 echo "y" | apt-get install whatever # Clone latest PCL apt-get update # apt-get install git # 之前已经有 git 的就无需安装 mkdir ../Tools cd ../Tools # for clone pcl-1.8.1 # git clone --branch pcl-1.8.1 https://github.com/PointCloudLibrary/pcl.git pcl-trunk git clone https://github.com/PointCloudLibrary/pcl.git pcl-trunk ln -s pcl-trunk pcl cd pcl # Install prerequisites apt-get install g++ apt-get install cmake cmake-gui apt-get install doxygen apt-get install mpi-default-dev openmpi-bin openmpi-common apt-get install libflann1.8 libflann-dev apt-get install libeigen3-dev apt-get install libboost-all-dev apt-get install libvtk6-dev libvtk6.2 libvtk6.2-qt #sudo apt-get install libvtk5.10-qt4 libvtk5.10 libvtk5-dev # I'm not sure if this is necessary. apt-get install 'libqhull*' apt-get install libusb-dev apt-get install libgtest-dev apt-get install git-core freeglut3-dev pkg-config apt-get install build-essential libxmu-dev libxi-dev apt-get install libusb-1.0-0-dev graphviz mono-complete apt-get install qt-sdk openjdk-9-jdk openjdk-9-jre apt-get install phonon-backend-gstreamer apt-get install phonon-backend-vlc apt-get install libopenni-dev libopenni2-dev apt-get install libflann-dev # Compile and install PCL mkdir release cd release cmake -DCMAKE_BUILD_TYPE=None -DBUILD_GPU=OFF -DBUILD_apps=ON -DBUILD_examples=ON .. make -j16 make install
Step 3: 安装深度学习环境
pip install -r requirements.txt
因为服务器上已经有了 glob 和 tensorflow-gpu 1.15.5，所以我将这两条注释掉了，在进行测试时无法正常 import open3d ，安装 open3d 而不是 open3d-python 可以解决问题。
最终我的 requirements.txt 如下：
numpy==1.14.5，open3d ，open3d-python，parse==1.12.0 ，scikit-learn==0.20.3 ，tensorboard==1.10 ，tqdm==4.31.1
Step 4: 编译项目代码
在 main.cpp 的开头加上 #include ，然后执行编译（否则会报错）：
cmake -DCMAKE_BUILD_TYPE=Release . && make
这里作者没有新建 build 文件夹，而是直接在项目根目录生成编译文件。
测试效果
我修改后的 demo.py 在附录中。
python ./demo.py
终端打印信息如下：
File: ./data/demo/cloud_bin_0.ply Number of Points: 258342 Size of the voxel grid: 0.3 Number of Voxels: 16 Smoothing Kernel: 1.75 Number of keypoints:1000 Starting SDV computation! 56 threads will be used!! Saving Features to a CSV file: ./data/demo/sdv/cloud_bin_0.ply_0.150000_16_1.750000.csv --------------------------------------------------------- LRF computation took 4025 miliseconds SDV computation took 7079 miliseconds --------------------------------------------------------- Config parameters successfully read in!! File: ./data/demo/cloud_bin_1.ply Number of Points: 268977 Size of the voxel grid: 0.3 Number of Voxels: 16 Smoothing Kernel: 1.75 Number of keypoints:1000 Starting SDV computation! 56 threads will be used!! Saving Features to a CSV file: ./data/demo/sdv/cloud_bin_1.ply_0.150000_16_1.750000.csv --------------------------------------------------------- LRF computation took 4457 miliseconds SDV computation took 7714 miliseconds --------------------------------------------------------- Run mode "test" selected. Loaded saved model ./models/32_dim/3DSmoothNet_32_dim.ckpt. Loading test file: ./data/demo/sdv/cloud_bin_0.ply_0.150000_16_1.750000.csv 1000 features computed in 6.972553253173828 seconds. Wrote file cloud_bin_0.ply_0.150000_16_1.750000_3DSmoothNet.npz Loading test file: ./data/demo/sdv/cloud_bin_1.ply_0.150000_16_1.750000.csv 1000 features computed in 0.30720067024230957 seconds. Wrote file cloud_bin_1.ply_0.150000_16_1.750000_3DSmoothNet.npz Inference completed perform nearest neighbor search and registration RegistrationResult with fitness=5.020000e-01, inlier_rmse=3.411006e-02, and correspondence_set size of 502 Access transformation to get result.
由于我是在服务器上配置的环境，没有可视化界面，所以保存了点云文件用 cloudcompare 打开，效果如下（蓝色是源点云，红色是目标点云，绿色是源点云配准后的结果）：
常见问题
问题一：安装 glob3 报错，经 import glob 测试，我的环境中本来就有 glob。
$ pip install glob3== # 查看包版本 Looking in indexes: http://mirrors.aliyun.com/pypi/simple, https://pypi.ngc.nvidia.com ERROR: Could not find a version that satisfies the requirement glob3== (from versions: none) ERROR: No matching distribution found for glob3==
问题二：No module named 'sklearn' ，之前 scikit-learn 安装失败。
pip install scikit-learn
问题三：测试 demo.py 时报错如下，将open3d.registration.Feature() 改为 open3d.pipelines.registration.Feature() 即可。
Traceback (most recent call last): File "./demo.py", line 87, in ref = open3d.registration.Feature() AttributeError: module 'open3d' has no attribute 'registration'
问题四：测试 demo.py 时报错如下，由于 open3d 版本的问题，导致函数发生了变化。
我这里安装的是 open3d==0.15.2 和 open3d-python==0.3.0.0 （open3d-python 是旧版本的 open3d ，对应的 python 版本也比较旧，我这里 python 版本是 3.8 ，所以我只能改函数）。
需要修改的如下：
将 read_point_cloud 改为 open3d.io.read_point_cloud
将 PointCloud() 改为 open3d.geometry.PointCloud()
将 registration_ransac_based_on_feature_matching改为open3d.pipelines.registration.registration_ransac_based_on1_feature_matching
将 TransformationEstimationPointToPoint 改为 open3d.pipelines.registration.TransformationEstimationPointToPoint
NameError: name 'read_point_cloud' is not defined NameError: name 'PointCloud' is not defined NameError: name 'Vector3dVector' is not defined NameError: name 'registration_ransac_based_on_feature_matching' is not defined NameError: name 'TransformationEstimationPointToPoint' is not defined NameError: name 'draw_geometries' is not defined
附录
demo.py
import tensorflow as tf
import copy
import numpy as np
import os
import subprocess
from open3d import *
def draw_registration_result(source, target, transformation):
source_temp = copy.deepcopy(source)
target_temp = copy.deepcopy(target)
source_temp.paint_uniform_color([1, 0.706, 0])
target_temp.paint_uniform_color([0, 0.651, 0.929])
source_temp.transform(transformation)
open3d.visualization.draw_geometries([source_temp, target_temp])
def execute_global_registration(
source_down, target_down, reference_desc, target_desc, distance_threshold):
result = open3d.pipelines.registration.registration_ransac_based_on_feature_matching(
source_down, target_down, reference_desc, target_desc,
False,
distance_threshold,
open3d.pipelines.registration.TransformationEstimationPointToPoint(False), 4,
[open3d.pipelines.registration.CorrespondenceCheckerBasedOnEdgeLength(0.9),
open3d.pipelines.registration.CorrespondenceCheckerBasedOnDistance(distance_threshold)],
open3d.pipelines.registration.RANSACConvergenceCriteria(4000000, 500))
return result
def refine_registration(source, target, source_fpfh, target_fpfh, voxel_size):
distance_threshold = voxel_size * 0.4
print(":: Point-to-plane ICP registration is applied on original point")
print(" clouds to refine the alignment. This time we use a strict")
print(" distance threshold %.3f." % distance_threshold)
result = registration_icp(source, target, distance_threshold,
result_ransac.transformation,
TransformationEstimationPointToPlane())
return result
# Run the input parametrization
point_cloud_files = ["./data/demo/cloud_bin_0.ply", "./data/demo/cloud_bin_1.ply"]
keypoints_files = ["./data/demo/cloud_bin_0_keypoints.txt", "./data/demo/cloud_bin_1_keypoints.txt"]
for i in range(0,len(point_cloud_files)):
args = "./3DSmoothNet -f " + point_cloud_files[i] + " -k " + keypoints_files[i] + " -o ./data/demo/sdv/"
subprocess.call(args, shell=True)
print('Input parametrization complete. Start inference')
# Run the inference as shell
args = "python main_cnn.py --run_mode=test --evaluate_input_folder=./data/demo/sdv/ --evaluate_output_folder=./data/demo"
subprocess.call(args, shell=True)
print('Inference completed perform nearest neighbor search and registration')
# Load the descriptors and estimate the transformation parameters using RANSAC
reference_desc = np.load('./data/demo/32_dim/cloud_bin_0.ply_0.150000_16_1.750000_3DSmoothNet.npz')
reference_desc = reference_desc['data']
test_desc = np.load('./data/demo/32_dim/cloud_bin_1.ply_0.150000_16_1.750000_3DSmoothNet.npz')
test_desc = test_desc['data']
# Save as open3d feature
ref = open3d.pipelines.registration.Feature()
ref.data = reference_desc.T
test = open3d.pipelines.registration.Feature()
test.data = test_desc.T
# Load point cloud and extract the keypoints
reference_pc = open3d.io.read_point_cloud(point_cloud_files[0])
test_pc = open3d.io.read_point_cloud(point_cloud_files[1])
indices_ref = np.genfromtxt(keypoints_files[0])
indices_test = np.genfromtxt(keypoints_files[1])
reference_pc_keypoints = np.asarray(reference_pc.points)[indices_ref.astype(int),:]
test_pc_keypoints = np.asarray(test_pc.points)[indices_test.astype(int),:]
# Save ad open3d point clouds
ref_key = open3d.geometry.PointCloud()
ref_key.points = open3d.utility.Vector3dVector(reference_pc_keypoints)
test_key = open3d.geometry.PointCloud()
test_key.points = open3d.utility.Vector3dVector(test_pc_keypoints)
result_ransac = execute_global_registration(ref_key, test_key,
ref, test, 0.05)
# First plot the original state of the point clouds
draw_registration_result(reference_pc, test_pc, np.identity(4))
# Plot point clouds after registration
print(result_ransac)
draw_registration_result(reference_pc, test_pc,
result_ransac.transformation)
# 保存配准点云结果
source_reg = copy.deepcopy(reference_pc)
source_reg.transform(result_ransac.transformation)
open3d.io.write_point_cloud("reg_result.pcd", source_reg)
open3d.io.write_point_cloud("target.pcd", test_pc)
open3d.io.write_point_cloud("source.pcd", reference_pc)
