Multi-view Incremental Segmentation of 3D Point Clouds for MobileRobots

Jingdao Chen1, Yong K. Cho2, and Zsolt Kira1

Abstract— Mobile robots need to create high-definition 3Dmaps of the environment for applications such as remotesurveillance and infrastructure mapping. Accurate semanticprocessing of the acquired 3D point cloud is critical for allowingthe robot to obtain a high-level understanding of the sur-rounding objects and perform context-aware decision making.Existing techniques for point cloud semantic segmentation aremostly applied on a single-frame or offline basis, with no wayto integrate the segmentation results over time. This paperproposes an online method for mobile robots to incrementallybuild a semantically-rich 3D point cloud of the environment.The proposed deep neural network, MCPNet, is trained topredict class labels and object instance labels for each pointin the scanned point cloud in an incremental fashion. A multi-view context pooling (MCP) operator is used to combine pointfeatures obtained from multiple viewpoints to improve theclassification accuracy. The proposed architecture was trainedand evaluated on ray-traced scans derived from the Stanford3D Indoor Spaces dataset. Results show that the proposedapproach led to 15% improvement in point-wise accuracy and7% improvement in NMI compared to the next best onlinemethod, with only a 6% drop in accuracy compared to thePointNet-based offline approach.

I. INTRODUCTION

Mobile robots are frequently used to autonomously ex-plore and survey unknown areas. To understand its envi-ronment, a mobile robot can use sensors such as cameras[1], depth sensors [2], and laser scanners [3] to perceive itssurroundings. Regardless of the hardware setup, the acquireddata can be summarized in point cloud form, which is anarray of 3D points containing geometric and optionally colorinformation. Recent studies [4][5] have increasingly exploredthe use of 3D point clouds as a richer and more versatilerepresentation of the environment. Retrieving semantic infor-mation from point clouds is valuable for the robot to performtasks such as obstacle detection, place recognition, and objectretrieval.

However, current methods for point cloud semantic seg-mentation such as PointNet [6] and SGPN [7] are lackingwhen applied to robotic real-time scanning because theyare designed to operate on point clouds one at a timeand do not incorporate information from new scans in anincremental fashion. In particular, semantic segmentation isusually performed on individual scans or performed offlineafter complete scan data is collected. To overcome these

1 Institute for Robotics and Intelligent Machines, Geor-gia Institute of Technology, Atlanta, GA 30332, USAjchen490@gatech.edu,zkira@gatech.edu

2 School of Civil and Environmental Engineering, Georgia Institute ofTechnology, Atlanta, GA 30332, USA yong.cho@ce.gatech.edu

shortcomings, this study proposes a multi-view incremen-tal segmentation method that can perform online instancesegmentation of 3D point clouds. The instance segmentationprocess assigns a class label and instance label to each pointin the scanned point cloud. The class label is computeddirectly by a neural network whereas the instance label isdetermined by agglomerative clustering. The segmentationresults are progressively updated in a dynamic fashion asthe robot traverses its environment and acquires laser scandata, allowing real-time object recognition and improvedscene understanding for the robot. The proposed work usesa novel multi-view context pooling (MCP) module to ag-gregate information from previous views before performinginstance segmentation. The key idea of this work is to usecontextual information from not all, but a select few of theprevious scans, to improve segmentation of the current scan.This online incremental segmentation framework allows newscans to be processed within tenths of a second compared tohalf a minute if segmentation of the entire point cloud wereto be recomputed with an offline method.

In summary, the main contributions of this work are asfollows: (i) we develop a multi-view context pooling (MCP)module to combine information from multiple viewpoints (ii)we propose a joint network for online point cloud clusteringand classification under a single framework, and (iii) weintroduce a simulated ray-tracing dataset for labeled dynamiclaser scan data [8].

II. RELATED WORK

Semantic processing of point cloud data acquired from amobile robot can be carried out in several forms such asclustering and classification. The clustering process aims tosubdivide a large point cloud into smaller chunks that formsemantically cohesive units. Clustering methods usually relyon generating seed points, then performing a region growingprocedure to create a point cloud cluster around the seedpoint [9]. Cluster assignment is propagated to neighboringpoints based on criteria such as distance, similarity of normalvectors, and similarity of color [10]. More sophisticatedmethods for clustering use deep learning frameworks to infera point feature embedding that can be used to predict pointgrouping proposals, for example, Similarity Group ProposalNetworks (SGPN) [7]. In this case, the distance betweenpoints in the learned embedding space is used as the criteriafor grouping points together.

On the other hand, classification of point cloud data canbe carried out at the scale of the point level, such that eachpoint has an individual label. Some methods project the 3D

arX

iv:1

902.

0676

8v1

[cs

.RO

] 1

8 Fe

b 20

19

point cloud into a 2D form and perform segmentation on theresulting image [11][12]. Other methods use 3D convolutionsto operate on a voxel grid and compute features in a layeredfashion [13][14]. Due to the poor computational scalabilityof 3D convolutions, an alternative is to compute both pointfeatures and scene features derived from pooling operations,which are then concatenated to predict class probabilitiesfor each point [6][15]. Further advancements to this line ofwork use superpoint graphs [16], recurrent networks [17],or coarse-to-fine hierarchies [4] to incorporate neighborhoodinformation and local dependencies to the prediction stage.However, these methods are usually applied in the offlinesetting, i.e. as a post-processing step after complete pointcloud data is obtained, and do not take into account occlusioneffects that comes into play with robotic scanning.

In contrast, for robotics applications, point cloud data ofa scene is usually incrementally obtained in separate scansas the robot moves to different points around the site ofinterest. To make use of multi-view information, featuresfrom multiple viewpoints can be combined using operationssuch as global view pooling [18], grouped pooling [19], orjoint viewpoint prediction and categorization [20]. However,these methods perform classification at the object level inthe offline setting, where the views are combined withcomplete point cloud information. For classification at thepoint level, several works [3][21] use a method where pointfeatures are computed repeatedly for each observation andmerged to determine the final classification. However, thefinal view merging process is still performed offline usingcomputationally-heavy methods such as conditional randomfields.

In summary, conventional methods for semantic process-ing of point cloud data are either performed on individualscans or performed offline. The proposed work addressesthese limitations by using an incremental method to incor-porate new scan information in the instance segmentationtask as opposed to considering scans on an individual basis.

III. METHODOLOGYA. 3D scan data staging

The Stanford 3D Indoor Spaces (S3DIS) dataset [22]is used to generate training and test data in the form oflabeled 3D point clouds for this study. The dataset con-tains 6 building areas with point clouds segments that areindividually annotated. It also contains 13 classes of objectswhich includes both building components such as walls anddoors as well as furniture such as tables and chairs. Avirtual robot with laser scanners is placed in the environmentand acquires laser scans by ray-tracing. The virtual robot ismanually navigated in the building to ensure that all objectsin the environment are adequately scanned. The robot is alsoassumed to have cameras so that color information can bemapped onto the point cloud. The building environment isrepresented as a voxel grid and points are scanned whenevera laser ray intersects an occupied voxel. Note that the voxelgrid is only used as a look-up table and the original pointcloud coordinates (before rounding to the nearest voxel) are

used for segmentation. The robot traverses a pre-specifiedtrajectory across the environment and dynamically acquiresa scan every 0.2m. At each scan point, ray-tracing is carriedout 360◦ horizontally and 180◦ vertically around the robotwith a horizontal and vertical resolution of 1◦. This can beemulated in real robots by physically rotating a planar laserscanner using a motor and ensures that the ceiling and floorarea around the robot can be scanned.

The ray-tracing process allows the acquired data to fully-capture the effects of occlusion and clutter that are commonin real-world environments. Figure 1 shows the original pointcloud of a room as well as the resulting point cloud obtainedby ray-tracing along a trajectory. This process assumes thatthe robot is capable of accurate localization such that scandata can be registered automatically. The ray tracing code ismade publicly available [8].

(a) Original real point cloud of an indoor environment

(b) Synthetic point cloud from ray-tracing results along a trajectory

Fig. 1. Visualization of dynamic scanning by ray-tracing for a virtualmobile robot

B. Incremental point classification

The proposed incremental segmentation method involvesa neural network architecture, named Multi-view ContextPooling Network (MCPNet), that jointly predicts an instancelabel and a class label for each scanned point. The class labeldescribes the semantic class of a point (i.e. wall, door, table,etc.) whereas the instance label is a unique ID such that onlypoints from the same object instance have the same ID.

In order to perform instance segmentation on an onlinesetting, it is important to keep track of newly scanned pointsand their relationship to previously scanned points in a globalcoordinate system. Thus, a voxel grid with grid size of0.1m is used as a lookup-table to store the point coordinatesand their corresponding segmentation results. The voxel grid

Fig. 2. MCPNet architecture for incremental instance segmentation of 3D point clouds. The network consist of two branches: class labelling, describedin Section IIIB, and instance labelling, described in Section IIIC

ensures that points are only added to the lookup-table whennew regions are scanned and prevents the data from growingtoo quickly, since the point cloud is limited to one pointper voxel. The voxel grid is also key to efficiently retrievingneighbor points and context points for segmentation.

Each input scan acquired by the robot is first pre-processedto retain points only in an area of radius 2m around the robot.This is due to the fact that scan regions that are too far awayfrom the robot are too sparse to make accurate predictionsabout the object class. Thus, faraway points are discardedfor the current scan but can still be included in a future scanas the robot moves to a closer locations. Next, the pointcoordinates are normalized so that the x-y coordinates arecentered around the robot and the z-coordinate is zero at thefloor level. The points are then divided into batches of Nx6matrices, where N is the batch size and the columns are X-Y-Z-R-G-B values. If there are more than N points in thecurrent scan, data will be passed to the network in multiplebatches, where the last batch is sampled with replacement.

Figure 2 shows an outline of the proposed MCPNet archi-tecture used to process the resulting input matrices. The inputmatrix is first passed through a 1D convolution layer to forman intermediate Nx200 feature matrix. The network thensplits into two branches: the lower branch for classificationand the upper branch for clustering, which will be describedin detail in the next subsection. The lower branch uses amax pooling function to compute a global feature vectorrepresenting the input points, similar to PointNet[15]. This isthen concatenated with the feature embedding from the upperbranch and passed through another 1D convolution layer tocompute class probabilities for each point, represented as a

Nx13 matrix since there are 13 classes.The proposed architecture also incorporates a Multi-view

Context Pooling (MCP) module which incorporates contex-tual information from previous views to points in the currentscan to improve the classification accuracy. The input to theMCP module is an NxMx6 tensor that stores context pointsfor each of the N points in the current input. A context pointfor point pi is defined as any point from previous viewsthat is at most three cells away from pi in the global voxelgrid. The three-cell distance threshold is set so that onlypoints from previous views that are in close proximity to thecurrent view are used as context. M points are thus randomlysampled from among all the context points accumulatedfrom past scans. The input tensor is passed through twolayers of 1D convolutions as well as a max pooling layer. ANx200 matrix forms the output of the MCP module and isconcatenated to the Nx6 input matrix of the main network.

C. Incremental clustering

The upper branch of the proposed network (Figure 2)aims to project the point features to an embedding spacethat is suitable for clustering, i.e. points from the sameobject should be close together whereas points from differentobjects should be further apart in the embedding space.This is enforced by computing a triplet loss [23] betweensets of three points, p1, p2 which originate from the sameobject and p3 which originates from a different object, andminimizing the resulting loss function. Let the projectionto the embedding space be represented as a function, f :R6 7→ R50. At training time, minimizing the triplet lossencourages the distance between points that should belong

Fig. 3. Overall framework for multi-view incremental segmentation

to the same object to be greater than that of points thatshould belong to different objects in the embedding space,i.e. ||f(p1)− f(p2)||2 + α < ||f(p1)− f(p3)||2, where α isthe margin parameter.

At inference time, the instance labels for a new point cloudscan are determined by an agglomerative clustering scheme.First, a set of neighbor points are retrieved for each newpoint pi. A neighbor point to pi is defined as a point thatis at most one cell away from pi in the global voxel grid.Then, each point pi is only connected to a neighboring pointpj if the cosine similarity, f(pi)·f(pj)

||f(pi)|| ||f(pj)|| is greater than β,where β is a preset threshold. The following rules are thenapplied:• If no connections exist, the point is initialized as the

seed for a new cluster with a new instance ID.• If connections exist to a single existing cluster, the point

is added to that cluster and takes the correspondinginstance ID.

• If connections exist to multiple existing clusters, allconnected clusters are merged and their instance IDsare updated accordingly.

D. Network training

Training data is first collected by carrying out virtualscanning of the S3DIS dataset as described in Section IIIA. The proposed network is then trained from scratch in theoffline setting. The two-branches are jointly optimized duringtraining: the point-wise classification branch is trained withthe softmax cross entropy loss, whereas the point featureembedding branch is trained with the triplet semihard loss, asdescribed in [23]. The network is implemented in Tensorflowand trained with the ADAM optimizer. Training is carried outfor 100 epochs with a learning rate of 0.001 and batch sizeof N = 256 points.

E. Overall segmentation framework

Figure 3 summarizes the overall segmentation framework.Point cloud data is stored in a global look-up table thatefficiently maps voxel coordinates to point coordinates andsegmentation results. Each time the robot acquires a newlaser scan, points in previously-unseen voxels are addedas new entries to the look-up table. The current scan isfirst normalized and passed to the network in batches ofN points. Context points from both current and previousviews are also retrieved and passed to the MCP module.The network then computes a feature embedding as well asclassification scores for each input point. The network outputis recombined across all batches of the current scan and theglobal look-up table is updated on a point-by-point basis.The computed feature embedding is used to cluster the newpoints together with previous object instances. Finally, theclass label and instance label of points in the current scanare updated accordingly.

IV. RESULTS

A. Evaluation of classification accuracy

The instance segmentation results were first evaluated interms of the classification accuracy on the S3DIS dataset[22]. A cross-validation scheme is used where one area ofthe dataset is held out as test data whereas the remainingareas are used as training data, repeated for 6 buildingareas. As shown in Table I, the performance of the proposedmethod, with and without the MCP module, is comparedagainst four other conventional point cloud classificationmethods, which were applied online on individual scans.PointNet, PointNet++, and VoxNet were trained on onlythe classification task whereas the remaining methods weretrained on both the classification and clustering tasks. Theresult of applying PointNet and SGPN on an offline basis,

Fig. 4. Visual comparison of feature embedding results: (i) original point cloud (ii) SGPN embedding (iii) proposed embedding (without MCP) (iv)proposed embedding (with MCP)

that is on the complete point cloud after scanning, is alsoshown to obtain an upper bound for classification accuracy.Note that this baseline is obtained from the original post-scan dataset [22] which has significantly less occlusion andis additionally cleanly divided into rooms.

The evaluation metrics are: (i) Intersection-Over-Union(IOU), defined as the number of true positive points dividedby the total true positive, false positive, and false negativepoints (according to the convention of [16][17]) (ii) point-wise accuracy, defined as the number of true positive pointsdivided by total number of points (iii) object-wise accuracy,defined as the number of true positive object instancesdivided by total number of object instances. The IOU metricis additionally averaged over 13 classes. The object-wiseaccuracy has lower values compared to point-wise accuracysince it requires an object to be both correctly segmented andhave correctly classified points. Experimental results showthat the proposed method with MCP outperforms the othermethods in terms of both IOU and accuracy.

TABLE ICLASSIFICATION ACCURACY ON S3DIS DATASET

Method Avg IOU Point Acc. Obj. Acc.

PointNet (offline) 49.8 79.9 -SGPN (offline) 50.4 80.8 -

PointNet [6] 25.0 59.1 16.7PointNet++ [15] 23.8 60.2 14.1

SGPN [7] 24.9 60.3 14.1VoxNet [13] 18.2 45.5 6.5

Proposed 25.4 57.7 15.6Proposed + MCP 39.2 74.9 33.5

Figure 5 shows an analysis on the effect of the number ofcontext points used in the proposed MCP module (Figure 2)on the classification accuracy (trained on a smaller dataset).The graph shows that the average accuracy increases withnumber of context points but also incurs a higher computa-tional cost. In the implementation used in this study, a totalof 50 context points are used.

B. Evaluation of clustering performance

On the other hand, the clustering performance was mea-sured using three different metrics: normalized mutual in-formation (NMI), adjusted mutual information (AMI), andadjusted rand index (ARI), as defined in [24]. Table IIshows a comparison of the clustering metrics using different

Fig. 5. Classification accuracy and inference rate as a function of thenumber of context points

methods, with the region-growing scheme[9][10] based ondifferences in normal vectors and color shown as a baseline.In this section, PointNet, PointNet++, and VoxNet usedthe clustering technique proposed in [6], where connectedcomponents are formed between points with the same classlabel. On the other hand, the remaining methods used the ag-glomerative clustering technique described in Section III.C,with β = 0.98 for SGPN and β = 0.9 for MCPNet (tuned bycross-validation). Results show that even the simple normal+ color-based region-growing scheme can achieve a goodclustering, but the proposed method with MCP achieved thebest clustering result overall.

To validate the ability of the proposed network to projectpoint features to an embedding space that is effective forclustering, the feature embedding is visualized as follows:(i) the point cloud is passed through MCPNet to compute ahigh-dimensional feature vector for each point (ii) PrincipalComponent Analysis is used to reduce the feature vector to 3dimensions (iii) the RGB color channels of each point is setto their corresponding 3-dimensional feature vector. Visualresults in Figure 4 shows that the proposed method was ableto produce distinct separation between points from differentobjects.

Finally, Figure 6 shows the global instance segmentationresults at intermediate points along the incremental scanningprocess whereas Figure 7 shows close-up views of specificregions in the scanned environment.

Fig. 6. Incremental segmentation results on S3DIS test dataset: input point cloud with virtual robot trajectory (top row), clustering results (middle row)and classification results (bottom row). Color legend is shown for classification results.

Fig. 7. Close-up visualization of classification results with ceiling removed: (from left to right) (i) hallway with doors (ii) hallway with tables and chairs(iii) office room with board, bookcases, tables and chairs (iv) three adjacent offices

C. Efficiency evaluationTable III shows the computation time and memory usage

of each segmentation method, measured on an Intel Xeon E3-

TABLE IICLUSTERING PERFORMANCE ON S3DIS DATASET

Method NMI AMI ARI

Normal + color 78.5 63.4 25.0

PointNet [6] 72.1 59.2 12.1PointNet++ [15] 76.1 66.4 17.1

SGPN [7] 78.5 60.5 26.1VoxNet [13] 70.5 58.0 11.3

Proposed 75.9 64.8 18.3Proposed + MCP 85.6 74.2 39.7

1200 CPU with a NVIDIA GTX1080 GPU. Results showthat the proposed method with MCP led to an increase inprocessing time per scan, but at a reasonable amount suchthat it can still be run in real time. Whereas, the memoryusage is similar in magnitude to competing methods.

TABLE IIIEFFICIENCY COMPARISON FOR INSTANCE SEGMENTATION

MethodProcessing Time

per Scan (s)Network

Size (MB)CPU MemoryUsage (GB)

PointNet [6] 0.017 14.0 0.94PointNet++ [15] 0.027 11.5 0.97

SGPN [7] 0.019 14.9 2.49VoxNet [13] 0.017 0.5 1.22

Proposed 0.019 0.9 1.26Proposed + MCP 0.034 1.8 1.26

V. CONCLUSIONS

This study demonstrated a framework for performingincremental instance segmentation of laser-scanned pointcloud data on a mobile robot using multi-view contextualinformation. A method for simulating dynamic scan data byray-tracing building models was introduced. This ray-traceddataset, derived from the Stanford 3D Indoor Spaces (S3DIS)dataset, was used to train and test a deep neural networkfor instance segmentation and we release the code to allowreproduction of the dataset. Experimental results showed thatthe proposed approach led to 15% improvement in accuracyand 7% improvement in NMI compared to the next bestonline method.

ACKNOWLEDGMENT

The work reported herein was supported by the UnitedStates Air Force Office of Scientific Research (Award No.FA2386-17-1-4655) and by a grant (18CTAP-C144787-01)funded by the Ministry of Land, Infrastructure, and Transport(MOLIT) of Korea Agency for Infrastructure TechnologyAdvancement (KAIA). Any opinions, findings, and conclu-sions or recommendations expressed in this material arethose of the authors and do not necessarily reflect the viewsof the United States Air Force and MOLIT. Zsolt Kira waspartially supported by the National Science Foundation andNational Robotics Initiative (grant # IIS-1426998).

REFERENCES

[1] L.-F. Posada, A. Velasquez-Lopez, F. Hoffmann, and T. Bertram,“Semantic mapping with omnidirectional vision,” 2018 IEEE Inter-national Conference on Robotics and Automation (ICRA), pp. 1901–1907, 2018.

[2] P. Marion, P. R. Florence, L. Manuelli, and R. Tedrake, “Label fusion:A pipeline for generating ground truth labels for real rgbd data ofcluttered scenes,” in 2018 IEEE International Conference on Roboticsand Automation (ICRA). IEEE, 2018, pp. 3325–3242.

[3] M. Antonello, D. Wolf, J. Prankl, S. Ghidoni, E. Menegatti, andM. Vincze, “Multi-view 3d entangled forest for semantic segmentationand mapping,” in 2018 IEEE International Conference on Roboticsand Automation (ICRA), 2018.

[4] A. Dai, D. Ritchie, M. Bokeloh, S. Reed, J. Sturm, and M. Nießner,“Scancomplete: Large-scale scene completion and semantic segmenta-tion for 3d scans,” in Proc. Computer Vision and Pattern Recognition(CVPR), IEEE, 2018.

[5] J. Chen, Y. K. Cho, and J. Ueda, “Sampled-point network for classi-fication of deformed building element point clouds,” in Proceedingsof the 2018 IEEE Conference on Robotics and Automation (ICRA),2018.

[6] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learningon point sets for 3d classification and segmentation,” Proc. ComputerVision and Pattern Recognition (CVPR), IEEE, 2017.

[7] W. Wang, R. Yu, Q. Huang, and U. Neumann, “Sgpn: Similaritygroup proposal network for 3d point cloud instance segmentation,”in The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), June 2018.

[8] J. Chen, “Dynamic laser scanning dataset for multi-view incremen-tal segmentation,” https://github.com/jingdao/multiview segmentation,2018.

[9] R. Dub, M. G. Gollub, H. Sommer, I. Gilitschenski, R. Siegwart,C. Cadena, and J. Nieto, “Incremental-segment-based localization in3-d point clouds,” IEEE Robotics and Automation Letters, vol. 3, no. 3,pp. 1832–1839, July 2018.

[10] R. Finman, T. Whelan, M. Kaess, and J. J. Leonard, “Efficientincremental map segmentation in dense rgb-d maps,” in 2014 IEEEInternational Conference on Robotics and Automation (ICRA), May2014, pp. 5488–5494.

[11] B. Wu, A. Wan, X. Yue, and K. Keutzer, “Squeezeseg: Convolutionalneural nets with recurrent crf for real-time road-object segmentationfrom 3d lidar point cloud,” in 2018 IEEE International Conference onRobotics and Automation (ICRA), 2018.

[12] P. Wang, R. Yang, B. Cao, W. Xu, and Y. Lin, “Dels-3d: Deeplocalization and segmentation with a 3d semantic map,” in The IEEEConference on Computer Vision and Pattern Recognition (CVPR), June2018.

[13] D. Maturana and S. Scherer, “VoxNet: A 3D Convolutional NeuralNetwork for Real-Time Object Recognition,” in IROS, 2015.

[14] G. Riegler, A. O. Ulusoy, and A. Geiger, “Octnet: Learning deep3d representations at high resolutions,” in Proceedings of the IEEEConference on Computer Vision and Pattern Recognition, 2017.

[15] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deephierarchical feature learning on point sets in a metric space,” arXivpreprint arXiv:1706.02413, 2017.

[16] L. Landrieu and M. Simonovsky, “Large-scale point cloud semanticsegmentation with superpoint graphs,” in The IEEE Conference onComputer Vision and Pattern Recognition (CVPR), June 2018.

[17] F. Engelmann, T. Kontogianni, A. Hermans, and B. Leibe, “Exploringspatial context for 3d semantic segmentation of point clouds,” in IEEEInternational Conference on Computer Vision, 3DRMS Workshop,ICCV, 2017.

[18] H. Su, S. Maji, E. Kalogerakis, and E. G. Learned-Miller, “Multi-view convolutional neural networks for 3d shape recognition,” in Proc.ICCV, 2015.

[19] Y. Feng, Z. Zhang, X. Zhao, R. Ji, and Y. Gao, “Gvcnn: Group-viewconvolutional neural networks for 3d shape recognition,” in The IEEEConference on Computer Vision and Pattern Recognition (CVPR), June2018.

[20] A. Kanezaki, Y. Matsushita, and Y. Nishida, “Rotationnet: Joint objectcategorization and pose estimation using multiviews from unsuper-vised viewpoints,” in The IEEE Conference on Computer Vision andPattern Recognition (CVPR), June 2018.

[21] K. Tateno, F. Tombari, and N. Navab, “When 2.5d is not enough:Simultaneous reconstruction, segmentation and recognition on denseslam,” in 2016 IEEE International Conference on Robotics andAutomation (ICRA), May 2016, pp. 2295–2302.

[22] I. Armeni, O. Sener, A. R. Zamir, H. Jiang, I. Brilakis, M. Fischer,and S. Savarese, “3d semantic parsing of large-scale indoor spaces,”in Proceedings of the IEEE International Conference on ComputerVision and Pattern Recognition, 2016.

[23] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified em-bedding for face recognition and clustering,” in 2015 IEEE Conferenceon Computer Vision and Pattern Recognition (CVPR), June 2015, pp.815–823.

[24] N. X. Vinh, J. Epps, and J. Bailey, “Information theoretic measuresfor clusterings comparison: Variants, properties, normalization andcorrection for chance,” J. Mach. Learn. Res., vol. 11, pp. 2837–2854,Dec. 2010. [Online]. Available: http://dl.acm.org/citation.cfm?id=1756006.1953024