Interactive Editing of Signed Distance Fields

Interactive Editing of Signed Distance Fields
Ole Wegen, Jürgen Döllner, and Matthias Trapp
Hasso Plattner Institute, Faculty of Digital Engineering, University of Potsdam, Germany
30th International Conference on Computer Graphics, Visualization and Computer Vision 2022 (WSCG 2022)
This work was partially funded by the German Federal Ministry of Education and
Research (BMBF) through grants 01IS18092 ("mdViPro") and 01IS19006 ("KI-LAB-ITSE")

Motivation
 Automatic 3D reconstruction based on RGBD data is a key
functionality in robotics, autonomous driving, manufacturing, and digital twins
 Recent incorporation of required sensors (e.g., LiDAR) into
mobile devices makes 3D reconstruction accessible to millions of users
 Problem: Reconstruction results often contain artifacts or
irrelevant objects (due to limitations of sensors or algorithms)
 We investigated an editing approach for reconstructed 3D scenes
based on Signed Distance Fields (SDFs)
17th May 2022 Interactive Editing of Signed Distance Fields 2
Reconstructed Scene
Scene after Editing

Background: Signed Distance Fields
 An SDF stores signed distances to a surface for every voxel in a regular grid
 Can be created via iterative fusion of depth images [Curless & Levoy, 1996]
 Boolean operations on SDFs can be implemented via min/max operators [Hart, 1995]
 Our Goal: Provide interactive, GPU-aligned editing techniques for SDFs, reconstructed from real-
world data, using procedural shapes
Input SDF Intersection c = max(a,b) Subtraction c = max(-a,b) Union c = min(a,b)

Problem Statement & Contributions
Challenges for SDF Editing:
 Handling hybrid SDF scene representations:
procedural primitives + voxel grids
 Real-time rendering of hybrid SDF scene
representations for interactive editing
Contributions:
1. Interaction and manipulation techniques for voxel-
based SDFs using procedural shapes;
2. Approach for scene unification with subsequent SDF
recalculation for faster rendering;
3. A GPU-aligned implementation of the presented
techniques (w.r.t. data representation and algorithms)
Input scene Editing results

SDF Creation
TSDF Fusion Marching Cubes Mesh Sampling Jump Flooding
RGB-D
Data
TSDF Triangles
Point
Cloud
SDF
[Curless & Levoy, 1996] [Lorensen & Cline, 1987] [Rong & Tan, 2006]
Color
Volume
[Osada et al., 2002]

SDF Rendering via Sphere Tracing

Editing Concept
Low-Level (basic) operations based on procedural shapes:
 Creation, placement, and integration of procedural shapes
High-level operations based on copies of voxel subspace:
 Selection, transform, copy, and integration of voxel parts
Update
Selection
Modification
Selection of scene part Placing copy in the scene

SDF Scene Representation
Create Shape
Active Element
Shape
Grid Copy
Integrate
Procedural Shape
Copy Grid Part
Integrate
Grid Copy
Static Scene
List of
Shapes
Distance &
Color Grid
SDF
Recomputation
Unification
Input
Output
Invoke
Data
Process
1 2

Distance Query in SDF-Scene
float querySDF(float3 p, float *voxelGrid, /*...*/) {
float result = FLT_MAX;
if(bboxHit[0]) { result = queryVoxelGrid(p, voxelGrid); } // Handling voxel grid SDF
for(int i = 0; i < numberOfShapes; i++) { // Handling procedural shapes
if(bool(bboxHit[i+1])) {
float3 queryPoint = mul(sceneShapes[i].rotation, p-sceneShapes[i].position);
auto combinationFunc = sdfCombinationFunc[sceneShapes[i].integrationType];
auto shapeFunc = sdfShapeFunc[sceneShapes[i].type];
result = combinationFunc(result, shapeFunc(queryPoint, sceneShapes[i].size));
}
}
if(bool(bboxHit[numberOfShapes+1])) { // Handling active element
float3 queryPoint = mul(activeElement.rotation, p-activeElement.position);
auto combinationFunc = sdfCombinationFunc[activeElement. integrationType];
result = combinationFunc(result, activeElementSdf(queryPoint, activeElement));
}
return result;
}
Functions:
 sceneShapes
array of shapes added to the scene
 activeElement
currently selected shape or grid copy
 sdfCombinationFunc
array of functions impl. set operations
 sdfShapeFunc
array of signed distance functions for shapes
 queryVoxelGrid()
trilinear interpolation in a voxel grid
 mul()
applies a transformation matrix to a point
 activeElementSdf()
signed distance for the active element
(calling the corresponding sdfShapeFunc
for procedural shapes or queryVoxelGrid in
for voxel grid copies

Unification Operation
Problem:
Increased number of shapes in the scene leads to decreased tracing performance
Approach:
Provide the user a unification method for irreversibly persisting the changes
Implementation:
1. Recalculate distance value per voxel, considering the
list of procedural shapes (see querySDF(…))
2. Clear list of procedural shapes

SDF Recomputation (I)
Input Scene After copy: shadowing artefacts After SDF recomputation
Problem: Copying parts of voxel grids leads to shadowing artefacts and performance decrease

SDF Recomputation (II)
Approach:
Perform unification and recomputation after user finished copying operation
Implementation:
1. Perform unification (without considering the bounding boxes for copied parts)
2. Convert SDF Scene into a voxel grid of seed points for jump flooding (JFA)
3. Execute the JFA to obtain a new SDF (similar to SDF creation)

Applications: Structural Editing
SDF after moving scene objects
Original SDF scene from IPad scan

Applications: Scene Cleansing
May 2022 Interactive Editing of Signed Distance Fields 22
Original SDF scene from ScanNet scan SDF after scene cleansing (editing time ~ 5min)
(ScanNet: Dai et al., 2017)

Applications: Appearance Editing
Input SDF Edited SDF

Runtime Performance Evaluation (I)
Test system specification and setup:
 AMD Ryzen 5 5600x (6 cores, 3.7GHz) CPU, 32GB DDR4 RAM
 NVIDIA GeForce RTX 3090 GPU, 24GB VRAM
 Windows 10 OS, viewport resolution 1200 x 960 pixels (16x multisampling)
Virtual camera positions:
 Camera 1: partial scene
 Camera 2: complete (from top)
 Camera 3: complete (from below)
Camera 1 Camera 2 Camera 3

Runtime Performance Evaluation (II)
0
1
2
3
4
5
6
7
8
9
10
11
~1,5 Million ~10,8 Million ~86 Million ~697 Million
Time
per
frame
in
ms
Number of voxels
SDF rendering performance without soft shadows
0
1
2
3
4
5
6
7
8
9
10
11
~1,5 Million ~10,8 Million ~86 Million ~697 Million
Time
per
frame
in
ms
Number of voxels
SDF rendering performance with soft shadows

Runtime Performance Evaluation (III)
Results: SDF rendering performance with additional scene objects
0
5
10
15
20
25
30
0 10 20 30 40 50
Time
per
frame
in
ms
Number of additional scene objects
Visualization of additional scene objects

Runtime Performance Evaluation (IV)
Unification and SDF recalculation performance:
Voxel Grid Resolution #Voxels Unification SDF Recalculation
119 × 134 × 95 ~ 1.5 M 3 ms 24 ms
229 × 260 × 181 ~ 10.8 M 17 ms 204 ms
457 × 520 × 362 ~ 86.0 M 146 ms 1024 ms
917 × 1044 × 728 ~ 697.0 M 903 ms 8362 ms

Conclusions & Future Work
Conclusions:
 Approach for interactive editing
of voxel-based signed distance fields
 Hybrid representation of voxel grid
and procedural shapes
 Enables manual refinement of
reconstructed scenes in real-time
Future Work:
 Address the challenges of memory consumption for high-resolution voxel grids
 Enable appearance manipulation, e.g., manipulation the color volume directly
Reconstructed Scene Scene after Editing

References
 Robert Osada, Thomas Funkhouser, Bernard Chazelle, and David Dobkin. “Shape Distributions”.
In: ACM Trans. Graph. 21.4 (Oct. 2002), pp. 807–832.
 William E. Lorensen and Harvey E. Cline. “Marching cubes: A high resolution 3D surface construction algorithm”.
In: Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH. Ed. by
Maureen C. Stone. ACM, 1987, pp. 163–169.
 John Hart. “Sphere Tracing: A Geometric Method for the Antialiased Ray Tracing of Implicit Surfaces”.
In: The Visual Computer 12 (June 1995), pp. 527–545.
 Angela Dai, Angel X. Chang, Manolis Savva, Maciej Halber, Thomas A. Funkhouser, and Matthias Nießner.
“ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes”. In: 2017 IEEE Conference on Computer Vision and
Pattern Recognition, CVPR. IEEE Computer Society, 2017, pp. 2432–2443
 Brian Curless and Marc Levoy. “A Volumetric Method for Building Complex Models from Range Images”.
In: Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH. Ed. by John
Fujii. ACM, 1996, pp. 303–312.
 Guodong Rong and Tiow-Seng Tan. “Jump flooding in GPU with applications to Voronoi diagram and distance transform”.
In: Proceedings of the Symposium on Interactive 3D Graphics, Jan. 2006, pp. 109–116

Interactive Editing of Signed Distance Fields

Recommandé

Recommandé

Contenu connexe

Tendances

Tendances (20)

Similaire à Interactive Editing of Signed Distance Fields

Similaire à Interactive Editing of Signed Distance Fields (20)

Plus de Matthias Trapp

Plus de Matthias Trapp (20)

Dernier

Dernier (20)

Interactive Editing of Signed Distance Fields