🚀 NeuralGS: Bridging Neural Fields and 3D Gaussian Splatting for Compact 3D Representation

arXiv Code(coming soon)

Zhenyu Tang1*, Chaoran Feng1*, Xinhua Cheng1, Wangbo Yu1, Junwu Zhang1,
Yuan Liu2†, Xiaoxiao Long2, Wenping Wang3, Li Yuan 1†,
1Peking University
2Hong Kong University of Science and Technology
3Texas A&M University
*These authors contributed equally to this work.
†Corresponding author.

We propose NeuralGS, a novel framework that effectively adopts the neural field representation to encode the attributes of 3D Gaussians with multiple tiny MLPs, only requiring a small model size even for a large-scale scene.

📌 Abstract


3D Gaussian Splatting (3DGS) demonstrates superior quality and rendering speed, but with millions of 3D Gaussians and significant storage and transmission costs. Recent 3DGS compression methods mainly concentrate on compressing Scaffold-GS, achieving impressive performance but with an additional voxel structure and a complex encoding and quantization strategy. In this paper, we aim to develop a simple yet effective method called NeuralGS that explores in another way to compress the original 3DGS into a compact representation without the voxel structure and complex quantization strategies. Our observation is that neural fields like NeRF can represent complex 3D scenes with Multi-Layer Perceptron (MLP) neural networks using only a few megabytes. Thus, NeuralGS effectively adopts the neural field representation to encode the attributes of 3D Gaussians with MLPs, only requiring a small storage size even for a large-scale scene. To achieve this, we adopt a clustering strategy and fit the Gaussians with different tiny MLPs for each cluster, based on importance scores of Gaussians as fitting weights. We experiment on multiple datasets, achieving a 45$\times$ average model size reduction without harming the visual quality. The compression performance of our method on original 3DGS is comparable to the dedicated Scaffold-GS-based compression methods, which demonstrate the huge potential of directly compressing original 3DGS with neural fields.

💡 Compression Method

We propose a simple yet effective framework NeuralGS, which adopts the neural field representation to encode the attributes of 3D Gaussians with MLPs to enable a compact 3D representation. Specifically, as shown in Figure 1, we use a designed criterion to assess the importance of each Gaussian, allowing us to prune Gaussians that have minimal impact on renderings. To reduce variations among Gaussians, we cluster these 3D Gaussians based on their attributes, ensuring similarity within each cluster. Then, each cluster is then assigned a tiny MLP that fits the attributes of its 3D Gaussians. Given the varying contributions of each Gaussian to the renderings, we apply Gaussian’s importances as the fitting weights in the MLP fitting. We also incorporate a fine-tuning stage along with frequency loss to restore quality and preserve high-frequency details. We address the storage and rendering issue of 3D Gaussian Splatting (3DGS) by compressing the reconstructed scene parameters and rendering the compressed representation via GPU rasterization. To compress the scenes, we first analyze its components and observe that the Spherical Harmonics (SH) coefficients and the multivariate Gaussian parameters take up the majority of storage space and are highly redundant. Our compression pipeline consists of three steps:


Compression Pipeline
Figure 1: Overview of NeuralGS.

🔗 Results

We evaluate our method on four datasets for comparison on NVIDIA A100 GPUs.


Qualitative Comparisons

Compression Pipeline
Figure 2: Qualitative comprison on the mentioned four datasets.

Quantitative Comparisons

Table 1. Quantitative results evaluated on Mip-NeRF 360, Tanks&Temples, and Deep Blending datasets. We highlight the best-performing results in red and the second-best results in yellow for all compression methods
Compression Pipeline

Table 2. Quantitative results of the proposed method evaluated on the NeRF-Synthetic dataset. We highlight the best-performing results in red and the second-best results in yellow for all compression methods.
Table 2

BibTeX

@InProceedings{neuralgs,
                author    = {Zhenyu Tang, Chaoran Feng, Xinhua Cheng, Wangbo Yu, Junwu Zhang, Yuan Liu, Xiaoxiao Long, Wangping Wang, Li Yuan},
                title     = {NeuralGS: Bridging Neural Fields and 3D Gaussian Splatting for Compact 3D Representation},
                booktitle = {},
                month     = {March},
                month     = {February},
                year      = {2025},
                pages     = {}
            }