Stylized Dynamic NeRFs

Stylized Dynamic NeRFs provide a framework for stylizing dynamic NeRFs. Dynamic NeRFs are created using the TiNeuVox model. Stylization is then performed using the Implicit Neural Stylization pipeline.

Installation

We recommend users to use conda to install the running environment. The following dependencies are required:

pytorch=1.7.0
torchvision=0.8.0
cudatoolkit=11.0
tensorboard=2.7.0
opencv
imageio
imageio-ffmpeg
configargparse
scipy
matplotlib
tqdm
mrc
lpips
torch_scatter
tkinter

Data Preparation

To run our code on NeRF dataset, users need first download data from the DNeRF dropbox. Then extract package files according to the following directory structure:

├── configs
│   ├── ...
│
├── datasets
|   ├── dnerf_synthetic
|   |   └── bouncingballs
|   |   └── hellwarrior    # downloaded synthetic dataset
|   |   └── ...

The last step is to generate and process data via our provided script:

python gen_dataset.py --config <config_file>

where <config_file> is the path to the configuration file of your experiment instance. Examples and pre-defined configuration files are provided in configs folder. Note that dynamic configs end in _dynamic.txt.

To evaluate/train using our checkpoints, download checkpoints from here and move them to the ckpts directory.

├── ckpts
|   └── *.ckpt
│   └── ...

Testing

After generating datasets and downloading checkpoints, one can evaluate the stylized NeRF using:

bash script/infer_dynamic.sh

Training

One can train the base dynamic NeRF using:

bash scripts/train_dynamic.sh

and stylize the base dynamic NeRF:

bash scripts/train_dynamic_with_teacher.sh

Ensure that ckpt_path points to the correct checkpoint of the teacher model.