DEEPMD WARNING about freezing model when checkpoint is not old

[asedova]

Bug summary

Using the exact same build (pip install) of deepmd (v 2.2.9) that is using Tensorflow 2.15 and cuda 12, I train and then freeze a model without changing my environment at all.

I get the following DEEPMD WARNING that doesn't make sense to me:
DEEPMD WARNING The following nodes are not in the graph: {'train_attr/min_nbor_dist', 'fitting_attr/aparam_nall', 'spin_attr/ntypes_spin'}. Skip freezeing these nodes. You may be freezing a checkpoint generated by an old version.

The checkpoint was definitely not created by an older version but apparently something is failing the test here:

deepmd-kit/deepmd/entrypoints/freeze.py

Line 359 in b875ea8

different_set = set(output_node) - set(input_node)

DeePMD-kit Version

2.2.9

TensorFlow Version

2.15

How did you download the software?

pip

Input Files, Running Commands, Error Log, etc.

it's just dp train input.json --skip-neighbor-stat immediately followed by dp freeze -o graph.pb

System: NVIDIA A100 GPU, x86 AMD EPYC 7513 CPU, OS: Ubuntu 22.04

Steps to Reproduce

run any training then run freeze to freeze the model

Further Information, Files, and Links

none

[njzjz]

It's not a bug, but the warning message needs to be updated.

[asedova]

I figured-- wasn't sure of the best label

[Duanmu37]

Hi,
I encountered a similar issue when freezing the model, and I would greatly appreciate your help. Could you share your experience on how you resolved it? Any assistance to resolve this issue would be greatly appreciated.

2024-06-04 20:14:20.163089: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
WARNING:tensorflow:From /mnt/sg001/home/fz_licp_bzy/deepmd-kit/virtualenv/tensorflow/lib/python3.10/site-packages/tensorflow/python/compat/v2_compat.py:98: disable_resource_variables (from tensorflow.python.ops.resource_variables_toggle) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term
WARNING:tensorflow:Your environment has TF_USE_LEGACY_KERAS set to True, but you do not have the tf_keras package installed. You must install it in order to use the legacy tf.keras. Install it via: pip install tf_keras
2024-06-04 20:14:53.211848: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:388] MLIR V1 optimization pass is not enabled
DEEPMD WARNING The following nodes are not in the graph: {'fitting_attr/aparam_nall', 'spin_attr/ntypes_spin'}. Skip freezeing these nodes. You may be freezing a checkpoint generated by an old version.
DEEPMD INFO The following nodes will be frozen: ['o_atom_virial', 'descrpt_attr/rcut', 'fitting_attr/daparam', 'o_virial', 'descrpt_attr/ntypes', 'model_attr/model_version', 'o_atom_energy', 'model_attr/tmap', 'train_attr/training_script', 'o_energy', 'fitting_attr/dfparam', 'model_attr/model_type', 'train_attr/min_nbor_dist', 't_mesh', 'o_force', 'model_type']
WARNING:tensorflow:From /mnt/sg001/home/fz_licp_bzy/deepmd-kit/virtualenv/tensorflow/lib/python3.10/site-packages/deepmd/entrypoints/freeze.py:371: convert_variables_to_constants (from tensorflow.python.framework.convert_to_constants) is deprecated and will be removed in a future version.
Instructions for updating:
This API was designed for TensorFlow v1. See https://www.tensorflow.org/guide/migrate for instructions on how to migrate your code to TensorFlow v2.
WARNING:tensorflow:From /mnt/sg001/home/fz_licp_bzy/deepmd-kit/virtualenv/tensorflow/lib/python3.10/site-packages/tensorflow/python/framework/convert_to_constants.py:946: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This API was designed for TensorFlow v1. See https://www.tensorflow.org/guide/migrate for instructions on how to migrate your code to TensorFlow v2.
DEEPMD INFO 1192 ops in the final graph.