Merge devel into master#2187
Merged
Merged
Conversation
I just realized that `--coverage` is only passed to the UT executable but not libraries. This PR fixes it, and the codecov report should be updated. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Set `LAMMPS_VERSION_NUMBER` during `make yes-user-deepmd`. Fixes deepmodeling#1947. The corresponding CMake argument is deprecated.
This PR just fixes a typo in documentation. Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
Fixes deepmodeling#1958. There was an ImportError from Python when TF and TF's dependency were not installed in the same directory, and one built with PEP 517. The reason is that only TF's parent directory is added to `sys.path`. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
…eling#1961) Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Upload the finetune api. Add support for mixed_type when dp test.
It was broken by deepmodeling#1793.
fixes deepmodeling#1962 Co-authored-by: Han Wang <wang_han@iapcm.ac.cn> Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
Coverage for C++ libraries will be counted when testing Python. It's expected to see the codecov coverage increases. Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
Fixes deepmodeling#1933. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Fixes deepmodeling#1957. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
) Follow-up of deepmodeling#1975 to fix deepmodeling#1957. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu> Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu> Co-authored-by: Han Wang <92130845+wanghan-iapcm@users.noreply.github.com>
Fixes deepmodeling#1957. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
…g#1983) Otherwise it returns zero atomic virials.
# Patching CVE-2007-4559 Hi, we are security researchers from the Advanced Research Center at [Trellix](https://www.trellix.com). We have began a campaign to patch a widespread bug named CVE-2007-4559. CVE-2007-4559 is a 15 year old bug in the Python tarfile package. By using extract() or extractall() on a tarfile object without sanitizing input, a maliciously crafted .tar file could perform a directory path traversal attack. We found at least one unsantized extractall() in your codebase and are providing a patch for you via pull request. The patch essentially checks to see if all tarfile members will be extracted safely and throws an exception otherwise. We encourage you to use this patch or your own solution to secure against CVE-2007-4559. Further technical information about the vulnerability can be found in this [blog](https://www.trellix.com/en-us/about/newsroom/stories/research/tarfile-exploiting-the-world.html). If you have further questions you may contact us through this projects lead researcher [Kasimir Schulz](mailto:kasimir.schulz@trellix.com).
Same as deepmodeling#1647, but a function was missing. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Strange, I added duplicated `convert_13_to_21` in deepmodeling#1597. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Clang is the default compiler for macos. For errors see also https://docwiki.embarcadero.com/RADStudio/Sydney/en/Explicit_instantiation_of_%27namespace::templated_class%27_must_occur_in_namespace_%27namespace%27_(C%2B%2B) Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
This is also aimed at resolving macos clang errors: 1. link to protobuf if it's used and separated; 2. add link flags to lmp lib. 3. set rpath; remove gnu specific flags; Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
…eling#2000) Add a simple test for wheels. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Just add quotes for cmake empty variables. See https://stackoverflow.com/a/52481212/9567349 Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
TF 2.9 removes the tf_cxx11_abi_flag function. The previous implementation in deepmodeling#1736 was wrong as `_GLIBCXX_USE_CXX11_ABI` is set by the compiler but not the TF header... Here I propose a new way - try to compile against a TF function that has `std::string` with both two flags. This commit also merges some duplicated codes.
…epmodeling#2012) Fixes deepmodeling#1400. Fixes deepmodeling#2009. 1. Uses cmake native module `CMakeDetermineHIPCompiler` to find the search path; 2. for ROCm>=3.5.1, `hip-hcc hiprtc` is replaced by `amd_comgr amdhip64`, per ROCm/ROCm#1200. (I am not sure about the situation of `amd_comgr`?) 3. Removes `-hc` from the flag for ROCm>=3.5.1. 4. Bumps from C++11 to C++14 as C++ 14 required by `amd_comgr`. 5. Removes `--amdgpu-target=gfx906`. I don't see the reason why it is in the flag. 6. Fixes a typo in deepmodeling#1866. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
…2029) add two output files deepmodeling#2002
This commit allows loading float models using the double interface, or the other way around. The model precision is perceived from the type of `descrpt_attr/rcut`. All `run_model` and `session_input_tensors` are rewritten as templates. This is an essential step of deepmodeling#1948. Next, functions in the whole module can be migrated to templates.
On CPUs, the batch size will not be increased anymore. This PR also introduces an environment variable `DP_INFER_BATCH_SIZE`. It can be used to override the initial batch size. To clarify: higher batch size also has better performance on CPUs. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
These files have been compiled in libdeepmd.so, so they don't need to compile again.
…#2151) After I enable it, I can observe speedup in some cases. For `examples/water/se_e2_a`, compress training FP32 networks: ```sh CUDA_VISIBLE_DEVICES= OMP_NUM_THREADS=8 TF_INTRA_OP_PARALLELISM_THREADS=8 TF_INTER_OP_PARALLELISM_THREADS=2 dp train input.json -f frozen_model_compressed.pb ``` Before enabling it, ``` DEEPMD INFO batch 100 training time 1.67 s, testing time 0.03 s DEEPMD INFO batch 200 training time 1.25 s, testing time 0.03 s DEEPMD INFO batch 300 training time 1.23 s, testing time 0.03 s DEEPMD INFO batch 400 training time 1.24 s, testing time 0.03 s DEEPMD INFO batch 500 training time 1.24 s, testing time 0.03 s DEEPMD INFO batch 600 training time 1.23 s, testing time 0.03 s DEEPMD INFO batch 700 training time 1.24 s, testing time 0.03 s DEEPMD INFO batch 800 training time 1.24 s, testing time 0.03 s DEEPMD INFO batch 900 training time 1.24 s, testing time 0.03 s DEEPMD INFO batch 1000 training time 1.24 s, testing time 0.03 s ``` After enabling it, ``` DEEPMD INFO batch 100 training time 1.60 s, testing time 0.03 s DEEPMD INFO batch 200 training time 1.10 s, testing time 0.03 s DEEPMD INFO batch 300 training time 1.10 s, testing time 0.03 s DEEPMD INFO batch 400 training time 1.10 s, testing time 0.03 s DEEPMD INFO batch 500 training time 1.11 s, testing time 0.03 s DEEPMD INFO batch 600 training time 1.11 s, testing time 0.03 s DEEPMD INFO batch 700 training time 1.10 s, testing time 0.03 s DEEPMD INFO batch 800 training time 1.10 s, testing time 0.03 s DEEPMD INFO batch 900 training time 1.10 s, testing time 0.03 s DEEPMD INFO batch 1000 training time 1.13 s, testing time 0.03 s ``` About 10% improvement. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Fix deepmodeling#2048. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Added a note about loading the model. Signed-off-by: LiangWenshuo1118 <73432685+LiangWenshuo1118@users.noreply.github.com>
This patch supports DPRc + type embedding, including
1. refactor `exclude_types` + `type_one_side`. Considering
`exclude_types` is equivalent to $\forall i,j \in \text{exclude types},
\frac{\partial{E}}{\partial{r_{ij}}}=0$, a mask is created to apply to
the environmental matrix, which is faster than the previous
implementation in deepmodeling#1423.
2. implement `atom_ener` + type embedding.
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
This PR moves energy biases out of NN for all situations and stores them with interface precision. When using the FP64 precision interface and FP32 precision NN, this patch can improve the accuracy of the atomic energy when it has a large absolute value. For example, when the atomic energy is 11451.41234567 eV, the FP32 value is 11451.412 eV (places=3); but if we have an FP64 bias of 11450.000000 eV, the NN only needs to fit 1.41234567 eV, and the FP32 value is 1.4123456 eV (places=7). Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Fix deepmodeling#2015. Fix deepmodeling#2039. test log: https://gist.github.com/njzjz/7ebb2a9c6ac60f12475a084a0222dbc1 Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Our wheels do not link against any Python ABIs. The libraries only link against TensorFlow ABIs. So it's safe to retag them to `py37-none-<platform>` but keep the platform tag. We only need to build the wheel once for a platform to reduce the CI time, and the wheel should work for all Python versions. References: [PEP 425](https://peps.python.org/pep-0425/) Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu> Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
With deepmodeling#2180, now it only has 4 tasks to build wheels. But this task is still so slow.. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
…pmodeling#2185) In some machines, this is required for Python. Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
## add lammps module to the wheel In the next version, one can install DeePMD-kit using ```sh pip install deepmd-kit[gpu,lmp] ``` Then one can use `lmp` to run the LAMMPS program with `pair deepmd`. This works for Linux and macOS, and requires `libpython` installed to load TF libraries. The LAMMPS Python package was prebuilt in [my repo](https://github.com/njzjz/lammps-wheel). ## Build a Python package with LAMMPS module from the source ```sh export DP_LAMMPS_VERSION=stable_23Jun2022_update2 export DP_variant=cuda pip install -v . ``` But note that the same compilation condition (CXX ABI, MPI) should be used to build LAMMPS and its plugin. ## Other fixes - rename `op_abi` module to `deepmd_op` to share the same name between Python and C++ - fix macOS OP library suffix in api_cc: `libdeepmd_op.dylib` -> `libdeepmd_op.so` - fix Windows dlopen APIs in api_cc - fix compilation error on windows: `an explicit specialization or instantiation of a function template cannot have any default arguments` - avoid linking libdeepmd_lmp with the MPI library, which has been linked with the main LAMMPS Signed-off-by: Jinzhe Zeng <jinzhe.zeng@rutgers.edu>
Codecov ReportBase: 78.16% // Head: 74.86% // Decreases project coverage by
Additional details and impacted files@@ Coverage Diff @@
## master #2187 +/- ##
==========================================
- Coverage 78.16% 74.86% -3.31%
==========================================
Files 118 202 +84
Lines 9876 20011 +10135
Branches 0 1433 +1433
==========================================
+ Hits 7720 14982 +7262
- Misses 2156 4129 +1973
- Partials 0 900 +900
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. ☔ View full report at Codecov. |
njzjz
approved these changes
Dec 19, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
10 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.