Skip to content

Fast Simple Parallel Gzip NCD Calculation#1

Open
BAW2501 wants to merge 1 commit intoSentdex:mainfrom
BAW2501:main
Open

Fast Simple Parallel Gzip NCD Calculation#1
BAW2501 wants to merge 1 commit intoSentdex:mainfrom
BAW2501:main

Conversation

@BAW2501
Copy link

@BAW2501 BAW2501 commented Jul 29, 2023

Refactored code to improve performance and readability

  • Using the joblib.Parallel library to parallelize the computation of the NCD matrix.
  • Simpler logic for NCD calculation
  • Removing unnecessary comments and improving the formatting of the code.
  • tested code speed on 4 core machine

time: 8.31 s
vs
time: 39 s for video non Parallel code
vs
time: 16.3 s for video Parallel code

would love to see it validated using the 10k data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant