GitHub: Add bi-directional sync by savish28 · Pull Request #3565 · Cloud-CV/EvalAI

savish28 · 2021-08-12T00:12:08Z

Context: Github Bi-directional Sync

Related Pull Request: Cloud-CV/EvalAI-Starters#52

Models Considered:

Challenge
Challenge Phase

Deliverables:

Add github_token
Add sync for non-file fields
Add sync for file fields for challenge model
Add only updated fields commit

Testing without celery:
https://user-images.githubusercontent.com/32800267/129727539-80d18fd0-7cdc-489d-a3c6-0b346ce605c8.mp4

@KhalidRmb
cc: @Ram81

…ub_sync

Ram81 · 2021-08-23T03:54:05Z

+URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}
+
+
+class Github_Interface:


Please rename class to GithubInterface

Ram81 · 2021-08-24T15:34:25Z

+            return None
+        return response.json()
+
+    def return_github_url(self, url):


rename the method to get_github_url

Ram81 · 2021-08-24T15:36:30Z

+        url = "{0}{1}".format(base_url, url)
+        return url
+
+    def get_content_from_path(self, path):


Please add docstring for this method

Ram81 · 2021-08-24T15:36:35Z

+        return response
+
+    def get_data_from_path(self, path):
+        content_response = self.get_content_from_path(path)


Ram81 · 2021-08-24T15:36:42Z

+        return string_data
+
+    def update_content_from_path(self, path, content):
+        url = URLS.get("contents").format(self.GITHUB_REPOSITORY, path)


Ram81 · 2021-08-24T15:42:02Z

+
+@app.task
+def github_challenge_phase_sync(challenge_phase):
+    from .serializers import ChallengePhaseSerializer


move to global imports

Since it is a circular import.

Ram81 · 2021-08-24T15:42:19Z

+    from .serializers import ChallengePhaseSerializer
+
+    for obj in serializers.deserialize("json", challenge_phase):
+        challenge_phase_obj = obj.object


create a util method for deserializing and use that

Ram81 · 2021-08-24T15:42:32Z

+            "is_restricted_to_select_one_submission",
+            "is_partial_submission_evaluation_enabled",
+            "allowed_submission_file_types",
+        ]


move to a common util file

Ram81 · 2021-08-24T15:42:38Z

+            github.update_data_from_path("challenge_config.yaml", content_str)
+
+        # File fields Update
+        file_fields = ["description"]


Ram81 · 2021-08-24T15:44:04Z

+def github_sync_challenge(sender, instance, created, **kwargs):
+    if instance.github_repository and instance.github_token:
+        serialized_obj = serializers.serialize("json", [instance])
+        github_challenge_sync.delay(serialized_obj)


@savish28 the method names are confusing github_challenge_sync and github_sync_challenge (post save hook) it is not clear what these methods are for. Please rename the methods.

codecov-commenter · 2021-08-24T16:33:29Z

Codecov Report

Merging #3565 (2fd2db6) into master (96968d6) will decrease coverage by 1.40%.
The diff coverage is 37.15%.

@@            Coverage Diff             @@
##           master    #3565      +/-   ##
==========================================
- Coverage   72.93%   71.53%   -1.41%     
==========================================
  Files          83       20      -63     
  Lines        5368     3235    -2133     
==========================================
- Hits         3915     2314    -1601     
+ Misses       1453      921     -532

Impacted Files	Coverage Δ
frontend/src/js/controllers/authCtrl.js	`53.91% <6.38%> (-12.95%)`	⬇️
frontend/src/js/controllers/profileCtrl.js	`79.76% <20.00%> (-13.10%)`	⬇️
frontend/src/js/controllers/permissionCtrl.js	`36.36% <22.22%> (-63.64%)`	⬇️
frontend/src/js/controllers/challengeCtrl.js	`63.54% <37.00%> (-10.15%)`	⬇️
frontend/src/js/controllers/updateProfileCtrl.js	`82.55% <44.44%> (-10.30%)`	⬇️
frontend/src/js/controllers/challengeListCtrl.js	`95.74% <50.00%> (+1.06%)`	⬆️
...ntend/src/js/controllers/challengeHostTeamsCtrl.js	`70.50% <66.66%> (-1.18%)`	⬇️
frontend/src/js/controllers/teamsCtrl.js	`71.17% <75.00%> (ø)`
frontend/src/js/controllers/ChallengeInviteCtrl.js	`100.00% <100.00%> (ø)`
frontend/src/js/controllers/SubmissionFilesCtrl.js	`95.45% <100.00%> (ø)`
... and 30 more

Impacted Files	Coverage Δ
frontend/src/js/controllers/authCtrl.js	`53.91% <6.38%> (-12.95%)`	⬇️
frontend/src/js/controllers/profileCtrl.js	`79.76% <20.00%> (-13.10%)`	⬇️
frontend/src/js/controllers/permissionCtrl.js	`36.36% <22.22%> (-63.64%)`	⬇️
frontend/src/js/controllers/challengeCtrl.js	`63.54% <37.00%> (-10.15%)`	⬇️
frontend/src/js/controllers/updateProfileCtrl.js	`82.55% <44.44%> (-10.30%)`	⬇️
frontend/src/js/controllers/challengeListCtrl.js	`95.74% <50.00%> (+1.06%)`	⬆️
...ntend/src/js/controllers/challengeHostTeamsCtrl.js	`70.50% <66.66%> (-1.18%)`	⬇️
frontend/src/js/controllers/teamsCtrl.js	`71.17% <75.00%> (ø)`
frontend/src/js/controllers/ChallengeInviteCtrl.js	`100.00% <100.00%> (ø)`
frontend/src/js/controllers/SubmissionFilesCtrl.js	`95.45% <100.00%> (ø)`
... and 30 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2bc3487...2fd2db6. Read the comment docs.

RishabhJain2018 · 2021-09-08T22:42:53Z

@Ram81 @savish28 Is this PR ready for review?

savish28 · 2021-09-09T20:40:00Z

@RishabhJain2018 Yes it is!!

Ram81 · 2021-09-12T20:39:57Z

@@ -0,0 +1,37 @@
+# Fields from Challenge, ChallengePhase model to be considered for github_sync
+
+challenge_non_file_fields = [


@savish28 is there a way we can use the serializer or model to generate this list of fields? Currently, if a new field gets added we'll have to add it here manually. It might happen that some forgets adding it here which would mean there is no sync happening for that field. If we can use models/serializer for this list it would be great

@Ram81, we only wish to have only selected fields(that are editable from frontend) in the sync and not all fields, getting it from serializer would mean using all fields and that is undesirable because there are a lot of fields that are not in the challenge_config and might also be critical to share those.

Ram81 · 2021-09-12T20:40:56Z

+URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}
+
+
+class GithubInterface:


@savish28 can we move this class to a separate file called github_interface.py

Yeah sure :) Done.

Ram81 · 2021-09-12T20:42:20Z

+    if not github.is_repository():
+        return
+    try:
+        # Challenge Non-file field Update


Change to Challenge non-file field update

Ram81 · 2021-09-12T20:43:18Z

+            content_str = yaml.dump(challenge_config_yaml, sort_keys=False)
+            github.update_data_from_path("challenge_config.yaml", content_str)
+
+        # Challenge File fields Update


Change to Challenge file fields update

Ram81 · 2021-09-12T20:44:02Z

+                continue
+            github.update_data_from_path(field_path, getattr(challenge, field))
+    except Exception as e:
+        logger.info("Github Sync unsuccessful due to {}".format(e))


Why is this a info log? This should be a error log

Ram81 · 2021-09-12T20:48:19Z

+                    field_path, getattr(challenge_phase, field)
+                )
+            break
+


remove newline

Ram81 · 2021-09-12T20:48:31Z

+                github.update_data_from_path(
+                    field_path, getattr(challenge_phase, field)
+                )
+            break


why do we have a break here?

Because at a time only one challenge_phase is updated. Because it is initiated by post-hook of ChallengePhase

Ram81 · 2021-09-12T20:48:52Z

+            break
+
+    except Exception as e:
+        logger.info(


same here, Why is this not a error log?

Ram81 · 2021-09-12T20:49:29Z



+@receiver(signals.post_save, sender="challenges.Challenge")
+def challenge_sync(sender, instance, created, **kwargs):


rename to challenge_details_sync

Ram81 · 2021-09-12T20:49:46Z



+@receiver(signals.post_save, sender="challenges.ChallengePhase")
+def challenge_phase_sync(sender, instance, created, **kwargs):


change to challenge_phase_details_sync

savish28 · 2021-11-28T09:09:02Z

@Ram81 Ping! :)

Ram81 · 2021-11-28T15:08:58Z

Thanks for the ping @savish28. I will review this PR today

savish28 added 2 commits August 12, 2021 05:32

Github: add bi-directional sync non-file field challenge

4851c3d

Sync: Add challenge file fields

96af768

RishabhJain2018 requested a review from Ram81 August 17, 2021 19:07

savish28 added 4 commits August 19, 2021 14:03

Support compatibility with prev Evalai-starters

9c0585b

Merge branch 'master' of https://github.com/Cloud-CV/EvalAI into gith…

7b8f5ee

…ub_sync

Merge conflicts with master

43e3eec

Sync: challenge phase file and non file updates

4f90106

savish28 changed the title ~~GitHub: add bi-directional sync for non-file challenge field~~ GitHub: Add bi-directional sync Aug 20, 2021

Ram81 requested changes Aug 24, 2021

View reviewed changes

savish28 added 2 commits August 24, 2021 21:48

Doc: decouple utils and rename functions

33cb342

doc: rename model function methods

22d312c

savish28 added 2 commits August 24, 2021 22:07

doc: add docstrings in github_sync functions

5d71989

Remove circular imports

46e4a11

Ram81 requested changes Sep 12, 2021

View reviewed changes

savish28 and others added 3 commits September 13, 2021 02:39

Change model function names

8693a33

refactor code segregating github_inference

2fd2db6

Merge branch 'master' into github_sync

77986bf

Ram81 mentioned this pull request Feb 21, 2022

Easy challenge management on EvalAI Cloud-CV/GSoC-Ideas#46

Closed

10 tasks

RishabhJain2018 mentioned this pull request Jul 29, 2025

Rework and clarify bi‑directional GitHub sync feature (#3565) #4760

Open

This was referenced Dec 3, 2025

feat: Implement bi-directional GitHub sync for challenges (#4760) balaraj74/EvalAI#1

Merged

feat: Implement bi-directional GitHub sync for challenges (#4760) #4825

Open

		URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}


		class Github_Interface:

		@@ -0,0 +1,37 @@
		# Fields from Challenge, ChallengePhase model to be considered for github_sync

		challenge_non_file_fields = [

		URLS = {"contents": "/repos/{}/contents/{}", "repos": "/repos/{}"}


		class GithubInterface:



		@receiver(signals.post_save, sender="challenges.Challenge")
		def challenge_sync(sender, instance, created, **kwargs):



		@receiver(signals.post_save, sender="challenges.ChallengePhase")
		def challenge_phase_sync(sender, instance, created, **kwargs):

Uh oh!

Conversation

savish28 commented Aug 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Aug 24, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

RishabhJain2018 commented Sep 8, 2021

Uh oh!

savish28 commented Sep 9, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

savish28 commented Aug 12, 2021 •

edited

Loading

codecov-commenter commented Aug 24, 2021 •

edited

Loading