Skip to content

Feature/update git #353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
grallewellyn merged 4 commits into from
Nov 6, 2023
Merged

Feature/update git #353

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f1acbaa
Merge branch 'develop' of github.com:MAAP-Project/maap-documentation …
grallewellyn Sep 27, 2023
7316c7e
Merge branch 'develop' of github.com:MAAP-Project/maap-documentation …
grallewellyn Oct 18, 2023
4505e08
updated git section
grallewellyn Oct 25, 2023
a55dec1
added note about jupyter notebook checkpoints to .gitignore
grallewellyn Nov 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
0 ...e/getting_started/_static/clone_demo1.png → docs/source/_static/clone_demo1.png
View file
Open in desktop
File renamed without changes
0 ...e/getting_started/_static/clone_demo2.png → docs/source/_static/clone_demo2.png
View file
Open in desktop
File renamed without changes
0 ...e/getting_started/_static/clone_demo3.png → docs/source/_static/clone_demo3.png
View file
Open in desktop
File renamed without changes
0 ...e/getting_started/_static/clone_demo4.png → docs/source/_static/clone_demo4.png
View file
Open in desktop
File renamed without changes
0 ...e/getting_started/_static/clone_demo5.png → docs/source/_static/clone_demo5.png
View file
Open in desktop
File renamed without changes
0 ...e/getting_started/_static/clone_demo6.png → docs/source/_static/clone_demo6.png
View file
Open in desktop
File renamed without changes
0 ...e/getting_started/_static/clone_demo7.png → docs/source/_static/clone_demo7.png
View file
Open in desktop
File renamed without changes
0 ...ource/getting_started/_static/gitlab1.png → docs/source/_static/gitlab1.png
View file
Open in desktop
File renamed without changes
0 ...ource/getting_started/_static/gitlab2.png → docs/source/_static/gitlab2.png
View file
Open in desktop
File renamed without changes
0 ...ource/getting_started/_static/gitlab3.png → docs/source/_static/gitlab3.png
View file
Open in desktop
File renamed without changes
0 ...ource/getting_started/_static/gitlab4.png → docs/source/_static/gitlab4.png
View file
Open in desktop
File renamed without changes
Binary file added docs/source/_static/import_repository.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
143 changes: 13 additions & 130 deletions docs/source/getting_started/writing_code.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
".. note::\n",
Copy link
Collaborator

@wildintellect wildintellect Nov 6, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something is wrong and the image links aren't rendering in this notebook in the new section.


Reply via ReviewNB

Copy link
Collaborator

@wildintellect wildintellect Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They show up in the github preview but not the NB Review. The writing_code_overview.png shows up in both.

Copy link
Collaborator

@wildintellect wildintellect Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@grallewellyn difference seems to be ../ added to the front of the partially working images, should probably just be _static/...

Copy link
Member Author

@grallewellyn grallewellyn Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm I changed this because I moved all the clone_demo images from source/getting_started/_static/ to source/_static/ since I now also use the images in system_reference_guide
If I delete the build files and rebuild locally the images are still showing up
If I remove the ../ and delete the build files and rebuild locally the images don't show up
@wildintellect

Copy link
Contributor

@rtapella rtapella Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I build locally, the images are working for me

Screenshot 2023-11-06 at 11 15 47 AM

Copy link
Contributor

@rtapella rtapella Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am going to push this to the RtD site to test it out

Copy link
Contributor

@rtapella rtapella Nov 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.maap-project.org/en/feature-update-git/getting_started/writing_code.html

"If you have not used Github or git before, it is highly recommended that you [get acquainted with it](https://docs.github.com/en/get-started/quickstart/hello-world). For a quick reference to git commands there is a [Git Cheat Sheet](https://training.github.com/) in a variety of languages.\n",
"\n",
"To assist connections to the MAAP system from a Jupyter notebook, a helper library called `maap.py` provides Python-native calls to the underlying RESTful MAAP API. Often a separate Jupyter notebook is used to run and monitor jobs with API calls.\n",
"To assist connections to the MAAP system from a Jupyter notebook, a helper library called `maap.py` provides Python-native calls to the underlying RESTful MAAP API. Often a separate Jupyter notebook is used to run and monitor jobs with API calls. **When working with Jupyter notebooks, a manual save must be done to create a checkpoint.** Checkpoints are an emergency backup of the notebook and are different than the automatic saving of the notebook. \n",
"\n",
"![Writing code overview in context diagram](_static/writing_code_overview.png)"
]
Expand Down Expand Up @@ -59,149 +59,32 @@
"https://github.com/MAAP-Project/dps-unit-test\n",
"\n",
"1. Copy the Github clone link from https://github.com/MAAP-Project/dps-unit-test\n",
"![Copy .git link](_static/clone_demo2.png)\n",
"![Copy .git link](../_static/clone_demo2.png)\n",
"\n",
"2. Open the built-in Jupyter Github UI to the left of the file browser. Choose \"Clone a Repository\" and paste in the .git link you copied from the Github repository.\n",
"![Clone a Repository](_static/clone_demo3.png)\n",
"![Paste .git link](_static/clone_demo4.png)\n",
"2. Open the built-in Jupyter Github UI to the left of the file browser. Choose \"Clone a Repository\" and paste in the .git link you copied from the Github repository. You can also access this menu through the **Git** tab at the top of the Jupyter window. \n",
"![Clone a Repository](../_static/clone_demo3.png)\n",
"![Paste .git link](../_static/clone_demo4.png)\n",
"\n",
"3. You should see a new folder created with the repo you cloned. If you browse to that folder and open up the Jupyter Github UI again, it will show you some info about that repo.\n",
"![Algorithm folder was created](_static/clone_demo5.png)\n",
"![Browse to folder](_static/clone_demo6.png)\n",
"![Look at Github UI](_static/clone_demo7.png)\n",
"![Algorithm folder was created](../_static/clone_demo5.png)\n",
"![Browse to folder](../_static/clone_demo6.png)\n",
"![Look at Github UI](../_static/clone_demo7.png)\n",
"\n",
"4. If you want to make changes to the code and have your own copy of it to register, clone the code into a public repository in Github or in MAAP Gitlab."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### The MAAP GitLab Code repository\n",
"\n",
"After creating your MAAP account, you can create a code repository by navigating to the MAAP GitLab account at https://repo.maap-project.org. This GitLab account is connected to your ADE workspaces automatically when signing into the ADE.\n"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"This example walks through cloning a repository into the ADE. Cloning a repository allows you to open, edit, and run files contained within the cloned repository. In this example, we look at cloning the [\"MAAP-Project/maap-documentation\" Git repository](https://github.com/MAAP-Project/maap-documentation), so that you are able to experiment with the code examples contained within this user documentation."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"When inside of a workspace, navigate to **Git** tab at the top of the Jupyter window. Click it to see the option to **Clone**."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"![Git Clone](../_static/git_clone.png) "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"We can also access the \"Clone a repository\" dialogue box by selecting the **File Browser** ![File Browser](../_static/file_browser.png) tab on the JupyterLab sidebar, browsing to the location where we want our Git repository, and using the **Git** button located near the File Browser icon (also to the left of the file list) and choosing \"Clone a repository\". The dialogue box prompts you to enter the URI of the repository you wish to clone. For this example we enter \"https://github.com/MAAP-Project/maap-documentation.git\". \n",
"4. If you want to make changes to the code and have your own copy of it to register, clone the code into a public repository in Github or in MAAP Gitlab.\n",
"\n",
"For future reference, this URI can be found by visiting the [GitHub site for the \"MAAP-Project/maap-documentation\" Git repository](https://github.com/MAAP-Project/maap-documentation) and clicking the **Code** button. ![Code Button](../_static/code_button.png) "
"To open the IPython Notebook, go to a section directory and double-click on appropriate \".ipynb\" file. For more information about the using Git in Jupyterlab, see https://github.com/jupyterlab/jupyterlab-git ."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"With the **File Browser** tab on the JupyterLab sidebar selected, a folder named \"maap-documentation\" should now appear at the location where you did the Git Clone operation. Folders for the various sections of the guide can be found in the \"docs/source/\" directory. "
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"![docs/source/](../_static/docs_source.png)"
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"To open the IPython Notebook for an example, go to a section directory and double-click on appropriate \".ipynb\" file. For more information about the using Git in Jupyterlab, see https://github.com/jupyterlab/jupyterlab-git ."
]
},
{
"attachments": {},
"cell_type": "markdown",
"metadata": {},
"source": [
"### Working with the MAAP GitLab\n",
"\n",
".. note::\n",
" git can behave slowly and strangely over s3 bucket-based storage (i.e., `my-private-bucket` and `my-public-bucket`). It is recommended to set up your git-tracked repos on the root (somewhere inside of `~` or `/projects`).\n",
"\n",
"The MAAP GitLab instance is located at https://repo.maap-project.org/ . Make sure you can access this from the browser using your MAAP (EarthData Login) credentials.\n",
"\n",
"For NASA security reasons, MAAP cannot communicate with its GitLab instance over SSH. There also isn’t a username-password authentication option. Therefore, the recommended way to access MAAP repositories is to use GitLab Personal Access Tokens.\n",
"\n",
"1. In GitLab, in the top-right corner, click your user icon → “Preferences”\n",
"![Preferences](_static/gitlab1.png)\n",
"\n",
"2. In the “User settings” menu, navigate to “Access Tokens”.\n",
"![Access Tokens](_static/gitlab2.png)\n",
"\n",
"3. Create a new token with at least “read_repository” and “write_repository” permissions.\n",
"![New Token Configuration](_static/gitlab3.png)\n",
"\n",
"4. After clicking “create personal access token”, you’ll see a message like this pop up. Make sure you copy this token into a text file — you will not be able to access it again.\n",
"![Access Token Popup Message](_static/gitlab4.png)\n",
"\n",
"5. In the MAAP ADE, include this access token as part of the remote URL; e.g.,\n",
"\n",
"```\n",
"git clone https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"```\n",
"\n",
"For example:\n",
"\n",
"```\n",
"git clone https://ashiklom:JJVimxhV8nmRNDqcCNr7@repo.maap-project.org/ashiklom/fireatlas\n",
"````\n",
"\n",
"If you want to use multiple code repositories, it’s possible to configure a repository to have multiple remotes — e.g.,\n",
"\n",
"To add the `maap` remote and set the URL, use this:\n",
"```\n",
"git remote add maap https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"```\n",
"\n",
"If you already have the remote called `maap` set up, you can set the remote URL using this instead:\n",
"```\n",
"git remote set-url maap https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"```\n",
"\n",
"Then, you can use these commands to push your code and effectively synchronize between Github and MAAP GitLab (for algorithm registration):\n",
"### The MAAP GitLab Code repository\n",
"\n",
"Push to Github:\n",
"```\n",
"git push origin <branch name>\n",
"```\n",
"After creating your MAAP account, you can create a code repository by navigating to the MAAP GitLab account at https://repo.maap-project.org. This GitLab account is connected to your ADE workspaces automatically when signing into the ADE.\n",
"\n",
"Push to MAAP GitLab:\n",
"```\n",
"git push maap <branch name>\n",
"```"
"You can then follow the same steps above to clone a repository from the MAAP GitLab. \n"
]
},
{
Expand Down
156 changes: 153 additions & 3 deletions docs/source/system_reference_guide/work_with_git.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,159 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Committing Changes to a Git Repository\n",
"All projects imported from GitLab can be found in `/projects`. Instructions how to clone a repository from Git can be found [here](../getting_started/writing_code.html#Clone-a-Repository-with-GitHub) and instructions for working with MAAP GitLab can be found [here.](../getting_started/writing_code.html#Working-with-the-MAAP-GitLab)\n",
"# Working with Git"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Clone a Repository with GitHub\n",
"\n",
"Here is an example repository you can use for this getting started guide:\n",
"https://github.com/MAAP-Project/dps-unit-test\n",
"\n",
"1. Copy the Github clone link from https://github.com/MAAP-Project/dps-unit-test\n",
"![Copy .git link](../_static/clone_demo2.png)\n",
"\n",
"2. Open the built-in Jupyter Github UI to the left of the file browser. Choose \"Clone a Repository\" and paste in the .git link you copied from the Github repository. You can also access this menu through the **Git** tab at the top of the Jupyter window. \n",
"![Clone a Repository](../_static/clone_demo3.png)\n",
"![Paste .git link](../_static/clone_demo4.png)\n",
"\n",
"3. You should see a new folder created with the repo you cloned. If you browse to that folder and open up the Jupyter Github UI again, it will show you some info about that repo.\n",
"![Algorithm folder was created](../_static/clone_demo5.png)\n",
"![Browse to folder](../_static/clone_demo6.png)\n",
"![Look at Github UI](../_static/clone_demo7.png)\n",
"\n",
"4. If you want to make changes to the code and have your own copy of it to register, clone the code into a public repository in Github or in MAAP Gitlab.\n",
"\n",
"If you would like to create a template from an existing repository, create a new repository on GitHub where you have the option to \"Import a repository\" where you can supply the old repository git link. \n",
"\n",
"![Import a Repository](../_static/import_repository.png)\n",
"\n",
"Then, clone that new repository into the ADE via the steps above "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Working with the MAAP GitLab\n",
"\n",
".. note::\n",
" git can behave slowly and strangely over s3 bucket-based storage (i.e., `my-private-bucket` and `my-public-bucket`). It is recommended to set up your git-tracked repos on the root (somewhere inside of `~` or `/projects`).\n",
"\n",
"The MAAP GitLab instance is located at https://repo.maap-project.org/ . Make sure you can access this from the browser using your MAAP (EarthData Login) credentials.\n",
"\n",
"For NASA security reasons, MAAP cannot communicate with its GitLab instance over SSH. There also isn’t a username-password authentication option. Therefore, the recommended way to access MAAP repositories is to use GitLab Personal Access Tokens.\n",
"\n",
"1. In GitLab, in the top-right corner, click your user icon → “Preferences”\n",
"![Preferences](../_static/gitlab1.png)\n",
"\n",
"2. In the “User settings” menu, navigate to “Access Tokens”.\n",
"\n",
"<img src=\"../_static/gitlab2.png\" height=\"450\" alt=\"Access Tokens\">\n",
"\n",
"3. Create a new token with at least “read_repository” and “write_repository” permissions.\n",
"![New Token Configuration](../_static/gitlab3.png)\n",
"\n",
"4. After clicking “create personal access token”, you’ll see a message like this pop up. Make sure you copy this token into a text file — you will not be able to access it again.\n",
"![Access Token Popup Message](../_static/gitlab4.png)\n",
"\n",
"5. In the MAAP ADE, include this access token as part of the remote URL; e.g.,\n",
"\n",
"```\n",
"git clone https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"```\n",
"\n",
"For example:\n",
"\n",
"```\n",
"git clone https://ashiklom:JJVimxhV8nmRNDqcCNr7@repo.maap-project.org/ashiklom/fireatlas\n",
"````\n",
"\n",
"You can also clone the repository via the Git side tab and enter your personal access token instead of password when prompted. \n",
"![Git Side UI](../_static/clone_demo3.png)\n",
"\n",
"If you want to use multiple code repositories, it’s possible to configure a repository to have multiple remotes — e.g.,\n",
"\n",
"To add the `maap` remote and set the URL, cd into the repo and use this:\n",
"```\n",
"git remote add maap https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"```\n",
"\n",
"If you already have the remote called `maap` set up, you can set the remote URL using this instead:\n",
"```\n",
"git remote set-url maap https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"```\n",
"\n",
"Then, you can use these commands to push your code and effectively synchronize between Github and MAAP GitLab (for algorithm registration):\n",
"\n",
"Push to Github:\n",
"```\n",
"git push origin <branch name>\n",
"```\n",
"\n",
"Push to MAAP GitLab:\n",
"```\n",
"git push maap <branch name>\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initializing a new repository\n",
"**Note** It is recommended to create a new repository on Git, then clone it and paste files in, especially if you are starting from scratch and don't have any files yet \n",
"\n",
"1. Navigate to the folder you would like to make a Git repository \n",
"\n",
"2. Click the Git sidebar\n",
"![Git Side UI](../_static/clone_demo3.png) \n",
"\n",
"3. Select \"Initialize a Repository\"\n",
"\n",
"4. Create a new, empty repository on GitHub or MAAP GitLab instance with the same name as your folder \n",
"\n",
"5. Add a remote with the following code in your terminal\n",
"```\n",
"git remote add origin https://github.com/username/repo_name.git\n",
"git remote set-url origin https://github.com/username/repo_name.git\n",
"```\n",
"\n",
"Or if you are using MAAP GitLab\n",
"```\n",
"git remote add maap https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"git remote set-url maap https://username:AccessToken@repo.maap-project.org/username/repo_name\n",
"```\n",
"\n",
"And make code changes with \n",
"\n",
"```\n",
"git add .\n",
"git commit -m <message>\n",
"git push --set-upstream origin <branch>\n",
"```\n",
"And enter classic personal access token instead of password when prompted "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Committing Changes to a Git Repository\n",
"\n",
"\n",
"Before you update your project with your changes, you need to get a classic personal access token from GitHub. MAAP currently doesn't supported fine-grained tokens. Create a classic personal access token by following [these steps](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens#creating-a-personal-access-token-classic) in Developer settings. Remember to save your personal access token for later use. \n",
"\n",
"Jupyter notebook checkpoints bloat git unnecessarily, so you can add the following to your `.gitignore` file to prevent this:\n",
"```\n",
".ipynb_checkpoints\n",
"*/.ipynb_checkpoints/*\n",
"```\n",
"For more information on `.gitignore` files, see [here](https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files).\n",
"\n",
"Using the left side panel in the Jupyter interface, you can push changes to your Git project whether it is on the MAAP GitLab, or a public repository on GitHub. \n",
"\n",
"If you are more comfortable using the command line to interact with Git, you do not need to use the side panel. It will work the same way in the terminal, once you navigate to the project's filepath.\n",
Expand All @@ -21,10 +169,12 @@
"\n",
"<img src=\"../_static/update_project_push.png\" width=\"300\" height=\"150\" alt=\"update project push\">\n",
"\n",
"You will then be prompted for your Git username and personal access token that you got above \n",
"You then may be prompted for your Git username and personal access token that you got above \n",
"\n",
"<img src=\"../_static/update_project_git_login.png\" width=\"600\" height=\"150\" alt=\"update project git login\">\n",
"\n",
"You should then see \"Successfully pushed\"\n",
"\n",
"If you want to check your commit history, look at branches, and confirm your updates have been pushed, you can see this on the history tab.\n",
"\n",
"<img src=\"../_static/update_project_history.png\" width=\"220\" height=\"500\" alt=\"update project history\">"
Expand Down