Refine LLM getting started guide for uniformity, fix critical errors by GregoryComer · Pull Request #2911 · pytorch/executorch

GregoryComer · 2024-04-07T10:47:51Z

Update the LLM getting started guide for uniform tone and tense. Informally following the Google developer documentation style guide: https://developers.google.com/style. Also, resolve a number of outstanding issues with incorrect or misleading documentation and steps.

For reference, here are links to the current and proposed LLM guide:
https://docs-preview.pytorch.org/pytorch/executorch/2911/llm/getting-started.html (proposed)
https://pytorch.org/executorch/main/llm/getting-started.html (live)

pytorch-bot · 2024-04-07T10:47:54Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/2911

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 678ba94 with merge base dc7e4d5 ():

NEW FAILURE - The following job has failed:

Apple / upload-frameworks-ios (gh)
Credentials could not be loaded, please check your action inputs: Could not load credentials from any providers

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2024-04-08T10:40:44Z

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2024-04-08T10:57:18Z

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

GregoryComer · 2024-04-08T11:00:28Z

 if [[ -z $PYTHON_EXECUTABLE ]];
 then
-  if [[ -z $CONDA_DEFAULT_ENV ]] || [[ $CONDA_DEFAULT_ENV == "base" ]];
+  if [[ -z $CONDA_DEFAULT_ENV ]] || [[ $CONDA_DEFAULT_ENV == "base" ]] || [[ ! -x "$(command -v python)" ]];


Not sure how this happened, but heard of a case where python wasn't on the system path while in a conda env. Something to do with env/shell conflicts on mac? This change checks to make sure python actually exists. If not, default to python3, which seems to be the sanest option.

GregoryComer · 2024-04-08T11:00:51Z

+  PIP_EXECUTABLE=pip
+else
+  PIP_EXECUTABLE=pip3
+fi


On mac when not under conda, pip may not exist. We should use pip3 when PYTHON_EXECUTABLE=python3.

facebook-github-bot · 2024-04-08T18:21:07Z

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

byjlw

I added a couple of comments, but overall this is way better and feel free to merge

byjlw · 2024-04-08T18:19:28Z


 ## Prerequisites

-Let’s start by getting an ExecuTorch environment:


this is better. I like not having external links for instructions.

byjlw · 2024-04-08T18:20:19Z

-2. If you’re new to ExecuTorch follow [these steps](https://pytorch.org/executorch/main/getting-started-setup.html#set-up-your-environment) to set up your environment.
+::::{tab-set}
+:::{tab-item} conda
+Instructions on installing miniconda can be [found here](https://docs.anaconda.com/free/miniconda).


can we eliminate the need for Conda.
Can you re-write this so that it uses env?

O looks like you have both options here. Perfect. I'd change the instructions just a tad so it calls out two different options up front before giving the conda instructions

byjlw · 2024-04-08T18:24:27Z

-```
-string  prompt  =  "Hello world!";
-```
+        // Sample the next token from the logits.


so much better!

kirklandsign · 2024-04-08T18:49:02Z

 ```

-Also ExecuTorch provides different backend support for mobile acceleration. Simply call `to_backend()` with the specific backend partitioner on edge_manager  during exportation. Take Xnnpack delegation as an example:
+To export, run the script with `python export.py` (or python3, as appropriate for your environment). It will generate a `nanogpt.pte` file in the current directory.


Should be export_nanogpt.py?

kirklandsign · 2024-04-08T18:50:52Z

-et_program = edge_manager.to_executorch()
-```
+```cpp
+// main.cpp


Do we plan to put them to the same piece?

kirklandsign · 2024-04-08T18:55:16Z

+Finally, ensure that the runner links against the `xnnpack_backend` target in CMakeLists.txt.

 ```
 add_executable(nanogpt_runner nanogpt_runner.cpp)


is it main.cpp?

Thanks. I updated the text here to be more clear.

kirklandsign · 2024-04-08T19:02:34Z

+TODO: This is a placeholder. Finish.
 ```
-vector<int64_t> outputs = generate(llm_model, tokens, sampler, /*target_output_length*/20);
+https://github.com/GregoryComer/et-tutorials/blob/quantization/nanogpt/managed_tensor.h


Use something like https://raw.githubusercontent.com/GregoryComer/et-tutorials/quantization/nanogpt/managed_tensor.h

kirklandsign · 2024-04-08T19:17:03Z

@@ -348,15 +487,19 @@ target_link_libraries(
    xnnpack_backend) # Link the XNNPACK backend


Let's tell user exactly what's the diff?

Also etdump is causing issue?

kirklandsign · 2024-04-08T19:19:12Z

+The `get_delegation_info()` method provides a summary of what happened to the model after the `to_backend()` call:
+
+```python
 from executorch.exir.backend.utils import get_delegation_info


which file?

Good catch. I'll update.

facebook-github-bot · 2024-04-08T19:38:20Z

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2024-04-08T21:01:33Z

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2024-04-08T21:07:38Z

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2024-04-08T21:23:37Z

@GregoryComer has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

…ytorch#2911) Summary: Update the LLM getting started guide for uniform tone and tense. Informally following the Google developer documentation style guide: https://developers.google.com/style. Also, resolve a number of outstanding issues with incorrect or misleading documentation and steps. For reference, here are links to the current and proposed LLM guide: https://docs-preview.pytorch.org/pytorch/executorch/2911/llm/getting-started.html (proposed) https://pytorch.org/executorch/main/llm/getting-started.html (live) Pull Request resolved: pytorch#2911 Reviewed By: Gasoonjia, byjlw Differential Revision: D55867181 Pulled By: GregoryComer

facebook-github-bot · 2024-04-09T01:24:39Z

@GregoryComer merged this pull request in 01bac3d.

GregoryComer · 2024-04-09T07:48:46Z

@pytorchbot cherry-pick --onto release/0.2 -c docs

…2911) Summary: Update the LLM getting started guide for uniform tone and tense. Informally following the Google developer documentation style guide: https://developers.google.com/style. Also, resolve a number of outstanding issues with incorrect or misleading documentation and steps. For reference, here are links to the current and proposed LLM guide: https://docs-preview.pytorch.org/pytorch/executorch/2911/llm/getting-started.html (proposed) https://pytorch.org/executorch/main/llm/getting-started.html (live) Pull Request resolved: #2911 Reviewed By: Gasoonjia, byjlw Differential Revision: D55867181 Pulled By: GregoryComer fbshipit-source-id: 5e865eaa4a0ae52845963b15c221a3d272431448 (cherry picked from commit 01bac3d)

pytorchbot · 2024-04-09T07:54:04Z

Cherry picking #2911

The cherry pick PR is at #2939

Details for Dev Infra team

Raised by workflow job

…2911) Summary: Update the LLM getting started guide for uniform tone and tense. Informally following the Google developer documentation style guide: https://developers.google.com/style. Also, resolve a number of outstanding issues with incorrect or misleading documentation and steps. For reference, here are links to the current and proposed LLM guide: https://docs-preview.pytorch.org/pytorch/executorch/2911/llm/getting-started.html (proposed) https://pytorch.org/executorch/main/llm/getting-started.html (live) Pull Request resolved: #2911 Reviewed By: Gasoonjia, byjlw Differential Revision: D55867181 Pulled By: GregoryComer fbshipit-source-id: 5e865eaa4a0ae52845963b15c221a3d272431448 (cherry picked from commit 01bac3d)

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 7, 2024

GregoryComer force-pushed the llm-doc-update-part-1 branch 2 times, most recently from ae6fe1a to b9cd8ff Compare April 8, 2024 10:39

GregoryComer force-pushed the llm-doc-update-part-1 branch from b9cd8ff to 8c1bd79 Compare April 8, 2024 10:58

GregoryComer changed the title ~~Refine LLM getting started guide for uniformity~~ Refine LLM getting started guide for uniformity, fix critical errors Apr 8, 2024

GregoryComer commented Apr 8, 2024

View reviewed changes

byjlw approved these changes Apr 8, 2024

View reviewed changes

kirklandsign reviewed Apr 8, 2024

View reviewed changes

GregoryComer force-pushed the llm-doc-update-part-1 branch 2 times, most recently from 9f75846 to 6996c48 Compare April 8, 2024 19:35

Gasoonjia approved these changes Apr 8, 2024

View reviewed changes

GregoryComer force-pushed the llm-doc-update-part-1 branch from 6996c48 to 6ba87b4 Compare April 8, 2024 21:00

Refine LLM getting started guide for uniformity

678ba94

GregoryComer force-pushed the llm-doc-update-part-1 branch from 6ba87b4 to 678ba94 Compare April 8, 2024 21:22

facebook-github-bot closed this in 01bac3d Apr 9, 2024

facebook-github-bot added the Merged label Apr 9, 2024

GregoryComer mentioned this pull request Apr 9, 2024

[v0.2.0] Release Tracker #2666

Closed

This was referenced Apr 25, 2024

fix llama readme #3339

Closed

disclaimer #3376

Closed


		## Prerequisites

		Let’s start by getting an ExecuTorch environment:

		@@ -348,15 +487,19 @@ target_link_libraries(
		xnnpack_backend) # Link the XNNPACK backend

Conversation

GregoryComer commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot Bot commented Apr 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/2911

❌ 1 New Failure

Uh oh!

facebook-github-bot commented Apr 8, 2024

Uh oh!

facebook-github-bot commented Apr 8, 2024

Uh oh!

GregoryComer Apr 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Apr 8, 2024

Uh oh!

byjlw left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Apr 8, 2024

Uh oh!

facebook-github-bot commented Apr 8, 2024

Uh oh!

facebook-github-bot commented Apr 8, 2024

Uh oh!

facebook-github-bot commented Apr 8, 2024

Uh oh!

facebook-github-bot commented Apr 9, 2024

Uh oh!

GregoryComer commented Apr 9, 2024

Uh oh!

pytorchbot commented Apr 9, 2024

Cherry picking #2911

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

GregoryComer commented Apr 7, 2024 •

edited

Loading

pytorch-bot Bot commented Apr 7, 2024 •

edited

Loading

GregoryComer Apr 8, 2024 •

edited

Loading