Skip to content

Optimize Docker Build Layers and Add Sudo Privileges for Fast-LLM Container #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jlamypoirier merged 8 commits into from
Oct 21, 2024
Merged

Optimize Docker Build Layers and Add Sudo Privileges for Fast-LLM Container #2

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
f824139
refactor Dockerfile to separate dependency install and source code, p…
tscholak Oct 10, 2024
50de912
refactor Dockerfile to separate dependency install and source code, p…
tscholak Oct 10, 2024
9ae57e0
add util-linux
tscholak Oct 10, 2024
e868a6d
Merge branch 'main' into tscholak/tune_dockerfile
tscholak Oct 16, 2024
ade4b8e
address review comments
tscholak Oct 16, 2024
f24cef2
Merge branch 'tscholak/tune_dockerfile' of github.com:ServiceNow/Fast…
tscholak Oct 16, 2024
3c42057
address review comments
tscholak Oct 16, 2024
8219f7e
address review comments
tscholak Oct 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,37 @@
# syntax=docker/dockerfile:1.7-labs
FROM nvcr.io/nvidia/pytorch:24.07-py3

# git-lfs is needed to interact with the huggingface hub
# Install git-lfs for Huggingface hub interaction and sudo for system adjustments
RUN apt-get update \
&& apt-get install git-lfs \
&& apt-get install --no-install-recommends -y git-lfs sudo util-linux \
&& rm -rf /var/lib/apt/lists/* \
&& git lfs install

# Add a user for Fast-LLM with sudo privileges for runtime adjustments
ARG FAST_LLM_USER_ID=1000
RUN useradd -m -u $FAST_LLM_USER_ID -s /bin/bash fast_llm \
Copy link
Copy Markdown
Collaborator

@jlamypoirier jlamypoirier Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🥇

&& echo 'fast_llm ALL=(ALL) NOPASSWD: ALL' >> /etc/sudoers

RUN useradd -m -u $FAST_LLM_USER_ID -s /bin/bash -d /home/fast_llm fast_llm
USER fast_llm
WORKDIR /app
ENV PYTHONPATH=/app:/app:/app/Megatron-LM
ENV PATH=$PATH:/app:/home/fast_llm/.local/bin/

COPY --chown=fast_llm setup.py setup.cfg ./
COPY --chown=fast_llm fast_llm/__init__.py ./fast_llm/
# Environment settings for Python and PATH
ENV PYTHONPATH=/app:/app/Megatron-LM \
PATH=$PATH:/home/fast_llm/.local/bin/

# Copy the dependency files and install dependencies
COPY --chown=fast_llm setup.py setup.cfg pyproject.toml ./
Copy link
Copy Markdown
Collaborator

@jlamypoirier jlamypoirier Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why adding the toml? It's not used for the installation.

Copy link
Copy Markdown
Collaborator Author

@tscholak tscholak Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it appeared to me that the setup.* files and pyproject.toml fulfil similar purposes and could be grouped together. it is not uncommon to specify dependencies in the pyproject.toml file, because this is how setup tools usually works.

Copy link
Copy Markdown
Collaborator

@jlamypoirier jlamypoirier Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't do that in fast-llm though, the toml file is just there because black needs it.

Copy link
Copy Markdown
Collaborator Author

@tscholak tscholak Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I'll split it up then and copy pyproject.toml somewhere else

Copy link
Copy Markdown
Collaborator

@jlamypoirier jlamypoirier Oct 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really think it's worth copying at all... But if we do keep it let's keep it here so we don't add another line

RUN PIP_NO_INPUT=1 pip3 install --no-cache-dir -e ".[CORE,OPTIONAL,DEV]"
Copy link
Copy Markdown
Collaborator

@jlamypoirier jlamypoirier Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this back above the crsc compile, since this installation is much longer.

Copy link
Copy Markdown
Collaborator Author

@tscholak tscholak Oct 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


COPY --chown=fast_llm fast_llm/csrc/ ./fast_llm/csrc/
# Compile the C++ extensions (fast_llm/csrc)
COPY --chown=fast_llm ./fast_llm/csrc/ fast_llm/csrc/
RUN make -C ./fast_llm/csrc/

# Copy the rest of the code
COPY --chown=fast_llm ./Megatron-LM Megatron-LM
COPY --chown=fast_llm ./examples examples
COPY --chown=fast_llm ./tests tests
COPY --chown=fast_llm ./tools tools
COPY --chown=fast_llm ./fast_llm fast_llm
COPY --chown=fast_llm fast_llm/tools/train.py pyproject.toml ./

# Copy the main source code for Fast-LLM and install in editable mode
COPY --exclude=./fast_llm/csrc/ --chown=fast_llm ./fast_llm/ fast_llm/
Comment thread
tscholak marked this conversation as resolved.