Skip to content

[CI] Fix Tune Runner VM memory.swappiness no such file or directory - #13252

Merged
hangc0276 merged 1 commit into
apache:masterfrom
Demogorgon314:fix/memory.swappiness_No_such_file_or_directory
Dec 14, 2021
Merged

[CI] Fix Tune Runner VM memory.swappiness no such file or directory#13252
hangc0276 merged 1 commit into
apache:masterfrom
Demogorgon314:fix/memory.swappiness_No_such_file_or_directory

Conversation

@Demogorgon314

Copy link
Copy Markdown
Member

Failed CI: https://github.com/apache/pulsar/runs/4501006108?check_suite_focus=true#step:3:1

Motivation

The Tune Runner VM is flaky.

Sometimes, the parent directory of memory.swappiness can't find, we should check it first.

Run if [[ "$OSTYPE" == "linux-gnu"* ]]; then
10.1.0.235	fv-az198-145.gjkxsqtjg41etoza45cw35jkyb.bx.internal.cloudapp.net fv-az198-145
1
tee: /sys/fs/cgroup/memory/system.slice/snap-snapd-13640.mount/memory.swappiness: No such file or directory
Error: Process completed with exit code 1.

Modifications

Add a check of the parent directory of memory.swappiness

Documentation

  • no-need-doc

@github-actions github-actions Bot added the doc-not-needed Your PR changes do not impact docs label Dec 13, 2021
@Demogorgon314

Copy link
Copy Markdown
Member Author

/pulsarbot run-failure-checks

1 similar comment
@Demogorgon314

Copy link
Copy Markdown
Member Author

/pulsarbot run-failure-checks

@codelipenghui

Copy link
Copy Markdown
Contributor

@timmyyuan Please help review this PR.

@hangc0276
hangc0276 merged commit 26ac049 into apache:master Dec 14, 2021
@Demogorgon314
Demogorgon314 deleted the fix/memory.swappiness_No_such_file_or_directory branch December 14, 2021 04:52
@eolivelli eolivelli added cherry-picked/branch-2.9 Archived: 2.9 is end of life release/2.9.1 labels Dec 15, 2021
@lhotari

lhotari commented Mar 23, 2022

Copy link
Copy Markdown
Member

This was the wrong solution to the problem. The change makes the shell script basically a no-op.

The correct solution to the problem would have been to set shopt -s nullglob.

Here's an example from the Pulsar CI refactoring:

            (
              shopt -s nullglob
              # Set swappiness to 1 for all cgroups and sub-groups
              for swappiness_file in /sys/fs/cgroup/memory/*/memory.swappiness /sys/fs/cgroup/memory/*/*/memory.swappiness; do
                echo 1 | sudo tee $swappiness_file > /dev/null
              done
            )

I'll be including this fix as part of the Pulsar CI refactoring that I have announced on the dev mailing list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cherry-picked/branch-2.9 Archived: 2.9 is end of life doc-not-needed Your PR changes do not impact docs release/2.9.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants