Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 108 additions & 24 deletions packaging/rpm/make-microshift-app-images-rpm.sh
Original file line number Diff line number Diff line change
@@ -1,36 +1,102 @@
#!/usr/bin/env bash
set -eu

# print_usage prints help
function print_usage() {
>&2 echo "Usage: $0 [options]
$0 creates an RPM from from user specified container image(s). This enables a user to prepackage
container images in an immutable ostree OS environment where running scripts and installers may
not be an option. The ability to pre-stage images helps facilitate air-gapped installation
scenario where the node does not have connectivity to a container registry or has security policies
that do not allow for any changes to the host OS.

The following parameters are all required:
--image-list -i FILE file path and name of a file containing user images, one image per line in the file
--container-dir -c DIR container storage directory path where the RPM will place the user image
--rpmbuild-dir -r DIR path to the rpm build directory (will be created if it does not exist)

Example usage:
sudo ./make-rpm.sh \\
--image-list /<path>/images.txt \\
--container-dir /var/lib/containers/storage/overlay-images \\
--rpmbuild-dir /home/<user>/rpmbuild
"
}

# file_exists checks to see if a file exists
file_exists() {
local f="$1"
stat "$f" &>/dev/null
}

echo 'Checking Dependencies:'
for CMD in rpmbuild crio podman; do
printf '%-10s' "${CMD}"
if hash "${CMD}" 2>/dev/null; then
echo found
else
echo missing
CMD_MISSING=true
fi
done

if [[ "${CMD_MISSING-}" ]]; then
echo 'error: please install the missing dependencies and re-run the app'
exit 1
fi

# First arg: file path containing user images per line
# Second arg: container storage dir path
# Third arg: RPMBUILD_DIR
# parse user specified arguments
RPMBUILD_DIR=
IMAGES=
IMG_DIR=
while true; do
case "${1:-}" in
-i | --image-list )
IMAGES="$2"; shift 2 ;;
-c | --container-dir )
IMG_DIR="$2"; shift 2 ;;
-r | --rpmbuild-dir )
RPMBUILD_DIR="$2"; shift 2 ;;
*) break ;;
esac
done

RPMBUILD_DIR=$3
_img_dir_=$2
# Verify that all required options were specified.
if [[ -z "${IMAGES}" ]]; then print_usage; echo '--image-list option is required.'; exit 1; fi
if [[ -z "${IMG_DIR}" ]]; then print_usage; echo '--container-dir option is required.'; exit 1; fi
if [[ -z "${RPMBUILD_DIR}" ]]; then print_usage; echo '--rpmbuild-dir option is required.'; exit 1; fi

# if the file that would contain user images doesn't exist,
# exit to ensure user is passing a file and not the image name
if (! file_exists "${IMAGES}"); then
print_usage; echo "error: required file containing a list of images was not found at ${IMAGES}"; exit 1
fi

declare -a ARRAY

#link filedescriptor 10 with stdin (standard input)
# link filedescriptor 10 with stdin (standard input)
exec 10<&0

#stdin replaced with a file supplied as a first argument
exec < $1
let count=0
# stdin replaced with a file supplied as a first argument
exec < "$IMAGES"
count=0

#read user images into ARRAY
# read user images into ARRAY
while read LINE; do
ARRAY[$count]=$LINE
((count++))
((count+1))
done

#restore stdin from file descriptor 10 then close filedescriptor 10
# restore stdin from file descriptor 10 then close filedescriptor 10
exec 0<&10 10<&-

#Generate microshift-app-images.spec
# generate microshift-app-images.spec
touch ./microshift-app-images.spec
cat >./microshift-app-images.spec <<EOF
%global _img_dir $_img_dir_
%global imageStore %{buildroot}%{_img_dir}
%global _IMAGES_ ${ARRAY[@]}
%define __arch_install_post %{nil}

@nerdalert nerdalert Apr 1, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This skips post install checks that is currently breaking the rpm build on RHEL 8.5

%global imgDir $IMG_DIR
%global imageStore %{buildroot}%{imgDir}
%global IMAGES ${ARRAY[@]}

Name: microshift-app-images
Version: 1
Expand All @@ -48,7 +114,7 @@ This rpm creates a RO container storage for user applications, pull the app imag

%prep

if [ -d %{imageStore} ]
if [ -d %{imageStore} ]
then
sudo rm -rf %{imageStore}
fi
Comment on lines 117 to 120

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an FYI, this block fails consistently, both on main branch and here, with:

+ '[' -d /tmp/ushift-rpm-package//BUILDROOT/microshift-app-images-1-1.x86_64/home/jcope/.local/share/containers/storage/overlay-images ']'
+ sudo rm -rf /tmp/ushift-rpm-package//BUILDROOT/microshift-app-images-1-1.x86_64/home/jcope/.local/share/containers/storage/overlay-images
rm: cannot remove '/tmp/ushift-rpm-package//BUILDROOT/microshift-app-images-1-1.x86_64/home/jcope/.local/share/containers/storage/overlay-images/overlay': Device or resource busy
error: Bad exit status from /var/tmp/rpm-tmp.Y9aEP7 (%prep)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.Y9aEP7 (%prep)

I'll open an issue for this. cc @mangelajo

@nerdalert nerdalert Apr 1, 2022

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @copejon!

I dug around on this. Reason I missed it is because it doesn't show up on Fedora35 but does on RHEL8.5. I'm guessing its the rpmbuild versions (Fedora RPM version 4.17.0) vs. (RHEL RPM version 4.14.3) but I'm having to knock off some rust on RPM building so that's a guess.

  • Here is the buildroot exit:
Binary file /home/brent/rpmbuild/BUILDROOT/microshift-app-images-1-1.x86_64/var/lib/containers/storage/overlay-images/libpod/bolt_state.db matches
Found '/home/brent/rpmbuild/BUILDROOT/microshift-app-images-1-1.x86_64' in installed files; aborting
error: Bad exit status from /var/tmp/rpm-tmp.S0yRBl (%install)

In the meantime, an option is to skip the postinstall buildroot check with %define __arch_install_post %{nil} to get things working in master until we figure out the buildroot issue or if it isn't urgent maybe someone else might have an idea. I'm also chasing down an issue with building some different images that aren't resolving dependancies that I haven't nailed down yet.

Here is output from the build and install if you want to diff it with your output.
Ty!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As an FYI, this block fails consistently, both on main branch and here, with:

+ '[' -d /tmp/ushift-rpm-package//BUILDROOT/microshift-app-images-1-1.x86_64/home/jcope/.local/share/containers/storage/overlay-images ']'
+ sudo rm -rf /tmp/ushift-rpm-package//BUILDROOT/microshift-app-images-1-1.x86_64/home/jcope/.local/share/containers/storage/overlay-images
rm: cannot remove '/tmp/ushift-rpm-package//BUILDROOT/microshift-app-images-1-1.x86_64/home/jcope/.local/share/containers/storage/overlay-images/overlay': Device or resource busy
error: Bad exit status from /var/tmp/rpm-tmp.Y9aEP7 (%prep)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.Y9aEP7 (%prep)

I'll open an issue for this. cc @mangelajo

If you run as sudo I think that error should get resolved.

Expand All @@ -59,7 +125,7 @@ fi

mkdir -p %{imageStore}

declare -a ListOfImages=(%{_IMAGES_})
declare -a ListOfImages=(%{IMAGES})


for val in \${ListOfImages[@]}; do
Expand All @@ -69,15 +135,33 @@ sudo chmod -R a+rx %{imageStore}


%post
sudo sed -i '/^additionalimagestores =*/a "$_img_dir_",' /etc/containers/storage.conf
# if crio was already started, restart it so it read from new imagestore
systemctl is-active --quiet crio && systemctl restart --quiet crio

# only on install (1), not on upgrades (2)
if [ \$1 -eq 1 ]; then
sed -i '/^additionalimagestores =*/a "%{imgDir}",' /etc/containers/storage.conf
# if crio was already started, restart it so it reads from the new imagestore
systemctl is-active --quiet crio && systemctl restart --quiet crio || :
fi

%files
%{_img_dir}/*
%{imgDir}/*

%postun
# only on uninstall (0), not on upgrades(1)
if [ \$1 = 0 ]; then
sed -i '\\:"%{imgDir}":d' /etc/containers/storage.conf
systemctl is-active --quiet crio && systemctl restart --quiet crio || :
fi

EOF
cp ./microshift-app-images.spec $RPMBUILD_DIR/SPECS/microshift-app-images.spec

# if the target RPM build directory or directory structure doesn't exist, create it, exit if creation fails
if (! file_exists "${RPMBUILD_DIR}"/SPECS); then
echo "RPM build directory ${RPMBUILD_DIR} does not exist, attempting to create it"
mkdir -p "${RPMBUILD_DIR}"/{BUILD,BUILDROOT,RPMS,SOURCES,SPECS,SRPMS}
fi

# copy the rpm spec to the rpmbuild directory
cp ./microshift-app-images.spec "${RPMBUILD_DIR}"/SPECS/microshift-app-images.spec

# build the image with the specified spec
rpmbuild -bb --define "_topdir ${RPMBUILD_DIR}" $RPMBUILD_DIR/SPECS/microshift-app-images.spec