* [PATCH] Remove bashisms from spectrum shell scripts @ 2022-11-08 10:05 Henri Rosten 2022-11-08 13:27 ` Alyssa Ross 0 siblings, 1 reply; 5+ messages in thread From: Henri Rosten @ 2022-11-08 10:05 UTC (permalink / raw) To: devel; +Cc: Henri Rosten This commit removes bashisms from spectrum shell scripts. This change is needed to be able to use the scripts from POSIX-compliant shells which are not bash compatible - such as dash. Signed-off-by: Henri Rosten <henri.rosten@unikie.com> --- Documentation/scripts/undocumented-uuids.sh | 6 +++++- scripts/format-uuid.sh | 15 ++++++++++++++- scripts/make-gpt.sh | 7 +++---- 3 files changed, 22 insertions(+), 6 deletions(-) diff --git a/Documentation/scripts/undocumented-uuids.sh b/Documentation/scripts/undocumented-uuids.sh index 34c2d22..0434407 100755 --- a/Documentation/scripts/undocumented-uuids.sh +++ b/Documentation/scripts/undocumented-uuids.sh @@ -1,5 +1,6 @@ #!/bin/sh -eu # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ cd "$(dirname "$0")/../.." @@ -7,8 +8,11 @@ cd "$(dirname "$0")/../.." PATTERN='\b[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}\b' UUID_REFERENCE_PATH=Documentation/uuid-reference.adoc +tmp=$(mktemp) +grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u >$tmp git ls-files -coz --exclude-standard | grep -Fxvz "$UUID_REFERENCE_PATH" | xargs -0 git grep -Ehio --no-index --no-line-number "$PATTERN" -- | sort -u | - comm -23 - <(grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u) + comm -23 - $tmp +rm -f $tmp diff --git a/scripts/format-uuid.sh b/scripts/format-uuid.sh index fa07eb9..c7b94da 100755 --- a/scripts/format-uuid.sh +++ b/scripts/format-uuid.sh @@ -1,6 +1,19 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ -printf "%s\n" "${1:0:8}-${1:8:4}-${1:12:4}-${1:16:4}-${1:20}" +substr () { + str=$1 + beg=$2 + end=$3 + echo $(echo $str | cut -c $beg-$end) +} + +u1=$(substr $1 1 8) +u2=$(substr $1 9 12) +u3=$(substr $1 13 16) +u4=$(substr $1 17 20) +u5=$(substr $1 21 32) +printf "%s\n" "$u1-$u2-$u3-$u4-$u5" diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh index 56076d3..351aa69 100755 --- a/scripts/make-gpt.sh +++ b/scripts/make-gpt.sh @@ -1,6 +1,7 @@ #!/bin/sh -eu # # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is> +# SPDX-FileCopyrightText: 2022 Unikie # SPDX-License-Identifier: EUPL-1.2+ # # usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID]... @@ -38,7 +39,7 @@ scriptsDir="$(dirname "$0")" out="$1" shift -nl=$'\n' +nl='\n' table="label: gpt" # Keep 1MiB free at the start, and 1MiB free at the end. @@ -51,9 +52,7 @@ done rm -f "$out" truncate -s "$gptBytes" "$out" -sfdisk "$out" <<EOF -$table -EOF +printf "$table" | sfdisk "$out" n=0 for partition; do -- 2.25.1 ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Remove bashisms from spectrum shell scripts 2022-11-08 10:05 [PATCH] Remove bashisms from spectrum shell scripts Henri Rosten @ 2022-11-08 13:27 ` Alyssa Ross 2022-11-08 15:05 ` Henri Rosten 0 siblings, 1 reply; 5+ messages in thread From: Alyssa Ross @ 2022-11-08 13:27 UTC (permalink / raw) To: Henri Rosten; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 3743 bytes --] On Tue, Nov 08, 2022 at 12:05:44PM +0200, Henri Rosten wrote: > This commit removes bashisms from spectrum shell scripts. This change is > needed to be able to use the scripts from POSIX-compliant shells which > are not bash compatible - such as dash. > > Signed-off-by: Henri Rosten <henri.rosten@unikie.com> > --- > Documentation/scripts/undocumented-uuids.sh | 6 +++++- > scripts/format-uuid.sh | 15 ++++++++++++++- > scripts/make-gpt.sh | 7 +++---- > 3 files changed, 22 insertions(+), 6 deletions(-) > Thanks! I definitely prefer to stick to portable shell, awk, make etc. functionality, so I'm happy to receive this patch. I'll look into integrating shellcheck into the build to ensure the bashisms don't creep back in. I've left some comments below. > diff --git a/Documentation/scripts/undocumented-uuids.sh b/Documentation/scripts/undocumented-uuids.sh > index 34c2d22..0434407 100755 > --- a/Documentation/scripts/undocumented-uuids.sh > +++ b/Documentation/scripts/undocumented-uuids.sh > @@ -1,5 +1,6 @@ > #!/bin/sh -eu > # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> > +# SPDX-FileCopyrightText: 2022 Unikie > # SPDX-License-Identifier: EUPL-1.2+ > > cd "$(dirname "$0")/../.." > @@ -7,8 +8,11 @@ cd "$(dirname "$0")/../.." > PATTERN='\b[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}\b' > UUID_REFERENCE_PATH=Documentation/uuid-reference.adoc > > +tmp=$(mktemp) Let's clean this up with a trap, so that it gets cleaned up even if something fails: trap "rm -f $tmp" EXIT And since we have to give it a name, maybe we could use the opportunity to make it descriptive? "documented_uuids", maybe? > +grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u >$tmp > git ls-files -coz --exclude-standard | > grep -Fxvz "$UUID_REFERENCE_PATH" | > xargs -0 git grep -Ehio --no-index --no-line-number "$PATTERN" -- | > sort -u | > - comm -23 - <(grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u) > + comm -23 - $tmp > +rm -f $tmp > diff --git a/scripts/format-uuid.sh b/scripts/format-uuid.sh > index fa07eb9..c7b94da 100755 > --- a/scripts/format-uuid.sh > +++ b/scripts/format-uuid.sh > @@ -1,6 +1,19 @@ > #!/bin/sh -eu > # > # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is> > +# SPDX-FileCopyrightText: 2022 Unikie > # SPDX-License-Identifier: EUPL-1.2+ > > -printf "%s\n" "${1:0:8}-${1:8:4}-${1:12:4}-${1:16:4}-${1:20}" > +substr () { > + str=$1 > + beg=$2 > + end=$3 > + echo $(echo $str | cut -c $beg-$end) This should work fine if we drop the echo $(...), right? > +} > + > +u1=$(substr $1 1 8) > +u2=$(substr $1 9 12) > +u3=$(substr $1 13 16) > +u4=$(substr $1 17 20) > +u5=$(substr $1 21 32) > +printf "%s\n" "$u1-$u2-$u3-$u4-$u5" > diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh > index 56076d3..351aa69 100755 > --- a/scripts/make-gpt.sh > +++ b/scripts/make-gpt.sh > @@ -1,6 +1,7 @@ > #!/bin/sh -eu > # > # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is> > +# SPDX-FileCopyrightText: 2022 Unikie > # SPDX-License-Identifier: EUPL-1.2+ > # > # usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID]... > @@ -38,7 +39,7 @@ scriptsDir="$(dirname "$0")" > out="$1" > shift > > -nl=$'\n' > +nl='\n' > table="label: gpt" > > # Keep 1MiB free at the start, and 1MiB free at the end. > @@ -51,9 +52,7 @@ done > > rm -f "$out" > truncate -s "$gptBytes" "$out" > -sfdisk "$out" <<EOF > -$table > -EOF > +printf "$table" | sfdisk "$out" The heredoc previously used here should be POSIX: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04 Does it not work in dash? > > n=0 > for partition; do > -- > 2.25.1 > > [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Remove bashisms from spectrum shell scripts 2022-11-08 13:27 ` Alyssa Ross @ 2022-11-08 15:05 ` Henri Rosten 2022-11-09 12:32 ` Alyssa Ross 0 siblings, 1 reply; 5+ messages in thread From: Henri Rosten @ 2022-11-08 15:05 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel On Tue, Nov 08, 2022 at 01:27:27PM +0000, Alyssa Ross wrote: > On Tue, Nov 08, 2022 at 12:05:44PM +0200, Henri Rosten wrote: > > This commit removes bashisms from spectrum shell scripts. This change is > > needed to be able to use the scripts from POSIX-compliant shells which > > are not bash compatible - such as dash. > > > > Signed-off-by: Henri Rosten <henri.rosten@unikie.com> > > --- > > Documentation/scripts/undocumented-uuids.sh | 6 +++++- > > scripts/format-uuid.sh | 15 ++++++++++++++- > > scripts/make-gpt.sh | 7 +++---- > > 3 files changed, 22 insertions(+), 6 deletions(-) > > > > Thanks! I definitely prefer to stick to portable shell, awk, make etc. > functionality, so I'm happy to receive this patch. I'll look into > integrating shellcheck into the build to ensure the bashisms don't creep > back in. > > I've left some comments below. > > > diff --git a/Documentation/scripts/undocumented-uuids.sh b/Documentation/scripts/undocumented-uuids.sh > > index 34c2d22..0434407 100755 > > --- a/Documentation/scripts/undocumented-uuids.sh > > +++ b/Documentation/scripts/undocumented-uuids.sh > > @@ -1,5 +1,6 @@ > > #!/bin/sh -eu > > # SPDX-FileCopyrightText: 2022 Alyssa Ross <hi@alyssa.is> > > +# SPDX-FileCopyrightText: 2022 Unikie > > # SPDX-License-Identifier: EUPL-1.2+ > > > > cd "$(dirname "$0")/../.." > > @@ -7,8 +8,11 @@ cd "$(dirname "$0")/../.." > > PATTERN='\b[A-F0-9]{8}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{4}-[A-F0-9]{12}\b' > > UUID_REFERENCE_PATH=Documentation/uuid-reference.adoc > > > > +tmp=$(mktemp) > > Let's clean this up with a trap, so that it gets cleaned up even if > something fails: > > trap "rm -f $tmp" EXIT > > And since we have to give it a name, maybe we could use the opportunity > to make it descriptive? "documented_uuids", maybe? > I'll fix this in a follow-up version. > > +grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u >$tmp > > git ls-files -coz --exclude-standard | > > grep -Fxvz "$UUID_REFERENCE_PATH" | > > xargs -0 git grep -Ehio --no-index --no-line-number "$PATTERN" -- | > > sort -u | > > - comm -23 - <(grep -Eio "$PATTERN" "$UUID_REFERENCE_PATH" | sort -u) > > + comm -23 - $tmp > > +rm -f $tmp > > diff --git a/scripts/format-uuid.sh b/scripts/format-uuid.sh > > index fa07eb9..c7b94da 100755 > > --- a/scripts/format-uuid.sh > > +++ b/scripts/format-uuid.sh > > @@ -1,6 +1,19 @@ > > #!/bin/sh -eu > > # > > # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is> > > +# SPDX-FileCopyrightText: 2022 Unikie > > # SPDX-License-Identifier: EUPL-1.2+ > > > > -printf "%s\n" "${1:0:8}-${1:8:4}-${1:12:4}-${1:16:4}-${1:20}" > > +substr () { > > + str=$1 > > + beg=$2 > > + end=$3 > > + echo $(echo $str | cut -c $beg-$end) > > This should work fine if we drop the echo $(...), right? > True, the outer echo is not needed. I'll fix this too in a follow-up version. > > +} > > + > > +u1=$(substr $1 1 8) > > +u2=$(substr $1 9 12) > > +u3=$(substr $1 13 16) > > +u4=$(substr $1 17 20) > > +u5=$(substr $1 21 32) > > +printf "%s\n" "$u1-$u2-$u3-$u4-$u5" > > diff --git a/scripts/make-gpt.sh b/scripts/make-gpt.sh > > index 56076d3..351aa69 100755 > > --- a/scripts/make-gpt.sh > > +++ b/scripts/make-gpt.sh > > @@ -1,6 +1,7 @@ > > #!/bin/sh -eu > > # > > # SPDX-FileCopyrightText: 2021-2022 Alyssa Ross <hi@alyssa.is> > > +# SPDX-FileCopyrightText: 2022 Unikie > > # SPDX-License-Identifier: EUPL-1.2+ > > # > > # usage: make-gpt.sh GPT_PATH PATH:PARTTYPE[:PARTUUID]... > > @@ -38,7 +39,7 @@ scriptsDir="$(dirname "$0")" > > out="$1" > > shift > > > > -nl=$'\n' > > +nl='\n' > > table="label: gpt" > > > > # Keep 1MiB free at the start, and 1MiB free at the end. > > @@ -51,9 +52,7 @@ done > > > > rm -f "$out" > > truncate -s "$gptBytes" "$out" > > -sfdisk "$out" <<EOF > > -$table > > -EOF > > +printf "$table" | sfdisk "$out" > > The heredoc previously used here should be POSIX: > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04 > > Does it not work in dash? > Heredoc works in dash too, but I couldn't find out how to expand the newlines in variable inside heredoc so that it would work in dash. Therefore I simply removed the heredoc and used printf instead. > > > > n=0 > > for partition; do > > -- > > 2.25.1 > > > > ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Remove bashisms from spectrum shell scripts 2022-11-08 15:05 ` Henri Rosten @ 2022-11-09 12:32 ` Alyssa Ross 2022-11-09 12:59 ` Henri Rosten 0 siblings, 1 reply; 5+ messages in thread From: Alyssa Ross @ 2022-11-09 12:32 UTC (permalink / raw) To: Henri Rosten; +Cc: devel [-- Attachment #1: Type: text/plain, Size: 1132 bytes --] > > > @@ -38,7 +39,7 @@ scriptsDir="$(dirname "$0")" > > > out="$1" > > > shift > > > > > > -nl=$'\n' > > > +nl='\n' > > > table="label: gpt" > > > > > > # Keep 1MiB free at the start, and 1MiB free at the end. > > > @@ -51,9 +52,7 @@ done > > > > > > rm -f "$out" > > > truncate -s "$gptBytes" "$out" > > > -sfdisk "$out" <<EOF > > > -$table > > > -EOF > > > +printf "$table" | sfdisk "$out" > > > > The heredoc previously used here should be POSIX: > > > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04 > > > > Does it not work in dash? > > > > Heredoc works in dash too, but I couldn't find out how to expand the > newlines in variable inside heredoc so that it would work in dash. > Therefore I simply removed the heredoc and used printf instead. I think the problem there is this one: https://www.shellcheck.net/wiki/SC3003 -- In POSIX sh, $'..' is undefined. So I think we could fix it without introducing an extra printf process, by changing how we define `nl` to use a literal newline. The following test script worked for me in dash: nl=' ' cat <<EOF foo${nl}bar EOF [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Remove bashisms from spectrum shell scripts 2022-11-09 12:32 ` Alyssa Ross @ 2022-11-09 12:59 ` Henri Rosten 0 siblings, 0 replies; 5+ messages in thread From: Henri Rosten @ 2022-11-09 12:59 UTC (permalink / raw) To: Alyssa Ross; +Cc: devel On Wed, Nov 09, 2022 at 12:32:10PM +0000, Alyssa Ross wrote: > > > > @@ -38,7 +39,7 @@ scriptsDir="$(dirname "$0")" > > > > out="$1" > > > > shift > > > > > > > > -nl=$'\n' > > > > +nl='\n' > > > > table="label: gpt" > > > > > > > > # Keep 1MiB free at the start, and 1MiB free at the end. > > > > @@ -51,9 +52,7 @@ done > > > > > > > > rm -f "$out" > > > > truncate -s "$gptBytes" "$out" > > > > -sfdisk "$out" <<EOF > > > > -$table > > > > -EOF > > > > +printf "$table" | sfdisk "$out" > > > > > > The heredoc previously used here should be POSIX: > > > > > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_07_04 > > > > > > Does it not work in dash? > > > > > > > Heredoc works in dash too, but I couldn't find out how to expand the > > newlines in variable inside heredoc so that it would work in dash. > > Therefore I simply removed the heredoc and used printf instead. > > I think the problem there is this one: > > https://www.shellcheck.net/wiki/SC3003 -- In POSIX sh, $'..' is undefined. > > So I think we could fix it without introducing an extra printf process, > by changing how we define `nl` to use a literal newline. > > The following test script worked for me in dash: > > nl=' > ' > cat <<EOF > foo${nl}bar > EOF True, that works fine. Let me fix this with a new version in a moment. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-11-09 12:59 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-11-08 10:05 [PATCH] Remove bashisms from spectrum shell scripts Henri Rosten 2022-11-08 13:27 ` Alyssa Ross 2022-11-08 15:05 ` Henri Rosten 2022-11-09 12:32 ` Alyssa Ross 2022-11-09 12:59 ` Henri Rosten
Code repositories for project(s) associated with this public inbox https://spectrum-os.org/git/crosvm https://spectrum-os.org/git/doc https://spectrum-os.org/git/mktuntap https://spectrum-os.org/git/nixpkgs https://spectrum-os.org/git/spectrum https://spectrum-os.org/git/ucspi-vsock https://spectrum-os.org/git/www This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).