 |
| OUT OF DATE |
The well-known essentials of GNU build automation end-user operation work-flow was the subject of another post, in which I hopefully confirmed to work reasonably with a few adjustments under Solaris 11.3 and in which I successfully integrated some well-known ZFS essential advantages, such as source-tree snapshots.
In this post I hope to somewhat complement what's been discussed by presenting a suggestion of Shell script to hopefully simplify and avoid some common mistakes in the preparation of the GNU build automation work-flow environment.
For instance, assume that an application source code (tarball app-1.0.0.tar.gz) is to be built for both 32-bit and 64-bit targets using GNU tools and that the build operation is to take place at a local staff /software/build with the results installed locally under /software/prototype.
NOTE
I assume that the source code tarball is usually named under the following convention: name-version.tar.compression.
But there are cases this convention varies, for instance. node.js prepends a v before version, which slight variation, in this particular case, is a minor annoyance and fortunately won't break the following script. After the script is run, if desired, one can rename the datasets accordingly in order to normalize things, by removing the v before version number.
But there are other cases such as maven (apache-maven-3.5.3-src.tar.gz) where the tarball name would break the script. I could attempt fixing it, but as there are so many unpredictable possibilities this would certainly become a unmanageable.
The general solution is to conform the tarballs' names to my original assumption, usually with a simple manual procedure (as root) of extracting the archive and then storing everything back under a conformant archive name. For instance:
# cd /tmp
# gtar xf .../maven/3.5.3/apache-maven-3.5.3-src.tar.gz
# gtar czf maven-3.5.3.tar.gz maven-3.5.3
# cp maven-3.5.3.tar.gz .../maven/3.5.3/
As an alternative while recreating the tarball, one can attempt higher compression rates with j (for a .bz2) or J (for a .xz) instead of just z (for a more common .gz compressed file). On this particular example the sizes become: 2.6M (.gz), 2.2M (.bz2) and 1.8M (.xz).
A possible script (gnu-build-preparation) could be:
#!/bin/bash -
################################################################
if [[ -z "$1" ]];
then
echo
echo "Usage: $0 <tarball>"
echo
exit 1
else
if ! [[ -f "$1" ]];
then
echo
echo "$1 not found."
echo
exit 1
fi
fi
################################################################
DS_BASE=rpool
DS_SOFTWARE=${DS_BASE:+$DS_BASE/}software
DS_BUILD=build
DS_PROTOTYPE=prototype
################################################################
TAR=$(realpath "$1")
A=$(basename $1 |sed 's/\(.*\)-.*/\1/')
V=$(basename $1 |sed 's/.*-\(.*\)\.tar.*/\1/')
echo
echo Processing $TAR
echo
echo ------------------------------------
echo "App: $A"
echo "Ver: $V"
echo ------------------------------------
echo
echo In the process, the following ZFS datasets will be created:
echo
echo " $DS_SOFTWARE/$DS_BUILD/$A"
echo " $DS_SOFTWARE/$DS_BUILD/$A/$A-$V"
echo " $DS_SOFTWARE/$DS_BUILD/$A/$A-$V-gnu32"
echo " $DS_SOFTWARE/$DS_BUILD/$A/$A-$V-gnu64"
echo
echo " $DS_SOFTWARE/$DS_PROTOTYPE/$A"
echo " $DS_SOFTWARE/$DS_PROTOTYPE/$A/$A-$V"
echo " $DS_SOFTWARE/$DS_PROTOTYPE/$A/$A-$V/gnu32"
echo " $DS_SOFTWARE/$DS_PROTOTYPE/$A/$A-$V/gnu64"
echo
read -p "Enter \"y\" to proceed: " -r
if ! [[ "$REPLY" = "y" ]] && ! [[ "$REPLY" = "Y" ]];
then
echo "Exiting..."
exit 1
fi
################################################################
function dataset-exists()
{
zfs list -H "$1" 2>/dev/null
local rv=$?
echo $rv
}
################################################################
function dataset-get-mountpoint()
{
echo $(zfs get -H -o value mountpoint "$1" 2>/dev/null)
}
################################################################
function dataset-assert-mounted()
{
if [[ -z $(dataset-get-mountpoint "$1") ]]; then
echo "Unexpected: \"$1\" doesn't seem to be mounted."
exit 1
fi
}
dataset-assert-mounted "$DS_SOFTWARE"
dataset-assert-mounted "$DS_SOFTWARE/$DS_BUILD"
dataset-assert-mounted "$DS_SOFTWARE/$DS_PROTOTYPE"
################################################################
function dataset-app-version()
{
if [[ $(dataset-exists "$1") -eq 1 ]]; then
echo "Creating $1..."
zfs create "$1"
else
local MP=$(dataset-get-mountpoint "$1")
if [[ $(basename "$MP") == "$A" ]]; then
echo "No need to create $1."
else
echo
echo "$1 mounted as $MP."
echo "No further action can be safely taken."
echo "Exiting..."
exit 1
fi
fi
}
echo
dataset-app-version "$DS_SOFTWARE/$DS_BUILD/$A"
dataset-app-version "$DS_SOFTWARE/$DS_BUILD/$A/$A-$V"
################################################################
echo
DS_APPLICATION="$DS_SOFTWARE/$DS_BUILD/$A"
DS_APPLICATION_VERSION="$DS_APPLICATION/$A-$V"
MP=$(dataset-get-mountpoint "$DS_APPLICATION_VERSION")
if [[ $(ls -A "$MP" 2>/dev/null) ]]; then
echo
echo "Directory $MP not empty!"
echo "No further action can be safely taken."
echo "Exiting..."
exit 1
fi
(cd "$MP/.."; gtar xf "$TAR")
if ! [[ $(ls -A "$MP" 2>/dev/null) ]]; then
echo
echo "Directory $MP shouldn't be empty!"
echo "No further action can be safely taken."
echo "Exiting..."
exit 1
fi
zfs snapshot "$DS_APPLICATION_VERSION@source"
zfs set readonly=on "$DS_APPLICATION_VERSION"
################################################################
function set-target()
{
local T=$1
# This may be inappropriate in some cases.
# Some "packages" require/prefer just an empty dataset.
zfs clone "$DS_APPLICATION_VERSION"@source \
"$DS_APPLICATION_VERSION-$T"
# This may be too soon.
# The "package" may need some manual fix before @start
zfs snapshot "$DS_APPLICATION_VERSION-$T"@start
}
set-target gnu32
set-target gnu64
################################################################
zfs list -H -r -t all -o name "$DS_APPLICATION"
################################################################
#
# Create the final resting place of the results.
# Use an independent ZFS pool/dataset, not rpool/VARSHARE.
# At end consider snapshoting, just in case.
# Could be a staging area for packaging.
#
DS_PROTOTYPE="$DS_SOFTWARE/$DS_PROTOTYPE/$A"
DS_PROTOTYPE_VERSION="$DS_PROTOTYPE/$A-$V"
echo
echo "Creating "$DS_PROTOTYPE" subtree."
echo
function create-staging-area()
{
if [[ $(dataset-exists "$1") -eq 1 ]]; then
zfs create -p "$1"
fi
}
create-staging-area "$DS_PROTOTYPE_VERSION"
create-staging-area "$DS_PROTOTYPE_VERSION/gnu32"
create-staging-area "$DS_PROTOTYPE_VERSION/gnu64"
zfs list -H -r -t all -o name "$DS_PROTOTYPE"
################################################################
echo
echo "Creating pre-configuration script."
cat > "$MP/../setenv" <<EOF
#
# HOSTTYPE=$HOSTTYPE
# OSTYPE=$OSTYPE
# MACHTYPE=$MACHTYPE
#
# Up to Solaris 11.3 the default is 32-bits.
# MACHTYPE is typically i386-pc-solaris2.11
#
# Since Solaris 11.4 Beta the default is 64-bits.
# MACHTYPE is x86_64-pc-solaris2.11
#
# It's advisable not to override the above env vars
# buy you experiment with the --build or --target options
# to the standard GNU's configuration script: configure.
# (they are important to properly organize the built artifacts)
#
# Bottom line it seems that the -m option to the flags
# CFLAGS, CXXFLAGS and LDFLAGS always prevail after all.
# (in terms of the actual bitness of the built artifacts)
#
if [[ -z "\$1" ]];
then
echo "Usage: source setenv <bit-target>"
return 1
else
if [[ "\$1" -ne 32 ]] && [[ "\$1" -ne 64 ]];
then
echo "Valid bit-targets: 32 or 64."
return 1
fi
fi
# Despite setting bitness as shown next
# DON'T override HOSTTYPE and MACHTYPE
# Just remind the proper --build option
BITS=\$1
echo
case $BITS in
32)
echo --prefix=/software/prototype/.../gnu32
echo --build=i386-pc-solaris2.11
;;
64)
echo --prefix=/software/prototype/.../gnu64
echo --build=x86_64-pc-solaris2.11
;;
esac
function add-path()
{
local COMPONENT="\$1"
if [[ -d "\$COMPONENT" ]] ;
then
! [[ "\$PATH" =~ "\$COMPONENT" ]] && \
export PATH="\$COMPONENT:\$PATH"
fi
}
function add-pkgconfig-path()
{
local COMPONENT="\$1"
local TAIL="\${PKG_CONFIG_PATH:+:\$PKG_CONFIG_PATH}"
if [[ -d "\$COMPONENT" ]] ;
then
! [[ "\$TAIL" =~ "\$COMPONENT" ]] && \
export PKG_CONFIG_PATH="\$COMPONENT\$TAIL"
fi
}
function extend-env()
{
local BASE="\$1/gnu\$BITS"
add-path "\$BASE/bin"
add-pkgconfig-path "\$BASE/lib/pkgconfig"
}
add-path /usr/gnu/bin
#
# Other PATH and PKG_CONFIG_PATH settings.
# Put entries in reverse order of dependency.
# (following my PATH building suggestion)
# The samples below are pre-Solaris 11.4 Beta.
#
# ...
# extend-env /opt/tcl-8.5.19
# extend-env /opt/automake-1.15
# extend-env /opt/autoconf-2.69
# extend-env /opt/m4-1.4.18
# extend-env /opt/libtool-2.4.6
echo
# In general, not a good idea; evaluate.
# export CONFIG_SHELL=/bin/bash
echo CONFIG_SHELL=\$CONFIG_SHELL
echo
#
# The following compilers and linker overrides
# may be troublesome, specially the -m and -std options
# as it may be very source dependent; evaluate.
# The bare -std minimums should be gnu89 and gnu+03;
# since Solaris 11.4 Beta gnu11 and gnu++11 seem Ok.
#
FLAGS="-m\$BITS -march=core2"
export CC=/usr/bin/gcc
export CFLAGS="\$FLAGS -std=gnu89"
echo CC=\$CC CFLAGS=\$CFLAGS
echo
export CXX=/usr/bin/g++
export CXXFLAGS="\$FLAGS -std=gnu++03"
echo CXX=\$CXX CXXFLAGS=\$CXXFLAGS
echo
export LD=/usr/bin/ld
export LDFLAGS="\$FLAGS"
echo LD=\$LD LDFLAGS=\$LDFLAGS
echo
echo PATH=\$PATH
echo
echo PKG_CONFIG_PATH=\$PKG_CONFIG_PATH
unset CF
EOF
cat > "$MP/../unsetenv" <<EOF
function cleanup-env()
{
unset CONFIG_SHELL
unset CC
unset CFLAGS
unset CXX
unset CXXFLAGS
unset LD
unset LDFLAGS
unset PKG_CONFIG_PATH
export PATH=/usr/bin:/usr/sbin
unset -f add-path
unset -f add-pkgconfig-path
unset -f extend-env
unset -f cleanup-env
}
cleanup-env
EOF
touch "$MP/../NOTES"
For instance, assuming that a non-priviledged user has been delegated appropriate ZFS permissions on rpool/software/build and rpool/software/prototype in addition to the ownership and appropriate mode for the respective mountpoints, the output of the above script could be as follows:
$ pwd
/software/build
$./gnu-build-preparation ../source/.../ruby-2.5.0.tar.gz
Processing /software/source/.../ruby-2.5.0.tar.gz
------------------------------------
App: ruby
Ver: 2.5.0
------------------------------------
In the process, the following ZFS datasets will be created:
rpool/software/build/ruby
rpool/software/build/ruby/ruby-2.5.0
rpool/software/build/ruby/ruby-2.5.0-gnu32
rpool/software/build/ruby/ruby-2.5.0-gnu64
rpool/software/prototype/ruby
rpool/software/prototype/ruby/ruby-2.5.0
rpool/software/prototype/ruby/ruby-2.5.0/gnu32
rpool/software/prototype/ruby/ruby-2.5.0/gnu64
Enter "y" to proceed: y
Creating rpool/software/build/ruby...
Creating rpool/software/build/ruby/ruby-2.5.0...
rpool/software/build/ruby
rpool/software/build/ruby/ruby-2.5.0
rpool/software/build/ruby/ruby-2.5.0@source
rpool/software/build/ruby/ruby-2.5.0-gnu32
rpool/software/build/ruby/ruby-2.5.0-gnu32@start
rpool/software/build/ruby/ruby-2.5.0-gnu64
rpool/software/build/ruby/ruby-2.5.0-gnu64@start
Creating rpool/software/prototype/ruby subtree.
rpool/software/prototype/ruby
rpool/software/prototype/ruby/ruby-2.5.0
rpool/software/prototype/ruby/ruby-2.5.0/gnu32
rpool/software/prototype/ruby/ruby-2.5.0/gnu64
Creating pre-configuration script.
