Pages

Thursday, June 22, 2017

Source-tree snapshots

While setting to build projects which source-codes are distributed as a source-code tarball it's convenient to prepare some grounds before simply extracting the tarball to a plain directory on a "general" file-system. That's because while fine-tuning or experimenting on building the project it's often necessary to keep track or manage the state of the source-code tree in order to not mess things up.

In Solaris, part of this ground preparation is particularly powerful and elegant by virtue of the ZFS, specially due to the possibility of easy snapshot, clone, hold / release, send / receive and rollback operations. To take advantage of all this power it's necessary to set up some delegated ZFS permissions to the base dataset where the source-code tree will reside. This dataset could be remote, shared via NFS or SMB, but more typically it is local to an individual developer system and usually located as a descendant to its home directory.

For instance:

# zfs create rpool/export/home/user1/software

# zfs allow !!$
zfs allow rpool/export/home/user1/software
---- Permissions on rpool/export/home/user1 ----
Local+Descendent permissions:
    user user1 create,mount,snapshot


By default, more recent versions of Solaris already delegates some access to the users' dataset but that's just a basic start, so better tune it as described on my delegated ZFS permissions post:
 
# zfs unallow \
      -ldu user1 "create,mount,snapshot" \
      rpool/export/home/user1

# zfs allow -s @generic ...
# zfs allow -s @descendant ...

# zfs allow -ldu user1 @generic !!$
# zfs allow -du user1 @descendant !!$

And then, by virtue of inheritance, you'll get:

# zfs allow rpool/export/home/user1/software
---- Permissions on rpool/export/home/user1 ----

Permission sets:
    @descendant
        clone,compression,destroy,promote,quota,
        readonly,rename,reservation,share,sharenfs 
    @generic
        create,diff,hold,mount,receive,release,
        rollback,send,snapshot,userprop

With the above ground setting, the developer (user1) now have much more latitude in managing the ZFS file-system datasets under his / her control, but not impairing the home / top-level directory (dataset) itself.

In addition to the ZFS support, there is the possibility, of course, to take advantage of a Revision Control System (RCS), specially Mercurial, which although not as fast as Git, is as powerful but with the advantage of being lightweight and requiring less back-end maintenance. This time, I won't bother with an RCS as ZFS will be more than enough for the particular tasks of simply building a whole source-code tree. An RCS would be interesting for a development scenario where more granular controle is required.

For instance:

$ DS=... # home / top-level directory (dataset)

$ zfs list -o name,refer -tall -r $DS |sed "s,$DS,...,"
NAME                           REFER
...                            4.18G
.../software                     32K
.../software/Qt                  32K
.../software/Qt/qt-4.8.7        384M
.../software/Qt/qt-4.8.7@base   384M



$ zfs get -r compressratio $DS |sed "s,$DS,...,"
NAME                           PROPERTY       VALUE  SOURCE
...                            compressratio  1.06x  -
.../software                   compressratio  1.78x  -
.../software/Qt                compressratio  1.78x  -
.../software/Qt/qt-4.8.7       compressratio  1.79x  -
.../software/Qt/qt-4.8.7@base  compressratio  1.79x  -


$ cd $DS/software/Qt/qt-4.8.7
$ ll -o
...
-rwxr-xr-x ... configure.exe
-rw-r--r-- ... LICENSE.LGPLv21
-rw-r--r-- ... INSTALL
drwxr-xr-x ... include
drwxr-xr-x ... demos
drwxr-xr-x ... translations
drwxr-xr-x ... templates
drwxr-xr-x ... examples
drwxr-xr-x ... config.profiles
drwxr-xr-x ... bin
drwxr-xr-x ... tools
drwxr-xr-x ... qmake
drwxr-xr-x ... mkspecs
-rw-r--r-- ... README
drwxr-xr-x ... src
drwxr-xr-x ... doc
drwxr-xr-x ... util
-rw-r--r-- ... projects.pro
-rw-r--r-- ... LICENSE.LGPLv3
drwxr-xr-x ... lib
drwxr-xr-x ... imports
-rw-r--r-- ... LICENSE.LGPL
-rw-r--r-- ... LICENSE.GPL3
-rw-r--r-- ... LICENSE.FDL
-rw-r--r-- ... LGPL_EXCEPTION.txt
-rwxr-xr-x ... configure
-rw-r--r-- ... changes-4.8.7
drwxr-xr-x ... config.tests


The above would be the result of a ZFS dataset hierarchy for the Qt version 4.8.7 source-code tree. The @base ZFS snapshot is created just after extracting the tarball. Although this version is marked as legacy, it may be an interesting starting point for exploring its build on Solaris 11.3 (GA / Release). Note, for example, that more recent releases of VirtualBox on Solaris already uses Qt version 5 series, thus a version series to try building as well.

In building the source-code tree it's not uncommon to have to tune or fix a few files throughout some build cycles until you get it as straight as possible. This burden is often due to Solaris being relegated as a legacy-OS, which is unfortunate and not true at all. Better if instead of "relegated" the word would be substituted for "selected" or so. Anyway, what is important is that between each cycle it may be better to have a thorough clean-up and the best and faster way of achieving this is not by some (g)make clean or some VCS rollback, but with a simple ZFS command:

$ zfs rollback $DS/software/Qt/qt-4.8.7@base

Or yet let's say you wanna keep one "imperfect" build for depuration purposes. Again, this can easily be achieved with fast and simple ZFS commands:

$ cd $DS/software/Qt
$ QT487=$DS/software/Qt/qt-4.8.7
$ zfs rename $QT487 $QT487-defect-1
$ zfs clone $QT487-defect-1@base $QT487
$ zfs promote $QT487