C++ Locales under GCC

The C++ locales is a cool feature for software internationalization and everybody certainly knows about or have heard of it. But when trying to use it on anything but a Windows or Linux system one may find out a crippled implementation. This is the case when GCC is ported to a different platform such as Solaris.

At first I thought it could be a simple matter of getting a more updated version of GCC. Then I've tried to document the potential solution on a previous post. But while digging into the matter I've sadly realized it's a more complicated issue.

As a matter of fact the issue lies in GNU libstdc++ which unfortunately does not provide a complete implementation of Locales. Its implementation is target just for systems already possessing a working implementation of the GNU C Library (glibc) which offers a key function usually found at newlocale.c source module that exposes the "special property" of being thread safe" at the C++ level". Solaris, as a standard UNIX system, uses libc which is thread safe just at the C level. Hence, GNU works around the issue by providing what is called a "general stub" only supporting "C"  (= "POSIX") locale and otherwise throwing an exception. This has been made clear as per the following archived message: generic locale.

It seems that the main issue is an underlying libc limitation of not providing necessary API points for building up a multithreaded implementation of libstdc++ on top of it. As a consequence one entirely lacks a key C++ feature because of an unresolved multithreading issue. At this point it turns out easier to understand why there's a GNU C Library on the first place and why GNU libstdc++ relies on it instead of a particular system's libc.

But wondering about this multithreading issue between the C and C++ libraries when dealing with other locale dependent calls, one could realized that in most cases all that's necessary is to simply set the locale once on start-up code or while entering the main thread. From that point on, as long as a program remains pure C++ at the locale-dependent high-level I/O calls all should work pretty well.
 
I mean that in general a program isn't expected to be switching locales anymore after having properly intialized itself. I know I could be missing many reasons why the contrary should hold, but it seems they all would narrow down to some processing that should be carried out multiple times (in parallel) for different locales, such as processing some sort of localized I/O to several branch offices under a single process.

Therefore due to strictness, an all or nothing situation, one ends up with virtually nothing at all under Solaris and other less popular platforms for which nobody else carried about on providing a slightly more decent implementation of the C++ standard :-(
 

Shared Libraries - Solaris 11.3

Although I've already posted about shared library naming and versioning conventions and have already given a first example I consider interesting, there's, of course, much more about this subject.

It seems that fortunately Solaris 11.3 Information Library on its section on Developing Applications For Use With Solaris 11 at the chapter on Solaris 11.3 Linkers and Libraries Guide provides a lot of knowledgeable information on the topics. From what I've seen, a must read for expanding the vision on yet more powerful options that may lead someone to the next level towards the elite.

Shared Libraries - Example #1

Let's try a first example of a fictitious dynamic application (one that links to dynamic objects ‒ a shared library in this case). The key aspects I wish to address on this post are how to build these artifacts under Solaris 11.3 using GCC 4.8.2 and properly adhering to the versioning naming convention. I would like to emphasize that in the name of straightforward software engineering there will be no excessively elaborated interfaces strategies, for instance by using crazy things such as GUIDs (or so) or any other related / similar interface infrastructure (if you like GUIDs you may perhaps have some fun playing with a GUID generator). My strategy is pure C++ resulting in a much more powerful, efficient and higher-quality artifact than some sort of software crazy and buggy Frankenstein monster such as COM and related. I'll be using C++ namespaces and a main C++ class on each library module.

The software engineering implementation/deployment depicted on this example consists (for the sake of simplicity) on a single (do-nothing) shared library and a (stub) dynamic application linking to it just for testing and demonstration. I'll build 2 major versions of the library (libgreeting.so.1 and libgreeting.so.2) the first of them with some different minor versions and releases (for demonstration purposes). For each major version of the library I'll show a backward compatibility tehcnique for avoiding or minimizing the rebuild of the application (myapp-1). Accompanying each new major version of the library there will be (for the sake of simplicity) a new / updated header-file which as I said will provide for backward compatibility with previous major version(s). The header-files shouldn't be modified within minor versions and releases as this would be a clear deviation of whole strategy. For demonstration purposes, the implementation of each interface will consist on multiple source-code files. Besides the interface mechanism the library will be able to "report" its hard-coded versioning information (as a kind of reflection) which will always correspond to the name of the shared library version final binary object, although (again for the sake of simplicity) there will be no provisions to enforce this correspondence.

I can only hope that the software engineering aspects I'll be exploring side-by-side do not hinder the main goals of this post about building and dealing with shared libraries!

Shared Libraries - naming & versioning

The convention of UNIX shared libraries names may be somewhat puzzling because it appears different systems may not strictly adhere to the convention. What makes most sense appears to be the convention followed on Linux and (apparently) not quite strictly followed on Solaris. On this post I'll try to make a better sense of this in order to discover the right (or at least best) way of doing things in Solaris.

GCC under Solaris 11.4 Beta (refresh)

About 2 months ago I've posted about the availability of GCC 5.5.0 under Solaris 11.4 Beta. Today I've just got to know through my Linked-in connections that everything has been refreshed and that now GCC 7.3 is available. I'm still not believing but at some point I'm going to check it out!

The RLE configuration

It's been a while since I last dropped a line into this blog. That's not because I've neglected it but because there's been so much to learn in C++ and a wealth of other technologies and metodologies that it gets hard to find time to write all of my findings. But anyway here I am and let's go again.

GCC under Solaris 11.4 Beta

It's been quite a while since the last publicly available version of GCC under Solaris.
Particularly under Solaris 11.3 I've described how to install it long ago.
With the release of Solaris 11.4 Beta I decided to quickly review it.

# pkg info -r gcc
          Name: developer/gcc
       Summary: GCC
      Category: Dev.../C (org.opensolaris.category.2008) Dev.../C++
                (org.opensolaris.category.2008) Dev.../Fortran
                (org.opensolaris.category.2008) De.../GNU
                (org.opensolaris.category.2008) Dev.../Objective C
                (org.opensolaris.category.2008)
         State: Not installed
     Publisher: solaris
       Version: 5.5.0
        Branch: 11.4.0.0.0.12.0
Packaging Date: Wed Jan 03 02:32:06 2018
          Size: 2.52 kB
          FMRI: pkg://.../developer/gcc@5.5.0-11.4.0.0.0.12.0:...
   Project URL: http://gcc.gnu.org/
    Source URL: http://ftp.gnu.org/.../gcc-5.5.0.tar.gz

GNU - Build preparation

OUT OF DATE

It's no doubt very convenient and trustworthy to install new software through IPS or even the legacy SVr4 packages. But when those options are not available fortunately there's still a chance for the GNU build automation, which although not as convenient can also perform a good job.

The well-known essentials of GNU build automation end-user operation work-flow was the subject of another post, in which I hopefully confirmed to work reasonably with a few adjustments under Solaris 11.3 and in which I successfully integrated some well-known ZFS essential advantages, such as source-tree snapshots.

In this post I hope to somewhat complement what's been discussed by presenting a suggestion of Shell script to hopefully simplify and avoid some common mistakes in the preparation of the GNU build automation work-flow environment.

For instance, assume that an application source code (tarball app-1.0.0.tar.gz) is to be built for both 32-bit and 64-bit targets using GNU tools and that the build operation is to take place at a local staff /software/build with the results installed locally under /software/prototype.

NOTE

I assume that the source code tarball is usually named under the following convention: name-version.tar.compression.

But there are cases this convention varies, for instance. node.js prepends a v before version, which slight variation, in this particular case, is a minor annoyance and fortunately won't break the following script. After the script is run, if desired, one can rename the datasets accordingly in order to normalize things, by removing the v before version number.

But there are other cases such as maven (apache-maven-3.5.3-src.tar.gz) where the tarball name would break the script. I could attempt fixing it, but as there are so many unpredictable possibilities this would certainly become a unmanageable.
 
The general solution is to conform the tarballs' names to my original assumption, usually with a simple manual procedure (as root) of extracting the archive and then storing everything back under a conformant archive name. For instance:
 
# cd /tmp
# gtar xf .../maven/3.5.3/apache-maven-3.5.3-src.tar.gz
# gtar czf maven-3.5.3.tar.gz maven-3.5.3
# cp maven-3.5.3.tar.gz .../maven/3.5.3/
 
As an alternative while recreating the tarball, one can attempt higher compression rates with j (for a .bz2) or J (for a .xz) instead of just z (for a more common .gz compressed file). On this particular example the sizes become: 2.6M (.gz), 2.2M (.bz2) and 1.8M (.xz).

GNU - Build Automation

Solaris 11.3 includes many modern packages, among them some essential GNU build automation ones. The problem is that the pace of update of these third-party packages is much faster than what get updated through support repository, needless to say the public release repository. In order to better keep up with this third-party pace, there's no other better alternative than to get the sources and build ourselves what's needed, in the hope (and unfortunately sometimes it's no more than just a hope) that the sources and the build tools are compatible and available to Solaris.

Source-tree snapshots

While setting to build projects which source-codes are distributed as a source-code tarball it's convenient to prepare some grounds before simply extracting the tarball to a plain directory on a "general" file-system. That's because while fine-tuning or experimenting on building the project it's often necessary to keep track or manage the state of the source-code tree in order to not mess things up.

In Solaris, part of this ground preparation is particularly powerful and elegant by virtue of the ZFS, specially due to the possibility of easy snapshot, clone, hold / release, send / receive and rollback operations. To take advantage of all this power it's necessary to set up some delegated ZFS permissions to the base dataset where the source-code tree will reside. This dataset could be remote, shared via NFS or SMB, but more typically it is local to an individual developer system and usually located as a descendant to its home directory.

GCC - library search dirs

Again, among a lot of detail to pay attention when compiling and linking a program is where the default libraries are being looked for. The general answer found everywhere is: "In the default system locations". But that may be vague, so here's my refinement to that general anwser, for instance, for GCC:

GCC - why build

I was studying C++11 when I faced an awful bug regarding locales in the Solaris 11.3 x86_64 platform either on plain GCC 4.8.2 or Developer Studio 12.5 which for C++11 relies on a GCC library by means of the -std=c++11 compiler flag. Simply put the feature doesn't work at all, except for C or POSIX locales, which are essentially the one and the same as the traditional USA localization since the legacy C era. It doesn't matter if I attempt any of the following forms:

...
std::locale user_native( "" )
...

 or

...
std::locale en_US( "en_US.UTF-8" )
...

Both forms fail badly with the following ugly message:

terminate ... throwing ... 'std::runtime_error'
... locale::facet::_S_create_c_locale ... not valid
Abort

Logging - Basics #2

After my introduction to logging it became clear to me that I needed to improve the initial ideas. As a process, the enhancements couldn't be achieved all at once, but gradually, step by step, according to the Natural laws :-) More specifically, I tried to address points 1, 2 and 4 from my initial attempt.

A new abstract type is to provide a general interface for all kinds of logging devices, not only file-system as firstly attempted. With this in mind, a log just need to be opened, written-to and finally closed when done. I tried to reflect such a model as shown next.

A CSS styling tool

I've been writing some code for my blog posts and every time I need to insert some excerpts of these codings it's a somewhat painful task to format it the precise way I'd like to. So I reached a threshold point where it would be worthy to set aside some time to look for some tool to help me achieve my goal. Fortunately, NetBeans, the IDE I've been using more frequently leads me half-way to where I want, by providing a Print to HTML... option:

But, unfortunately, this option does a nice job that goes beyond what I need. And the problem is that by going beyond it turns itself useless to address my current needs. It's true that it offers a tiny bit of flexibility:

But it is insufficient. The problem is that the syntax-highlighting is formatted via CSS internal styles of type text/css within a HTML comment within the well-known <style> tag. What is supposed to reflect the syntax-highlighting is within <span> tags of specific classes. Hence, I can't use it like that (not even portions of it) right away here in Blogger as I want:

<!DOCTYPE ... > 
<html>
<head> 
<title>framework.hxx</title> 
<meta ... > 
<style type="text/css">
<!--
body {...}
pre {...}
table {...}
.highlight-caret-row {...}
.ST1 {...}
.preprocessor {...}
.comment {...}
.literal {...}
.pragma-omp-keyword-directive {...}
.ST0 {...}
--> 
</style>
</head> 
<body>
...
11 <span class="literal">virtual</span>
  ~<span class="ST1">file</span>() {}
...

That is, not even portions of the above produce:
(unless with a completely inviable bit-by-bit editing)

11 virtual ~file() {}

Logging - Basics #1

Logging is an important aspect of computer science as it provides a way of tracing back exactly or approximately to some degree what have happened in relation to a certain computation in-between a certain time interval. Of course there are many kinds of log with respect to what, where and how the operation is done and what, where and how its results are archived for later retrieval and analysis. This definition should suffice for a quick startup :-)

Depending on the system there may be different kinds of logging support and facilities but I don't know of a single system that doesn't provide at least one kind of logging per each of its subsystems. But that is of course strongly platform dependent, thus not easily or possibly portable. Because of this it is generally useful for an application (or program) to also plan its own logging features that will work across any platform it supports.

On this post I present one very simple solution consisting on a mixed strategy of encapsulating an old-style (C) logging in a C++ object. It's definitely not a final and optimal solution. There is a long way to perfect it, but nevertheless it's a start. As having traces of an old-style logging it suffers from typical problems from the past related to the care that must be taken with respect to printf-like functions and its formatting strings and respective variadic parameters. As being a helper C++ concrete object the logging interface could be more elaborated and generalized by a new interface layer consisting of an abstract C++ class factoring out the most fundamental aspects of logging. But for now it's just as it is, and it works despite of these main deficiencies.

The implementation here is target at a 64-bit Unix system but it should be easily ported to another platform such as a Windows system. The log is recorded to a standard line-oriented file where each record begins with a simple timestamp marker. Each line is "freely" composed by means of an encapsulated standard C printf-like call. Let's see how...

Studio under Solaris

I already talked about NetBeans under Solaris which is great together with GCC, but even greater is the Solaris / Developer Studio and its native optimizing compiler and tools. By the time of this writing the current version is 12.5 (developerstudio-125 is the pkg5 name) which by now (finally!) virtually includes full (regression: locales don't work!) support to C++ 11 and already some selected features of C++ 14.

Appart from the lenghty download from the Oracle public release Internet repository, my installation notes on the links above present some more powerful considerations that the graphical Package Manager cannot offer, so check it out!

NOTE

The public release Internet address is https://pkg.oracle.com/solarisstudio/release but you'll only be able to access if you first register at http://pkg-register.oracle.com/ and later generate a PKCS12(1openssl) a.k.a. a PFX file to install on your browser.

NOTE

After launching Solaris / Developer Studio for the very first time, it may be convenient to adjust the default project folder much the same as its done for NetBeans. Instead of looking for the projectui.propeties file at ~/.netbeans, do so at ~/.solstudio or ~/.oracledevstudio.

   

Formatting issues

Stream formatting is very cool in C++ but with the power and flexibility comes the complexity. It's true that many limitations of the traditional C-style I/O were addressed by the streams framework and new I/O operators (put-to and get-from), manipulators and much more. But that didn't happen on a blink-of-an-eye. It's been evolving since many years, sometimes being completely revamped. A sad legacy is that in some (or many?) circumstances, backward compatibility takes precedence over some "fixes" or adjustments. Then it's not rare to have to carry on with some old "misbehavior" at some corners.

For example, consider the following output:
(I could argue that it has "unexpected" behavior; disregard the "[" and "]")

Formatting flags ...       // OK

*******Hello, there!       // OK; better if left aligned

Number 1: .........e       // OK
Number 2: c^^^             // "Should be" [12^^]
Number 3: 14               // "Should be" [20]

Floating 1: 8.33^^^^^^     // "Should be" [      8.33]
Floating 2: 1.83           // "Should be" [  1.83]
Floating 3: 3.33^^^^^^     // "Should be" [     3.333]

Going home...^^            // "Should be" [**Going home...]

I mean "should be" assuming that:

• member function calls would permanently change the formatting state
• manipulators calls would temporarily override the formatting state

But as you can see that's not the case and there's no even a "congruent misbehavior". Some member functions and manipulators (fill(), hex, left) permanently change behavior, others (width(), setw()) temporarily override, yet others misbehave strangely as precision (which should affect only how many decimal places to use and not the integral part as well!). In essence each situation is very simple, but when mixed on code, things can become quickly strange, perhaps a nightmare. A programmer can become quickly stressed and choose a less than optimal quick way out! But that's bad in the long run; that's not quality programming. Even the most simple programs such as rc-03 or the one listed below (which generated the above output) can face such annoyances when trying to print formatted output.

Formatting

I/O streams formatting is achieved by more than one framework class working together. At the core are controlling flags, but there are other attributes that can't be described/signaled by a flag, such as the (desired) width of a certain I/O field and how to pad it as necessary. Nevertheless, I'll concentrate on the flags as they form the core.

One the second ISO C++ 11 standard (ISO/IEC 14882-2011(E)), section 27.5.3.1.2, we have the description of the streams formatting flags type, std::ios_base::fmtflags, which builds on a bitmask type (section 17.5.2.13), which is an enumerated type (section 7.2), which is a compound type (section 3.9.2), and so on...

Among other things, this means that, as a bitmask, it has most (if not all) the well-defined bit operations one might expect. But the problem is that this not speak aloud as it should be, which may cause some confusion or obfuscation, resulting in much less than optimal utilization of the feature. This is one is just one out of several other similar situation appearing to establish a mild kind of obscurantism that do not help in getting the most out of the streams framework, which is very nice, flexible and powerful.

The SemWare Editor

If you do not yet know The SemWare Editor get to know it ASAP!
It isn't a full-blown IDE and in reality it never intended to be so.
What it really is is a powerful professional text editor.
Sadly it only has versions for Windows and DOS.
Even under Windows it isn't a GUI application.
It runs as a console window and thats OK!
No regrets, unless you're too lazy! ;-)

TSE:

Is fast! Very fast! Very much faster!
Much faster than any GUI editor you may know.

Is very, if not completely, customizable!
Its degree of customization is unbelievable.

Is possibly more powerful than VIM.
But certainly orders of magnitude easier to master.

TSE's internal macro language is very familiar and easy to learn.
To better understand its capabilities, check its features!

Trying to compare NetBeans' editor against TSE is unfair!
The NetBeans' degree of keyboard customizations is a joke.
TSE wins by far. But NetBeans wins as an IDE, of course.
But regarding text editing the game is over, TSE wins.

Even if you have a UNIX box for compiling and running, it would be advisable considering a (local or remote, virtual or not) Windows box just for the editing activity taking advantage of the superior text editing capabilities of TSE.

OPINION

In fact, the more professional you get, the less you need all those bells and whistles for text editing and building. You power and productivity will bias towards the bare bone tools without all those layers of fat of GUIs. Nevertheless, IDEs such as NetBeans survives because they integrate quite a lot of other goodies that may be important at some point of a project, remembering that if a project is growing extremely complex then there's probably some flaw or something wrong with its original design and/or engineering. The industry is rich of such defects, ranging from operating systems to all kinds of services and applications. That's all about Quality which is frequently sacrificed for Economic Pressures. And, by the way, much of this has to do with why I'm not fond of DOS, Windows, .NET, Java, Visual Studio, NetBeans and so on, on and on... All full of fat, overheads and inefficiencies and flaws which frequently require a lot of fixes, including security patches. One generally spends a great deal of time and money with maintenance to get a proportionally much lower ROI. A nightmare, in fact, a regularly employed deceiving strategy. I do not agree with this status-quo. I believe that systems and applications could be much more efficient and present much greater quality than they exhibit today.

   

NetBeans 8.1 under Solaris

If you are up to GCC under Solaris you may eventually get tempted to try a GUI development environment a.k.a Integrated Development Environment (IDE). The only major player that works on Solaris is NetBeans, but even if it weren't the only one, it would certainly be amongst the best ones. In fact, the only other major IDE that compete with NetBeans on other platforms is Eclipse, but NetBeans seems to have an edge in terms of software flexibility and openness.

The platform independent installation option of NetBeans is extremely simple to deal with. Basically, just download it and extract it to /opt. The extraction process may be tied to a subdirectory named netbeans, so that if you intend to keep multiple versions of side by side, this would probably be a problem. To workaround this lack of flexibility in the delivery method I suggest that you append version suffixes to the directory name, such as -81. Then, before setting to the extraction process, make sure you have a symbolic link from netbeans to netbeans-81.
 
NOTE

I'm completely ignoring here any slightly more sophisticated ZFS approach to dedicate a dataset under rpool/VARSHARE to be mounted under or linked at /opt. The one before the last sentence of the 1^st § and the 1^st sentence of the 2^nd § of an example of a more elaborate file-system setup can shed some light on what I mean by "a slightly more sophisticated file-system approach".

$ zfs list -o name,mountpoint .../netbeans-81
NAME              MOUNTPOINT
.../netbeans-81   /opt/netbeans-81
 
Please, note that rpool/VARSHARE/netbeans-81 was used above just as an example. It's not the recommended dataset to do so. In general, there should be one or more dedicated datasets to hold /opt subdirectories.

That is:

# cd /opt
# ln -s netbeans-81 netbeans
# unzip -q /tmp/netbeans-8.1-201510222201-cpp.zip
# rm netbeans

$ ls -lhtr /opt/netbeans-81/
total 670
-rw-r--r--  1 root root ... THIRDPARTYLICENSE.txt
-rw-r--r--  1 root root ... README.html
-rw-r--r--  1 root root ... netbeans.css
-rw-r--r--  1 root root ... LICENSE.txt
-rw-r--r--  1 root root ... CREDITS.html
drwxr-xr-x  7 root root ... nb
drwxr-xr-x  2 root root ... etc
drwxr-xr-x 11 root root ... harness
drwxr-xr-x  9 root root ... ide
drwxr-xr-x  9 root root ... platform
drwxr-xr-x  7 root root ... cnd
drwxr-xr-x  7 root root ... dlight
drwxr-xr-x  2 root root ... bin

Perform the first run as root but before that set up properly the X11 authorization and the DISPLAY environment variable as described in this other post X11 & SSH & SU. Launch NetBeans by executing and the file /opt/netbeans-81/bin/netbeans, accept the terms of the agreement and then exit at your own discretion. At this point you're pretty much done.

I would recommend to setup the default project folder, otherwise one will get a ~/NetBeansProjects which may be less than ideal. To address this, after a first launch, look for the projectui.properties file in ~/.netbeans and edit the value of projectsFolder entry (and any other eventual ocurrences of the unwanted path):

$ find ~/.netbeans -name projectui.properties

One final detail you may like to set up is a GNOME launcher under the menu Application | Developer Tools. In addition to referencing /opt/netbeans-81/bin/netbeans you'll probably reference the nice icon located at /opt/netbeans-81/nb/netbeans.png :