Kyle

Building GCC 4.8 on RHEL5.8

Mon, 29 Apr 2013 15:42:42 -0600

I didn't find this solution anywhere else on the internet, so I figured I'd post it here... When building GCC 4.8.0 on an up-to-date RHEL 5.8 system, it died, complaining that in the file @libstdc++-v3/libsupc++/unwind-cxx.h@, there's a macro (@PROBE2@) that cannot be expanded. It complains about things like this: bc. In file included from ../../../../libstdc++-v3/libsupc++/unwind-cxx.h:41:0, from ../../../../libstdc++-v3/libsupc++/eh_throw.cc:26: ../../../../libstdc++-v3/libsupc++/eh_throw.cc: In function ‘void __cxxabiv1::__cxa_throw(void*, std::type_info*, void (*)(void*))’: ../../../../libstdc++-v3/libsupc++/unwind-cxx.h:45:34: error: unable to find string literal operator ‘operator"" _SDT_S’ #define PROBE2(name, arg1, arg2) STAP_PROBE2 (libstdcxx, name, arg1, arg2) ^ The most helpful answer I could find was "this one":http://gcc.gnu.org/ml/gcc-help/2013-04/msg00012.html which didn't actually HELP so much as point a good finger at the culprit: SystemTap. Never heard of it? Neither had I. Their headers are apparently not compatible with C++11, and need to be updated. Don't have root? Heh, have fun with that. Of course, telling GCC to ignore SystemTap is not possible, that I can tell, unless SystemTap happened to be installed in an unusual place. So, instead, we have to resort to convincing GCC that it's not installed. Unfortunately, that can get tricky. What I ended up having to do was edit @x86_64-unknown-linux-gnu/libstdc++-v3/config.h@ and comment-out the line that says bc. #define HAVE_SYS_SDT_H 1 ...so that it reads this instead: bc. /*#define HAVE_SYS_SDT_H 1*/ It's not a *good* solution, of course, because I'm coming along behind the configuration logic and changing some of the answers without ensuring that there weren't conditional decisions made on that basis, AND since GCC builds itself several times to bootstrap into a clean, optimized product, you have to make that edit multiple (three) times. Basically, this is a horrible horrible hack around the problem. BUT, this works, is simple, and gets me a working compiler.

"Trump" Puts it into Simple Terms?

Mon, 02 Jul 2012 11:14:01 -0600

A friend recently sent me this pithy sentence, purportedly penned by Donald Trump: bq.. 'Let me get this straight ... We're going to be "gifted" with a health care plan we are forced to purchase and fined if we don't, Which, purportedly covers at least ten million more people, without adding a single new doctor, but provides for 16,000 new IRS agents, written by a committee whose chairman says he doesn't understand it, passed by a Congress that didn't read it (but exempted themselves from it), and signed by a Dumbo President who smokes, with funding administered by a treasury chief who didn't pay his taxes, for which we'll be taxed for four years before any benefits take effect, by a congress which has already bankrupted Social Security and Medicare, all to be overseen by an obese surgeon general and financed by a country that's broke!! What the hell could possibly go wrong?' p. Let's put aside the over-use of exclamation marks for the moment, as well as arguments about whether Trump actually said it (probably not, but whatever), and dismantle this piece of garbage. You are "gifted" with a system of roads that you are forced to pay for. You are "gifted" with the protection of a military that you are forced to pay for. Let's not get bent out of shape about the things government requires us to pay for. It's called living in a civilization; get used to it. As for covering more people… the expansion of Medicaid alone (expanding coverage to adults with up to 133% of poverty, aka $18,310/yr) is expected to cover as many as "17 million more Americans":http://www.wptv.com/dpp/money/consumer/affordable-care-act-answers-to-commonly-asked-questions, so I have no idea where "Trump's" number came from. Citations would be nice. As for those 16,000 new IRS agents? According to factcheck.org, "that's complete baloney":http://www.factcheck.org/2010/03/irs-expansion/. They describe the claim as "wildly inaccurate", stemming "from a partisan analysis based on guesswork and false assumptions, and compounded by outright misrepresentation." The Affordable Care Act "came from the Senate Finance Committee":http://www.finance.senate.gov/issue/?id=32be19bd-491e-4192-812f-f65215c1ba65, starting way back in 2007. The chair of that committee is Senator Max Baucus, and I cannot find a quote from him on the internet anywhere suggesting that he does not understand it (though I admit, absence of proof is not proof of absence). There's plenty on there about him saying he didn't read every page of it, though. But that kind of inaccuracy does not make me more interested in believing the author, even if it could be Trump. Indeed, whether Congress read it or not, they are "NOT exempt from it":http://www.forbes.com/sites/rickungar/2011/12/08/congress-exempted-from-obamacare/. In fact the law says "Members of Congress and congressional staff" will only be offered plans created by the law or offered through exchanges established by the law. I'm not sure what exactly Obama's smoking habit has to do with anything, apparently it means he's a hypocrite somehow? I don't see it. But now we're into the personal attacks portion of Trump's little diatribe, which, whether true or not, mean nothing about the law and are just ad hominem attacks. Presumably he thinks that the personal finances of the "treasury chief" or personal habits of the Surgeon General are relevant to health care legislation. On top of that, Trump apparently can't be bothered to type "Secretary of the Treasury" - Britain is the country that uses "Chief" to refer to the head of the treasury (as in "Chief Secretary to the Treasury"); I guess Trump got confused. As for Social Security being bankrupt, perhaps Trump should explain things to Forbes, who seems to think he has fallen afoul of a "common logical error":http://www.forbes.com/sites/johntharvey/2011/04/08/why-social-security-cannot-go-bankrupt/. By the same token, Medicare cannot go bankrupt either, but it's worth pointing out that Medicare is in much better financial shape "as a result of the Affordable Care Act":http://www.cbpp.org/cms/index.cfm?fa=view&id=3532. And is the United States broke? No. "As Bloomberg's David Lynch puts it":http://www.bloomberg.com/news/2011-03-07/bonds-show-why-boehner-saying-we-re-broke-is-figure-of-speech.html, all evidence, from the rate we pay on borrowing, to tax revenue as a percentage of the economy, to the trend in prices for insuring US debt, suggests quite the opposite. But, you know, when you're bloviating, why let facts stand in your way? This is a nearly random collection of GOP talking points (many of which conflict with reality), rearranged into invective against the health care law (which has significant flaws, but none of which have been touched on here). Surprise! I suppose, if this is truly Trump's work, that I should have expected as much given that he still wants to argue with Hawaii about Obama's birth certificate.

A Use for Volatile in Multi-threaded Programming

Fri, 08 Jun 2012 13:24:31 -0600

As anyone who has done serious shared-memory parallel coding knows, the @volatile@ keyword in C (and C++) has a pretty bad rap. There is no shortage of people trying to explain why this is the case. Take for example this article from the original author of Intel Threaded Building Blocks (a man who, I assure you, knows what he's talking about): "Volatile: Almost Useless for Multithreaded Programming":http://software.intel.com/en-us/blogs/2007/11/30/volatile-almost-useless-for-multi-threaded-programming/. There are others out there who decry @volatile@, and their arguments are right to varying degrees. The heart of the issue is that first, @volatile@ is EXPLICITLY IGNORABLE in the C specification, and second, that it provides neither ordering guarantees nor atomicity. Let me say that again, because I've had this argument: bq. *VOLATILE DOES NOT ESTABLISH ORDERING REQUIREMENTS* bq. *VOLATILE DOES NOT PROVIDE ATOMICITY* But I'm not here to talk about that; I want to talk about a place where I found it to be critical to correctness (as long as it isn't ignored by the compiler, in which case creating correct code is _painful_). Honestly, I was quite surprised about this, but it makes sense in retrospect. I needed a double-precision floating point atomic increment. Most increments, of the @__sync_fetch_and_add()@ variety, operate exclusively on integers. So, here's my first implementation (just the x86_64 version, without the PGI bug workarounds): bc. double qthread_dincr(double *operand, double incr) { union { double d; uint64_t i; } oldval, newval, retval; do { oldval.d = *operand; newval.d = oldval.d + incr; __asm__ __volatile__ ("lock; cmpxchgq %1, (%2)" : "=a" (retval.i) : "r" (newval.i), "r" (operand), "0" (oldval.i) : "memory"); } while (retval.i != oldval.i); return oldval.d; } Fairly straightforward, right? But this has a subtle race condition in it. The dereference of operand gets translated to the following assembly: bc. movsd (%rcx), %xmm0 movd (%rcx), %rax See the problem? In the assembly, it's actually dereferencing @operand@ TWICE; and under contention, that memory location could change values between those two instructions. Now, we might pause to ask: why is it doing that? We only told it to go to memory ONCE; why would it go to twice? Well, a certain amount of that is unfathomable. Memory accesses are usually slow, so you'd think the compiler would try to avoid them. But apparently sometimes it doesn't, and technically, dereferencing non-volatile memory multiple times is perfectly legal. The point is, this is what happened when compiling with basically every version of gcc 4.x right up through the latest gcc 4.7.1. In any event, there are two basic ways to fix this problem. The first would be to code more things in assembly; either the entire loop or maybe just the dereference. That's not an appealing option because it requires me to pick which floating point unit to use (SSE versus 387 versus whatever fancy new stuff comes down the pike), and I'd rather let the compiler do that. The second way to fix it is to use @volatile@. If I change that dereference to this: bc. oldval.d = *(volatile double *)operand; Then the assembly it generates looks like this: bc. movsd (%rcx), %xmm0 movd %xmm0, %rax Problem solved! As long as the compiler doesn't ignore the volatile cast, at least... So, for those who love copy-and-paste, here's the fixed function: bc. double qthread_dincr(double *operand, double incr) { union { double d; uint64_t i; } oldval, newval, retval; do { oldval.d = *(volatile double *)operand; newval.d = oldval.d + incr; __asm__ __volatile__ ("lock; cmpxchgq %1, (%2)" : "=a" (retval.i) : "r" (newval.i), "r" (operand), "0" (oldval.i) : "memory"); } while (retval.i != oldval.i); return oldval.d; } (That function will not work in the PGI compiler, due to a compiler bug "I've talked about previously.":http://www.memoryhole.net/kyle/2010/06/pgi_compiler_bug.html)

Google Breaks its own DKIM Signatures

Tue, 05 Jul 2011 09:14:08 -0600

So, Google, vaunted tech company that it is, seems to be doing something rather unfortunate. One of my friends/users, who uses Gmail as a repository for his email, recently notified me that email sent to him from other Gmail accounts showed up as "potentially forged". Interestingly, this only happened for email that was sent from Gmail to an external server (i.e. mine) that then got relayed back to Gmail. Examining the "raw original", here's the differences: # The relayed body has an extra newline at the end (this may be an artifact of Gmail's view-raw-message feature) # The relayed body quotes the display-name in the From header (or any other email header with a display-name) # The relayed body strips off the weekday name from the Date header Now, since this doesn't happen to messages sent from-Gmail-to-Gmail directly, and I'm very certain that my email server isn't doing it either (I sniffed the outbound SMTP traffic to prove it), I'm guessing that this message... "normalization", for lack of a better term... is a function of their ingress filter. But all of those changes are enough to invalidate the DKIM signature that Gmail generated... or, I suppose, anyone else's DKIM signature. Come on, Google, get your act together.

A C Lock-Free Hash Table Implementation

Thu, 02 Jun 2011 23:26:56 -0600

Well, I finally have one: a lock-free hash table implementation in C. The hash table I implemented is based on the work by Ori Shalev and Nir Shavit. Much of the code is similar to the example code they published in their paper "Split-Ordered Lists: Lock-Free Extensible Hash Tables":http://doi.acm.org/10.1145/872035.872049, back in 2006, but has been modified to support additional conveniences so it has a library-esque interface. There is one problem unique to this algorithm: It doesn't compact well if you go from lots and lots of entries back down to just a few entries. Not that it compacts _horribly,_ but it doesn't shrink all the way back down; it keeps some meta-data around (what the algorithm calls "dummy keys"). On the other hand, it does quite well if you tend to have a rapidly rotating set of keys (even with random-ish key values) but with a ceiling on the number of them in the hash table at any one time: it uses a static set of dummy keys in that case. In an effort to reduce the impact of the compaction problem, I didn't implement the dynamic-array extension, so this implementation is performance-limited to around 2048 concurrent keys or so (though you can insert as many as you like, more than that will begin to get slow). Of course, there are three problems with this algorithm (and implementation) that are pretty typical: # It has the ABA problem. I think this can be solved (or at least mitigated) with generation counters, so I don't see it as a big issue. # It has the blind-delete problem. This is a really tough one. I _think_ this can be solved, but my current best thinking on the matter ends up devolving to a highly-contended CAS operation, which is obviously sub-optimal. # For simplicity, I didn't use memory pools for the nodes, so I end up doing malloc/free calls on insertion and delete. This isn't a correctness problem, but can become a performance bottleneck. Anyway, without further ado, "here's the code":http://www.memoryhole.net/kyle/2011/06/02/lf_hash.c. You can download it and compile it yourself; it is self-contained C99 code with a trivial (single-threaded) driver in @main()@. I've talked about concurrent hash tables before, "here":http://www.memoryhole.net/kyle/2007/10/intels_tbb_hash_is_severely_li.html and "here":http://www.memoryhole.net/kyle/2007/09/i_wish_i_had_a_cbased_lockfree.html, and those discussions may be interesting to anyone who found this code interesting and/or useful. Having re-read Chris Purcells' paper, I should point out that while he does have some very worthwhile code, his algorithm also has the compaction problem. He *does*, however, have a good summary of several designs, and of the drawbacks of each (primarily in the area of a need for garbage collection).

Gmail, DKIM, and DomainKeys

Mon, 10 Jan 2011 14:04:21 -0600

I recently spent a bunch of time trying to resolve some delivery problems we had with Gmail. Some of it was dealing with idiosyncratic issues associated with *our* mail system, and some of it, well, might benefit others. In our mail system, we use @qmail-qfilter@ and some custom scripts to manipulate incoming mail, along with a "custom shell script":http://www.memoryhole.net/qmail/#dkim I wrote to manipulate outbound mail. Inbound mail, prior to this, was prepended with three new headers: DomainKey-Status, DKIM-Status (and friends), and X-Originating-IP. Outbound mail was signed with both a DomainKey and a DKIM signature. All of my DomainKey-based manipulation was based on "libdomainkeys":http://domainkeys.sourceforge.net/ and, in particular, their @dktest@ utility. Yes, that library is technically out-of-date, but for a long time there were more DomainKey-compliant servers out there than DKIM-compliant servers, so... it made sense. The DKIM-based manipulation is all based on Perl's "Mail::DKIM":http://search.cpan.org/~jaslong/Mail-DKIM/ module, which seems to be quite the workhorse. Our situation was this: we have several users that use Gmail as a kind of "back-end" for their mail on this server. All of their mail needs to be forwarded to Gmail, and when they send from Gmail, it uses SMTP-AUTH to relay their mail through our server. This means that their outgoing mail is signed first by gmail, then by us. The domain of the outgoing signature is defined by the sender. So, first problem: we use procmail to forward mail. This means that all mail that got sent to these Gmail users got re-transmitted with a return-address of nobody@our-domain.com (the procmail default). Thus, we *signed* all of this relayed mail (because the sender was from one of the domains we have a secret-key for). This became a problem because all spam that got sent to these users got relayed, and signed, and so *we* got blamed for it (thus causing gmail to blacklist us occasionally). Gmail has a few "recommendations on this subject":http://mail.google.com/support/bin/answer.py?hl=en&answer=175365. Their first recommendation is to stop changing the return address (which is exactly the opposite of the "recommendation of SPF-supporters":http://www.openspf.org/FAQ/Forwarding, I'd like to point out). They also suggest doing our own spam detection and putting "SPAM" in the subject of messages our system thinks is spam. I used Gmail's recommended solution (which would also prevent us from signing outbound spam), adding the following lines to our procmailrc: bc. SENDER=`formail -c -x Return-Path` SENDMAILFLAGS="-f${SENDER}" This caused new problems. All of a sudden, mail wasn't getting through to some of the Gmail users AT ALL. Gmail would *permanently* reject the messages with the error message: bc. 555 5.5.2 Syntax error. u18si57222290ibk.46 It turns out that messages sent *From* the Gmail users often had _multiple_ @Return-Path@ headers. The same is true of messages from many mailing lists (including Google Apps mailing lists). This means that @formail@ would dutifully print out a multi-line response, which would then cause garbage (more or less) into the @sendmail@ binary, thereby causing invalid syntax, which is why Gmail was rejecting messages. On top of that, formail doesn't strip off the surrounding wockas, which caused sendmail to encode the @Return-Path@ header incorrectly, like this: bc. Return-Path: < > This reflects what would happen during the SMTP conversation with Gmail's servers: the double-wockas would be there as well, which is, officially, invalid SMTP syntax. The solution we're using now is relatively trivial and works well: bc. SENDER=`formail -c -x Return-Path | head -n 1 | tr -d'<>'` SENDMAILFLAGS="-f${SENDER}" Let me re-iterate that, because it's worth being direct. Using Gmail's suggested solution caused messages to %{color:red}_*+DISAPPEAR+*_%. %{color:red}_*+IRRETRIEVABLY+*_%. Granted, that was my fault for not testing it first. But still, come on Google. That's a BAD procmail recommendation. There were a few more problems I had to deal with, relating to DomainKeys and DKIM, but these are someone idiosyncratic to our mail system (but it may be of interest for folks with a similar setup). Here I should explain that when you send from Gmail through another server via SMTP-AUTH, Gmail signs the message with its DK key, both with a DKIM and with a DomainKeys header. This is DESPITE the fact that the @Return-Path@ is for a non-gmail domain, but because the @Sender@ is a gmail.com address, this behavior is completely legitimate and within the specified behavior of DKIM. The first problem I ran into was that, without a new @Return-Path@, the @dktest@ utility from DomainKeys would refuse to sign messages that had already been signed (in this case, by Gmail). Not only that, but it would refuse in a very bad way: instead of spitting out something that looks like a @DomainKey-Signature:@ header, it would spit out an error message. Thus, unless my script was careful about only appending things that start with @DomainKey-Signature:@ (which it wasn't), I would get message headers that looked like this: bc. Message-Id: <4d275412.6502e70a.3bf6.0f6dSMTPIN_ADDED@mx.google.com> do not sign email that already has a dksign unless Sender was found first DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gmail.com; h=mime-version That's an excerpt, but you can see the problem. It spit an invalid header (the error) into the middle of my headers. This kind of thing made Gmail mad, and rightly so. It made me mad too. So mad, in fact, that I removed libdomainkeys from my toolchain completely. Yes, I could have added extra layers to my script to detect the problem, but that's beside the point: that kind of behavior by a tool like that is malicious. The second problem I ran into is, essentially, an oversight on my part. My signing script chose a domain (correctly, I might add), and then handed the signing script a filename for the private key of that domain. HOWEVER, since I didn't explicitly tell it what domain the key was for, it attempted to discover the domain based on the other headers in the message (such as @Return-Path@ and @Sender@). This auto-discovery was only accurate for users like myself who don't use Gmail to relay mail through our server. But for messages from Gmail users, who relay via SMTP-AUTH, the script would detect that the mail's sender was a Gmail user (similar problems would arise for mailing lists, depending on their sender-rewriting behavior). So what it would do is assume that the key it had been handed was for that sender's domain (i.e. gmail.com), and would create an invalid signature. This, thankfully, was easy to fix: merely adding an explicit @--domain=$DOMAIN@ argument to feed to the signing script fixed the issue. But it was a weird one to track down! It's worth pointing out that the libdomainkeys @dktest@ utility does not provide a means of doing this. Anyway, at long last, mail seems to be flowing to my Gmail users once again. Thank heaven!

PGI Compiler Bug

Wed, 16 Jun 2010 11:01:35 -0600

I ran across another PGI compiler bug that bears noting because it was so annoying to track down. Here's the code: bc. static inline uint64_t qthread_cas64( volatile uint64_t *operand, const uint64_t newval, const uint64_t oldval) { uint64_t retval; __asm__ __volatile__ ("lock; cmpxchg %1,(%2)" : "=&a"(retval) /* store from RAX */ : "r"(newval), "r"(operand), "a"(oldval) /* load into RAX */ : "cc", "memory"); return retval; } Now, both GCC and the Intel compiler will produce code you would expect; something like this: bc. mov 0xffffffffffffffe0(%rbp),%r12 mov 0xffffffffffffffe8(%rbp),%r13 mov 0xfffffffffffffff0(%rbp),%rax lock cmpxchg %r12,0x0(%r13) mov %rax,0xfffffffffffffff8(%rbp) In essence, that's: # copy the newval into register @%r12@ (almost any register is fine) # copy the operand into register @%r13@ (almost any register is fine) # copy the oldval into register @%rax@ (as I specified with "a") # execute the ASM I wrote (the compare-and-swap) # copy register @%rax@ to the variable I specified Here's what PGI produces instead: bc. mov 0xffffffffffffffe0(%rbp),%r12 mov 0xffffffffffffffe8(%rbp),%r13 mov 0xfffffffffffffff0(%rbp),%rax lock cmpxchg %r12,0x0(%r13) mov %eax,0xfffffffffffffff8(%rbp) You notice the problem? That last step became @%eax@, so only the lower 32-bits of my 64-bit CAS get returned! The workaround is to do something stupid: be more explicit. Like so: bc. static inline uint64_t qthread_cas64( volatile uint64_t *operand, const uint64_t newval, const uint64_t oldval) { uint64_t retval; __asm__ __volatile__ ("lock; cmpxchg %1,(%2)\n\t" "mov %%rax,(%0)" : : "r"(&retval) /* store from RAX */ "r"(newval), "r"(operand), "a"(oldval) /* load into RAX */ : "cc", "memory"); return retval; } This is stupid because it requires an extra register; it becomes this: bc. mov 0xfffffffffffffff8(%rbp),%rbx mov 0xffffffffffffffe0(%rbp),%r12 mov 0xffffffffffffffe8(%rbp),%r13 mov 0xfffffffffffffff0(%rbp),%rax lock cmpxchg %r12,0x0(%r13) mov %rax,(%rbx) Obviously, not a killer (since it can be worked around), but annoying nevertheless. A similar error happens in this code: bc. uint64_t retval; __asm__ __volatile__ ("lock xaddq %0, (%1)" :"+r" (retval) :"r" (operand) :"memory"); It would appear that PGI completely ignores the bitwidth of output data!

qsort_r

Fri, 13 Nov 2009 16:41:22 -0600

Once upon a time, in 2002, the BSD folks had this genius plan: make the standard C qsort() function safe to use in reentrant code by creating qsort_r() and adding an argument (a pointer to pass to the comparison function). So "they did":http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/stdlib/qsort.c?annotate=1.12, and it was good. Five years later, in 2007, the GNU libc folks said to themselves "dang, those BSD guys are *smart*, I wish we had qsort_r()". Then some idiot said: WAIT! We cannot simply use the same prototype as the BSD folks; they use an evil license! We can't put that into GPL'd code! So the GNU libc folks solved the problem by "reordering the arguments":http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/include/stdlib.h?rev=1.42&content-type=text/x-cvsweb-markup&cvsroot=glibc. And now we have two, incompatible, widely published versions of qsort_r(), which both do the exact same thing: crash horribly if you use the wrong argument order. __ Okay, here's an alternate history: ... Then some lazy idiot said: WAIT! The existing qsort_r() is a poor design that requires a second implementation of qsort()! If we throw out compatibility with existing qsort_r() code, we can implement qsort() as a call to qsort_r() and no one will ever know! __ Either way, we all lose. (I have no argument with the alternate history point... but why'd you have to call it the exact same thing??? Call it qsort_gnu() or something! Make it easy to detect the difference!)

More Compiler Complaints: PGI Edition

Wed, 10 Jun 2009 15:39:43 -0600

Continuing my series of pointless complaining about compiler behavior (see "here":http://www.memoryhole.net/kyle/2008/08/more_compiler_complaints_sparc.html and "here":http://www.memoryhole.net/kyle/2008/01/apples_compiler_idiocy.html for the previous entries), I recently downloaded a trial version of PGI's compiler to put in my Linux virtual machine to see how that does compiling qthreads. There were a few minor things that it choked on that I could correct pretty easily, and some real bizarre behavior that seems completely broken to me. h3. Subtle Bugs in _My_ Code Let's start with the minor mistakes it found in my code that other compilers hadn't complained about: bc. static inline uint64_t qthread_incr64( volatile uint64_t *operand, const int incr) { union { uint64_t i; struct { uint32_t l, h; } s; } oldval, newval; register char test; do { oldval.i = *operand; newval.i = oldval.i + incr; __asm__ __volatile__ ("lock; cmpxchg8b %1\n\t setne %0" :"=r"(test) :"m"(*operand), "a"(oldval.s.l), "d"(oldval.s.h), "b"(newval.s.l), "c"(newval.s.h) :"memory"); } while (test); return oldval.i; } Seems fairly straightforward, right? Works fine on most compilers, but the PGI compiler complains that "%sli" is an invalid register. Really obvious error, right? Right? (I don't really know what the %sli register is for either). Turns out that because @setne@ requires a byte-sized register, I need to tell the compiler that I want a register that can be byte-sized. In other words, that @"=r"@ needs to become @"=q"@. Fair enough. It's a confusing error, and thus annoying, but I am technically wrong (or at least I'm providing an incomplete description of my requirements) here so I concede the ground to PGI. h3. Unnecessary Pedantry And then there are places where PGI is simply a bit more pedantic than it really needs to be. For example, it generates an error when you implicitly cast a @volatile struct foo *@ into a @void *@ when calling into a function. Okay, yes, the pointers are different, but... most compilers allow you to implicitly convert just about any pointer type into a @void *@ without kvetching, because you aren't allowed to *dereference* a void pointer (unless you cast again, and if you're casting, all bets are off anyway), thus it's a safe bet that you want to work on the *pointer* rather than what it points to. Yes, technically PGI has made a valid observation, but I disagree that their observation rises to the level of "warning-worthy" (I have no argument if they demote it to the sort of thing that shows up with the @-Minform=inform@ flag). h3. Flat-out Broken But there are other places where PGI is simply wrong/broken. For example, if I have (and use) a @#define@ like this: bc. #define PARALLEL_FUNC(initials, type, shorttype, category) \ type qt_##shorttype##_##category (type *array, size_t length, int checkfeb) \ { \ struct qt##initials arg = { array, checkfeb }; \ type ret; \ qt_realfunc(0, length, sizeof(type), &ret, \ qt##initials##_worker, \ &arg, qt##initials##_acc, 0); \ return ret; \ } PARALLEL_FUNC(uis, aligned_t, uint, sum); PGI will die! Specifically, it complains that @struct qtuisarg@ does not exist, and that an identifier is missing. In other words, it blows away the whitespace following initials so that this line: bc. struct qt##initials arg = { array, checkfeb }; \ is interpreted as if it looked like this: bc. struct qt##initials##arg = { array, checkfeb }; \ But at least that's easy to work around: rename the struct so that it has a _s at the end! Apparently PGI is okay with this: bc. struct qt##initials##_s arg = { array, checkfeb }; \ ::sigh:: Stupid, stupid compiler. At least it can be worked around. h3. Thwarting The Debugger PGI also bad at handling static inline functions in headers. How bad? Well, first of all, the DWARF2 symbols it generates (the default) are incorrect. It gets the line-numbers right but the file name wrong. For example, if I have an inline function in @qthread_atomics.h@ on line 75, and include that header in @qt_mpool.c@, and then use that function on line 302, the DWARF2 symbols generated will claim that the function is on line 75 of @qt_mpool.c@ (which isn't even correct if we assume that it's generating DWARF2 symbols based on the pre-processed source! and besides which, all the other line numbers are from non-pre-processed source). You CAN tell it to generate DWARF1 or DWARF3 symbols, but then it simply leaves out the line numbers and file names completely. Handy, no? h3. Everyone Else is Doing It... Here's another bug in PGI... though I suppose it's my fault for outsmarting myself. So, once upon a time, I (think I) found that some compilers require assembly memory references to be within parentheses, while others require them to be within brackets. Unfortunately I didn't write down which ones did what, so I don't remember if I was merely being over-cautious in my code, or if it really was a compatibility problem. Nevertheless, I frequently do things like this: bc. atomic_incr(volatile uint32_t *op, const int incr) { uint32_t retval = incr; __asm__ __volatile__ ("lock; xaddl %0, %1" :"=r"(retval) :"m"(*op), "0"(retval) :"memory"); return retval; } Note that weird @"m"(*op)@ construction? That was my way of ensuring that the right memory reference syntax was automatically used, no matter what the compiler thought it was. So, what does PGI do in this instance? It actually performs the dereference! In other words, it behaves as if I had written: bc. atomic_incr(volatile uint32_t *op, const int incr) { uint32_t retval = incr; __asm__ __volatile__ ("lock; xaddl %0, (%1)" :"=r"(retval) :"r"(*op), "0"(retval) :"memory"); return retval; } when what I really wanted was: bc. atomic_incr(volatile uint32_t *op, const int incr) { uint32_t retval = incr; __asm__ __volatile__ ("lock; xaddl %0, (%1)" :"=r"(retval) :"r"(op), "0"(retval) :"memory"); return retval; } See the difference? Again, it's not hard to fix so that PGI does the right thing. And maybe I was being too clever in the first place. But dagnabit, my trick should work! And, more pointedly, it DOES work on other compilers (gcc and icc at the bare minimum, and I've tested similar things with xlc).

Muslim Demographics

Tue, 05 May 2009 11:16:28 -0600

I recently got sent a link to the "Muslim Demographics":http://www.youtube.com/watch?v=6-3X5hIFXYU video on YouTube. It's pretty alarmist, so I composed an email response. Since it might be interesting to have available to Google, here's my thoughts, as an antidote to the panic the video is peddling. Note that "snopes.com":http://www.snopes.com/politics/religion/demographics.asp has their own page discussing this video. They don't address the accuracy of the facts presented, but its an interesting read nevertheless. It is probably true that there is an aspect of evolution to religion. If we think of religion as a gene, the dominant religion will be defined (over the long run) by the extent to which it benefits or promotes reproduction, just as any piece of DNA does. Of course, given evangelism, it may be more apt to think of religion as similar to a virus that gets passed from person to person, but I doubt that'd be a popular viewpoint. In any case, of course, religion *isn't* a gene, because we'd have to assume that people stick with the religion they're born with. Christianity started with just 12 Christians, after all, and every last one of them was male (and thus couldn't pass on any genes without help from an originally non-Christian woman). That said, I'm skeptical of this video's claims. For one thing, practically no sources are cited (2.11 is the bare minimum to maintain a society? Says who? How do we know?). For another, it's interesting to note what gets left out. For example, the video says the Muslim population skyrocketed from 82,000 in the UK to several million, but according to the CIA World Factbook, Muslims are a whopping "2.7% of the UK population":https://www.cia.gov/library/publications/the-world-factbook/geos/uk.html#People. In the Netherlands, the CIA World FactBook says that Muslims are a crushing "5.8% of the population":https://www.cia.gov/library/publications/the-world-factbook/geos/nl.html#People. Not only that, but it says that the Netherlands have a fertility rate of 1.66. Woo! Scary! Now that I come to think of it... let me look this up. table{border-collapse:collapse}. |_{border-right:1px solid black;border-bottom:1px solid black;padding:2px}. Country|_{border-right:1px solid black;border-bottom:1px solid black;padding:2px}. Video Claimed Fertility Rate|_{border-bottom:1px solid black;padding:2px}. CIA FactBook Fertility Rate| |{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. France|{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. 1.8|{border-bottom:1px dotted grey;padding:2px}. "1.98":https://www.cia.gov/library/publications/the-world-factbook/geos/fr.html#People | |{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. England|{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. 1.6|{border-bottom:1px dotted grey;padding:2px}. "1.66":https://www.cia.gov/library/publications/the-world-factbook/geos/uk.html#People | |{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. Greece|{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. 1.3|{border-bottom:1px dotted grey;padding:2px}. "1.37":https://www.cia.gov/library/publications/the-world-factbook/geos/gr.html#People | |{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. Germany|{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. 1.3|{border-bottom:1px dotted grey;padding:2px}. "1.41":https://www.cia.gov/library/publications/the-world-factbook/geos/gm.html#People| |{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. Italy|{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. 1.2|{border-bottom:1px dotted grey;padding:2px}. "1.31":https://www.cia.gov/library/publications/the-world-factbook/geos/it.html#People | |{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. Spain|{border-right:1px solid black;border-bottom:1px dotted grey;padding:2px}. 1.1|{border-bottom:1px dotted grey;padding:2px}. "1.31":https://www.cia.gov/library/publications/the-world-factbook/geos/sp.html#People | |{border-right:1px solid black;padding:2px}. EU|{border-right:1px solid black;padding:2px}. 1.38|{padding:2px}. "1.51":https://www.cia.gov/library/publications/the-world-factbook/geos/ee.html#People | Yikes - they didn't get a single one correct! The closest was England, and even there they rounded in the wrong direction. So should we really believe that French Muslims have a birth rate of 8.1? Think about that, EIGHT kids on average, which means that for every childless Muslim woman, there's another out there with SIXTEEN KIDS. That sounds totally plausible, right? The "Population Reference Bureau":http://www.prb.org/Articles/2008/muslimsineurope.aspx (who I've never heard of, but they were linked to by "about.com":http://geography.about.com/b/2008/03/04/muslim-fertility-rate-in-europe.htm) says that in Austria, Muslim women had a fertility rate of 3.1 in 1981, but that by 2001 the rate was a mere 2.3 (apparently they didn't get the memo from their French kindred). That reflects the falling fertility rates in Muslim countries the immigrants came from. For example, in Turkey the fertility rate dropped from 3.3 in 1985 to 2.2 in 2003. According to the CIA World FactBook, Turkey's current fertility rate is "2.21":https://www.cia.gov/library/publications/the-world-factbook/geos/tu.html#People. In Morocco it fell from 4.5 to 2.5 in the same time period (CIA says "2.51":https://www.cia.gov/library/publications/the-world-factbook/geos/mo.html#People). And get this: in Iran, it fell from 5.6 to 2.1 in 2003. The CIA World FactBook currently pegs the Iranian fertility rate at a paltry "1.71":https://www.cia.gov/library/publications/the-world-factbook/geos/ir.html#People! 1.71! 1.71! The Iranian culture cannot survive! The US fertility rate is "2.05":https://www.cia.gov/library/publications/the-world-factbook/geos/us.html#People! We will crush them with our progeny! MUAHAHAHAHA! Oh, wait, does that not serve the purpose of getting people riled up? I'm thinking of a word... _fearmongering_! That's the one. It reminds me a lot of grade school. I remember playing with a bunch of computer "simulations" that showed that the world population was exploding and that we'd run out of food by the turn of the millenium. The reason they were wrong is that they made assumptions without realizing it. For example, they assumed that food production would stay constant, and that fertility rates would stay constant. Surprise! They didn't. And THAT reminds me of another quote: bq. Scientists have shown that the moon is moving away at a tiny, although measurable distance from the earth every year. If you do the math, you can calculate that 65 million years ago, the moon was orbiting at a distance of about 35 feet from the earth's surface. This would explain the death of the dinosaurs ... the tallest ones, anyway. Don't believe everything you see on YouTube. ;)

SlingLink Killed My Network!

Mon, 27 Apr 2009 17:55:24 -0600

This isn't the most accurate title, but... I've got a "SlingLink Turbo":http://www.slingmedia.com/go/slt that I use for connecting my Macs upstairs to my cable modem downstairs. I went with a network-over-powerline option, because I've been having all kinds of intermittent interference problems with my wireless connectivity. So, rather than running an extra-long patch cable up the stairs and taping it down to the carpet, I went the SlingLink route. It seems to be designed specifically for SlingBox applications, but it forwards plain ol' ethernet signals, and it's about $40 cheaper than the NetGear equivalents. Huzzah for getting a bargain! First impression: fabulous! I went along happily for *weeks*, enjoying my newfound reliable network connection. Then I tried downloading the latest Ubuntu ISO images via BitTorrent, and within five to ten minutes, the internet connection went offline. If I went downstairs and turned the cable modem off-and-on again, the internet would come back. For five to ten minutes. Then it'd go down again. Surely, I say to myself, that's a cable-modem problem, right? I had to have the tech guys from Time Warner's Cable group come out (twice!) before I finally figured out that it wasn't their fault (the first time they said they replaced the splitter, and presto, the network was fine! I didn't go after the ISO again for a while so...). Turns out I didn't need to restart the cable modem, all I had to do was restart the SlingLink node and I'd get another five to ten minutes out of it. But it ONLY happens when BitTorrent is running; otherwise, the network connection is rock solid! Weird, no? So, to experiment, I tried limiting the BitTorrent connections: no dice. Then I tried limiting the BitTorrent *bandwidth* and all of a sudden the network would stay up. Somewhere between 100Kb/s and 150Kb/s is the cutoff. Something about BitTorrent's bandwidth seems to either confuse the SlingLink node OR triggers some sort of antiviral cutoff in the SlingLink hardware (either way is annoying). For the record, it's not a pure bandwidth issue: I can transfer files over the SlingLink network at speeds of over 400Kb/s. As soon as I introduce BitTorrent, though... down she goes. Maybe it's a packet-size issue. Maybe it's a connection-tracking issue. I have no idea. But at least now I know that SlingLink has its limitations. And now, so do you.

Congress's CRA Caused the Crash?

Mon, 06 Oct 2008 23:46:57 -0600

It's recently been suggested to me that the true cause of the subprime mess is that Congress forced banks to loan to poor people (also known as "people who cannot afford to repay the mortgage"), perhaps under the guise of fairness or avoiding racism or classism or something like that. There would appear to be some truth to this idea, but not like you'd think (surprise! it's a complicated subject). Congress has indeed encouraged banks to loan money to poor folks, starting with the "Community Reinvestment Act":http://www.federalreserve.gov/dcca/cra/ (CRA), enacted in 1977, under Carter's administration. The original idea was to ban a practice known as "redlining", which essentially was a practice of defining financial no-fly-zones. Anyone who wanted a loan who happened to live within the redlined area wouldn't be considered by any bank. This resulted in de-facto slums, among other things. Removing the problem is a touchy one, and you'd think would require very specific rules. But rather than be very specific, the law as enacted used very vague language (apparently to pacify critics of the bill who complained that it would be a train-wreck if it required banks to lend to people who couldn't afford it). According to "Wikipedia":http://en.wikipedia.org/wiki/Community_Reinvestment_Act: bq. Congress included little prescriptive detail and simply directs the banking regulatory agencies to ensure that banks and savings associations serve the credit needs of their local communities in a safe and sound manner. With the Financial Institutions Reform Recovery and Enforcement Act, in 1984, the credit ratings that banks used to evaluate people's creditworthiness was made public (using a four-tier system) and was required to contain supporting details, so that their decisions could be audited if necessary. The "Federal Housing Enterprises Financial Safety and Soundness Act of 1992":http://assembler.law.cornell.edu/usc-cgi/get_external.cgi?type=pubL&target=102-550 supposedly created the Housing and Urban Development (HUD) organization, and according to "Fannie Mae's website":http://www.fanniemae.com/aboutfm/charter.jhtml: bq. The Federal Housing Enterprises Financial Safety and Soundness Act ("FHEFSSA") of 1992 modernized the regulatory oversight of Fannie Mae and Freddie Mac. It created the Office of Federal Housing Enterprise Oversight ("OFHEO") as a new regulatory office within HUD with the responsibility to "ensure that Fannie Mae and Freddie Mac are adequately capitalized and operating safely." OFHEO is funded by assessments on Fannie Mae and Freddie Mac and is authorized to act without HUD oversight on a range of regulatory issues enumerated in the statute. FHEFSSA established risk-based and minimum capital standards for Fannie Mae and Freddie Mac. *And, it established HUD-imposed housing goals for financing of affordable housing and housing in central cities and other rural and underserved areas.* (emphasis mine) I couldn't find all the specifics of those housing goals in a quick look through the text of the Act, but it's long and complicated (maybe someone else will spend the time). Anyway, in an effort to finance these things effectively, Fannie and Freddie created and sold mortgage-backed securities. THIS is where the door was built (aka: the HUD), but the door isn't yet "open." More on this in a moment. Because of the CRA requirements of transparency (imposed in 1984), banks were having trouble selling themselves, especially to out-of-state buyers. Not that the sales wouldn't happen, but the sales would be slowed by CRA-based protests (I'm not sure what exactly they were protesting). So, Congress passed, in 1994, the Riegle-Neal Interstate Banking and Branching Efficiency Act, which removed the requirements that the banks had to publish the reasons why they made lending decisions under many circumstances, in order to facilitate sale of the banks. The publishing requirements were still in place if you weren't selling your bank, though. In 1995, Bill Clinton's HUD made the policy decision to allow Fannie and Freddie to purchase subprime mortgage securities. Supposedly, HUD expected that Freddie and Fannie would impose their usual high standards upon subprime lenders. They did... mostly (hey, it's still a subprime loan, after all), for a while, anyway. But this was still a bad idea (more on this in a moment). In 1999, Congress passed the "Gramm-Leach-Bliley Act":http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=106_cong_public_laws&docid=f:publ102.106, which repealed the Depression-era Glass-Steagall Act's prohibition against banks acting as any combination of investment bank, commercial bank, and/or insurance company. Having more than one of those under one roof was a bad idea in 1929 because it creates conflict-of-interest on the part of the bankers/brokers, but apparently it's a good idea in 1999. This paved the way for the formation of Citigroup and other investment/commercial/insurance banks. The GLB Act also reduced the number of banks to which the CRA applied, and altered their requirements so that they'd have to report on their CRA compliance less often. In 2003, the "Federal Reserve Bank of New York":http://www.newyorkfed.org/research/epr/03v09n2/0306apga.pdf said: bq. Today, less than 30 percent of all home purchase loans are subject to intensive review under the CRA. In some metropolitan areas, this share is less than 10 percent. This was considered bad news because it meant that the remaining 70 percent of loans were virtually un-auditable, and the reasons that loan decisions were made were untraceable. And Congressman Ron Paul made a speech predicting that these policies of increased lending to poor people would result in a credit collapse and a bailout. Spooky, eh? What about predatory lending practices? In 2002, according to "Wikipedia":http://en.wikipedia.org/wiki/Community_Reinvestment_Act: bq. Kathleen C. Engel and Patricia A. McCoy published a study of the predatory lending implications of the CRA, noting that by the late 1990s, predatory high cost mortgages to “gullible borrowers” were leading to foreclosures against low-income people of color and the elderly. As a result, the FDIC decided to strongly discourage the practice of "predatory lending". And any bank found guilty of predatory lending was considered a higher risk, resulting in a lower CRA performance rating for that bank, both of which made FDIC insurance significantly more expensive. Interestingly, this move seems to have been quite effective. A study sponsored by a New York law firm (Traiger & Hinckley LLP, whoever they are), in January 2008 "found":http://www.traigerlaw.com/publications/traiger_hinckley_llp_cra_foreclosure_study_1-7-08.pdf: bq. Our study suggests that without the CRA, the subprime crisis and related spike in foreclosures might have negatively impacted even more borrowers and neighborhoods. Compared to other lenders in their assessment areas, CRA Banks were less likely to make a high cost loan, charged less for the high cost loans that were made, and were substantially more likely to eschew the secondary market and hold high cost and other loans in portfolio. Moreover, branch availability is a key element of CRA compliance, and foreclosure rates were lower in metropolitan areas with proportionately greater numbers of bank branches. The FDIC's measures to prevent irresponsible and predatory lending worked, and worked really well. So, if CRA wasn't the problem, what was? What brought down Fannie Mae? Remember the HUD? They expected that Fannie and Freddie would buy only the best quality subprime loans. But that backfired: what that ended up doing was making the market for these loans, including the even-crappier ones, bigger. According to Patricia McCoy, teacher of securities law at the University of Connecticut, cited in a "Washington Post":http://www.washingtonpost.com/wp-dyn/content/article/2008/06/09/AR2008060902626_2.html article, _"That just pumped more capital into a very unregulated market that has turned out to be a disaster."_ The HUD's policies and encouragement are probably largely to blame for that market being particularly popular. It isn't clear where the HUD's policies started in 1992, but by 2000 the HUD required Fannie and Freddie to use 50% of their budget for "affordable" housing (sub-prime housing is one category that is considered "affordable"). Bush's HUD upped that requirement to 56% in 2004. Clinton's HUD opened the door by allowing subprime investing. But for some reason, it didn't really take hold as a preferred strategy for investing in "affordable" housing. Consider this: between 2005 and 2008, Fannie made loans to risky (sub-prime) borrowers of more than $270 billion dollars. That staggering sum is more than *three times* as much as all earlier years *combined*. But if it was _all_ the HUD's fault, then between 2005 and 2008, Fannie should have only increased subprime investment by about 6%, not 300%. And this would seem to have only a tangential relationship with the 30-year-old CRA. Something happened in 2004 or 2005! (One allegation I've heard is that Countrywide Mortgage started strong-arming Fannie into buying more of their riskiest loans at about that point.) According to a "New York Times article":http://www.nytimes.com/2008/10/05/business/05fannie.html?_r=1&oref=slogin, Fannie Mae's CEO, Daniel H. Mudd, under pressure from competing banks stealing his market and from Congress members demanding that they loan more to low-income folks, was between a rock and a hard place: bq.. So Mr. Mudd made a fateful choice. Disregarding warnings from his managers that lenders were making too many loans that would never be repaid, he steered Fannie into more treacherous corners of the mortgage market, according to executives. For a time, that decision proved profitable. In the end, it nearly destroyed the company and threatened to drag down the housing market and the economy. p. Nevertheless, folks like the venerable "Wall Street Journal":http://online.wsj.com/article/SB122204078161261183.html?mod=special_page_campaign2008_mostpop blame the CRA for encouraging loose lending standards: bq. This 1977 law compels banks to make loans to poor borrowers who often cannot repay them. And yet somehow, the CRA's actual effect appears to have been the exact opposite: banks that were covered by the CRA were LESS likely to make sub-prime loans than banks that had were unaffected by the CRA. Those independent mortgage companies (unaffected by the CRA) made "high-priced loans" at more than twice the rate of banks governed by the CRA, "according to Janet L. Yellen":http://www.frbsf.org/news/speeches/2008/0331.html, President of the Federal Reserve Bank of San Francisco. According to the "congressional testimony of Michael Barr":http://www.house.gov/apps/list/hearing/financialsvcs_dem/barr021308.pdf (law professor at University of Michigan), 50% of the subprime loans were made by non-CRA mortgage companies, and another 25-30% came from mortgage companies only partially regulated by the CRA---fully CRA-regulated institutions are responsible for "perhaps" one-in-four sub-prime loans. To quote the professor: _"the worst and most widespread abuses occurred in the institutions with the least federal oversight."_ And even more definitively, the "Bank for International Settlements":http://en.wikipedia.org/wiki/Bank_for_International_Settlements (an international organization of central banks) "concluded":http://www.bis.org/publ/work259.pdf?noframes=1: bq. Contrary to some media commentary, there is no evidence that the Community Reinvestment Act was responsible for encouraging the subprime lending boom and subsequent housing bust. This Act only applies to depositories, and did not cover most of the important subprime lenders. Depositories showed a lesser tendency to write subprime loans than lenders not subject to the Act (Yellen 2008). Weird... you mean banks behave more responsibly when someone is peering over their shoulder? Say it ain't so! It seems to me that a large part of the Fannie problem was the HUD, enabling large scale sub-prime investing, and encouraging Fannie and Freddie to invest heavily in "affordable" housing. But that's not the entirety of the problem; the HUD opened the door, but cannot have been responsible for the huge increase in sub-prime investment starting in 2005. However, even minimal investment in sub-prime mortgages (that HUD was trying to create) created demand in the sub-prime mortgage-backed securities market, which drove up the cost and encouraged other folks to invest as well. I think the real lesson to be learned here is that there is no simple answer to questions like "what caused this economic mess?", no matter how much politicians would love to say "it's the other guy and his party's fault".

The Bailout

Tue, 30 Sep 2008 00:45:26 -0600

!(scaled)http://www.memoryhole.net/kyle/archives/images/debt_bailout.jpg!

That's MY $700 Billion

Mon, 22 Sep 2008 13:52:57 -0600

I just want to share a little of what I've been reading, because I find it interesting. For example, consider "Bloomberg's take":http://www.bloomberg.com/apps/news?pid=20601070&sid=alfUj1r0Z10o&refer=politics on the candidates' reactions to the latest financial news (namely, the $700 billion proposed bailout of the entire financial sector). Here's a representative snippet: bq.. [Bill] Donaldson, who was tapped by Bush to head the SEC, says Obama called him last year about the financial-regulatory problems. He has never heard from McCain. "Obama has been talking about the need for better financial regulation well before this crisis hit and has done some real thinking about it," says Donaldson, a lifelong Republican. "McCain comes across as someone who suddenly realized changes have to be made." p. Additionally, they compare the people that Obama and McCain turned to for advice. Both of them turned to three people for advice. Obama called three former Treasury Secretaries. Two are described as particularly effective (Rubin and Summers), and the third became a well-regarded chairman of the Federal Reserve (Paul Volcker -- described as "towering"). McCain, on the other hand, called a Reagan administration adviser (Martin Feldstein) who was regularly cut out of the decision making process by Reagan's chief-of-staff, a former undersecretary of the Treasury not known for doing much (John Taylor), and the former CEO of HP who was fired rather publicly and received $21 million from the golden parachute clause in her contract (Carly Fiorina). Bloomberg clearly prefers Obama's choices. And just when you thought fiscal conservatives would be mad enough about the government buying AIG ($75 billion) and bailing out Fannie Mae and Freddie Mac ($25 billion) and bailing out Bear Stearns so that JPMorgan would buy it ($29 billion), now comes the Big Bailout. Let me reformat that: the *BIG BAILOUT*. An order of magnitude bigger. The taxpayer is now on the hook for an additional _seven hundred billion dollars_. Let's put that in perspective: according to "President Bush's budget":http://www.ssa.gov/budget/2009bud.pdf, we're expected to spend $659 billion on the oh-so-odious Social Security in 2008. The US Government total 2008 budget tips the scale at 2.931 _trillion_ dollars, according to the "Heritage Foundation":http://www.heritage.org/Research/Taxes/wm1829.cfm, which means this bailout would be **TWENTY THREE PERCENT OF OUR ANNUAL BUDGET!!!** Imagine if a guy that makes $100k/yr (which amounts to $77k/yr after taxes, i.e. $6,416/month) suddenly had to spend $17,710. That's what we're talking about. Are you screaming yet? Jared Bernstein has an "article in the Huffington Post":http://www.huffingtonpost.com/jared-bernstein/watching-history-unfold_b_128116.html. Here we go: bq. You hear that implosion reverberating through financial markets? It's the sound of decades of conservative ideology collapsing. ... The week that just ended revealed the myth of market fundamentalism: the notion associated with mainstream, Milton Freidman'esque economics, and amplified by anti-government conservatives that unfettered markets will provide society with the best outcomes. Such simplicity, such elegance...such nonsense. He has a list of what he calls "glaringly obvious truths" that I think is very worth reading. Since you probably won't go read that article, here's a few key ones (and my reactions): # Deregulated markets cannot police themselves; they tend toward speculation, vastly underpriced risk, and deeply damaging bubbles; _(sounds accurate to me, given what we've seen; or perhaps it's more accurate to say that deregulated markets DO NOT police themselves, even if they *can*)_ # An economy that generates growth while leaving most families behind is a broken economy _(ok)_ # We can neither achieve broad prosperity nor compete globally without robust growth in key sectors which we have ignored or underfunded, including manufacturing, green production, and cradle-to-retirement public education; crafting evermore clever financial instruments will not pave the way to dependable, broadly shared growth _(translation: creative accounting is no substitute for good fundamentals)_ # No private sector firm should be too big to fail; any firm of that magnitude must be nationalized _(I would say, rather than nationalized, "broken up", but the question becomes: at what point do we know that a company is too big? And what kind of a disincentive for smart business practices does this become? It's fun to say that too big is bad, but is the cure worse than the disease?)_ # Capital markets are dysfunctional; borrowing and lending standards are ignored; lax capital requirements lead to constant over-leveraging; shadow accounts thwart transparency _(that's not a truism, that's describing the current situation)_ # We apparently can quickly find (or borrow) the money to do the stuff the authorities deem necessary, be it war or bailout; thus, we can also find the money we need for investment in people, from health care to education to infrastructure, etc. _(typical complaint... and saying "we don't have the money" is obviously a poor excuse. These discussions should go back to "should we do this")_ There's more, but those are the most central to his point. Is he saying that capitalism can't work? Maybe. I disagree with such an extremist statement. But consider this: the current Treasury Secretary Paulson (whatever you think of him) said about this crisis: "**Raw capitalism is dead.**":http://www.time.com/time/printout/0,8816,1842123,00.html Another fellow, named Robert Borosage, "puts it slightly differently in his article":http://www.huffingtonpost.com/robert-l-borosage/financial-crisis-time-for_b_128114.html: bq.. Call it extortion. Every American is now being told to ante up $2000 -- an estimated $700 billion in all -- to bail out the banks from their bad bets, or they'll bring down the entire economy. In the speculative frenzy that allowed the Masters of the Universe to pocket millions personally, the banks filled their coffers with toxic paper that no one wants to buy. Now they sensibly don't want to lend money to each other, since no one knows if the other is solvent. So they go on strike, and threaten to trigger a global depression, if they don't get rescued. p. He then lists seven of his own suggestions, which are also worth reading. Granted, I don't see how his fourth suggestion (curb excessive CEO pay) would do much more than make us all feel better, but the heart of the matter is: these guys ran their companies, along with anyone who depended on them, into the ground. They shouldn't be allowed to profit. The government's bailout money should, if used at all, be only used to keep the _economy_ afloat. CEO golden parachutes should be considered just as forfeit as if the company went bankrupt. Because that's what these companies are: *BANKRUPT*. Does nobody else in the world get chills down their spine from just uttering that word? I still want to know just what would happen if these companies were simply forced to sleep in the beds they've made for themselves. If AIG did such a bad job, what would happen if they simply went belly-up? Wouldn't other folks in the market be forced to realize that they really DO need to perform due diligence? That there's a REASON things are called risky? So people get fearful, so credit starts to be harder to get, so people stop investing in things they don't understand... isn't that a _good_ thing? The fundamental problem here, that started this whole mess, is that they were being too free with the credit and loaned money to anyone with a pulse and never bothered checking whether it was a good idea to loan them money! Credit _should_ be harder to get! People who can't afford a house _shouldn't be buying one_! And if you don't know how risky an investment is or if you don't understand where the return comes from _DON'T BUY IT_! There's a shocking idea for you... These people deserve to lose lots of money for making stupid decisions, and anyone who loaned those folks money (aka investors) won't be getting it back. But we don't want some sort of cascade to happen where nobody invests in anybody ever again. How can we make people learn their lessons without taking the stock market down with them? There are a lot of ordinary folks in the stock market these days, including ones that may not even realize it, due to their retirement accounts: the day of the pension has passed, and 401k's rule the land. When the market goes down, they take the little guys with them. Buying all of these bad investments, such that they now become OUR bad investments, seems like a bad idea (no matter whether it's the government doing it or our mutual fund managers). What do we-the-people want with bad investments? We don't! And yet, this bailout seems more like, as Bernstein puts it: privatizing the profits and socializing the losses. Oh goody, sign me up for that! You notice all the pro-deregulation folks are lining up behind this bailout, don't you? It's called voting yourself money. It's also called "saving your own hide"... at everyone else's expense.

More Compiler Complaints: Sparc Edition

Mon, 18 Aug 2008 18:01:12 -0600

Unlike my "previous whining about compilers":http://www.memoryhole.net/kyle/2008/01/apples_compiler_idiocy.html, this one I have no explanation for. It's not me specifying things incorrectly, it's just the compiler being broken. So, here's the goal: atomically increment a variable. On a Sparc (specifically, SparcV9), the function looks something like this: bc. static inline int atomic_inc(int * operand) { register uint32_t oldval, newval; newval = *operand; do { oldval = newval; newval++; __asm__ __volatile__ ("cas [%1], %2, %0" : "=&r" (newval) : "r" (operand), "r"(oldval) : "cc", "memory"); } while (oldval != newval); return oldval+1; } Seems trivial, right? We use the CAS instruction (compare and swap). Conveniently, whenever the comparison fails, it stores the value of @*operand@ in the second register (i.e. %0 aka newval), so there are no extraneous memory operations in this little loop. Right? Right. Does it work? _NO._ Let's take a look at the assembly that the compiler (gcc) generates with -O2 optimization: bc. save %sp, -0x60, %sp ld [%i0], %i5 /* newval = *operand; */ mov %i0, %o1 /* operand is copied into %o1 */ mov %i5, %o2 /* oldval = newval; */ cas [%o1], %o2, %o0 /* o1 = operand, o2 = newval, o0 = ? */ ret restore %i5, 0x1, %o0 Say what? Does that have ANYTHING to do with what I told it? Nope! %o0 is never even initialized, but somehow it gets used anyway! What about the increment? Nope! It was optimized out, apparently (which, in fairness, is probably because we didn't explicitly list it as an input). Of course, gcc is awful, you say! Use SUN's compiler! Sorry, it produces the *exact same* output. But let's be a bit more explicit about the fact that the @newval@ register is an input to the assembly block: bc. static inline int atomic_inc(int * operand) { register uint32_t oldval, newval; newval = *operand; do { oldval = newval; newval++; __asm__ __volatile__ ("cas [%1], %2, %0" : "=&r" (newval) : "r" (operand), "r"(oldval), "0"(newval) : "cc", "memory"); } while (oldval != newval); return oldval+1; } Now, Sun's compiler complains: @warning: parameter in inline asm statement unused: %3@. Well gosh, isn't that useful; way to recognize the fact that "0" declares the input to be an output! But at least, gcc leaves the @add@ operation in: bc. save %sp, -0x60, %sp ld [%i0], %i5 /* oldval = *operand; */ mov %i0, %o1 /* operand is copied to %o1 */ add %i5, 0x1, %o0 /* newval = oldval + 1; */ mov %i5, %o2 /* oldval is copied to %o2 */ cas [%o1], %o2, %o0 ret restore %i5, 0x1, %o0 Yay! The increment made it in there, and %o0 is now initialized to something! But what happened to the @do{ }while()@ loop? Sorry, that was optimized away, because gcc doesn't recognize that @newval@ can change values, despite the fact that _it's listed as an output!_ Sun's compiler will at least leave the while loop in, but will often use the WRONG REGISTER for comparison (such as %i2 instead of %o0). But check out this minor change: bc. static inline int atomic_inc(int * operand) { register uint32_t oldval, newval; do { newval = *operand; oldval = newval; newval++; __asm__ __volatile__ ("cas [%1], %2, %0" : "=&r" (newval) : "r" (operand), "r"(oldval), "0"(newval) : "cc", "memory"); } while (oldval != newval); return oldval+1; } See the difference? Rather than using the output of the @cas@ instruction (@newval@), we're throwing it away and re-reading @*operand@ no matter what. And guess what suddenly happens: bc. save %sp, -0x60, %sp ld [%i0], %i5 /* oldval = *operand; */ add %i5, 0x1, %o0 /* newval = oldval + 1; */ mov %i0, %o1 /* operand is copied to %o1 */ mov %i5, %o2 /* oldval is copied to %o2 */ cas [%o1], %o2, %o0 cmp %i5, %o0 /* if (oldval != newval) */ bne,a,pt %icc, atomic_inc+0x8 /* then go back and try again */ ld [%i0], %i5 ret restore %i5, 0x1, %o0 AHA! The while loop returns! And best of all, both GCC and Sun's compiler suddenly, magically, (and best of all, _consistently_) use the correct registers for the loop comparison! It's amazing! For some reason this change reminds the compilers that @newval@ is an output! It's completely idiotic. So, we can get it to work... but we have to be inefficient in order to do it, because otherwise (inexplicably) the compiler refuses to acknowledge that our output register can change. In case you're curious, the gcc version is: @sparc-sun-solaris2.10-gcc (GCC) 4.0.4 (gccfss)@ and the Sun compiler is: @cc: Sun C 5.9 SunOS_sparc 2007/05/03@