10% of everything is not crap

Posted on August 8th, 2010

I finished the book. A few of Bach’s points stand out as especially significant to my own life. But first, I want to talk about his story about his fellow testers at Apple.

At first I thought I would learn a lot from the other testers. There were more than 400 of them in my building. But talking to them revealed a startling truth: Nobody cared.
Almost nobody. In the first six months I worked at Apple, out of all the testers in the software testing division, I met maybe 10 who were also reading testing books. The rest muddled through without much ambition to master their craft. It was clear that catching the college kids would not be difficult, after all.

The pattern I experienced at Apple would be confirmed almost everywhere I traveled in the computer industry: Most people have put themselves on intellectual autopilot. Most don’t study on their own initiative, but only when they are forced to do so. Even when they study, they choose to study the obvious and conventional subjects. This has the effect of making them more alike instead of more unique. It’s an educational herd mentality.

This is almost right.

I have been thinking a lot, recently, about Sturgeon’s Law. Theodore Sturgeon is a science fiction writer who was once on a panel with with other writers from other genres. One of his fellow panelists threw out the observation that

90% of science fiction writing is crud

To which Sturgeon replied

90% of everything is crud

It’s usually quoted as crap rather than crud and, since it’s better to be useful than correct, I’ll go with that formulation.

Most people understand Sturgeon’s Law as a pessimistic observation of the rottenness that surrounds us: 90% of teachers are crap; 90% of software professionals are crap; 90% of restaurants are crap; 90% of beers are crap; 90% of tv shows are crap. But I prefer to think of Sturgeon’s Law as a strategy for avoiding hasty judgment in an unfamiliar domain.

If you are at the top of your game in software testing (or science fiction or beer drinking or whatever), you probably surround yourself with other people who think like you and have similar interests to you. When you compare your own circle (beer drinkers in Portland; historical fiction writers) with an unfamiliar circle (beer drinkers in Denver; science fiction writers), you are comparing the best of your circle with the average of another circle. That’s not a fair comparison because, if 90% of everything is crap, the average is crap too.

Sturgeon’s Law is about the 10% that is not crap. You have to go find the best before you decide that college graduates are all automatons or that beer drinkers in Denver drink piss or that video games are mindless (compared to movies) or whatever.

Some consequences:

If you are a liberal and all your liberal friends are smart, you need to go look for some smart conservatives before you pass judgment on conservatives as a whole.

If you are a responsible software tester, go look for some smart software developers before you decide that developers are irresponsible.

I could go on.

I have a hunch that this observation explains a whole bunch of phenomena: kids these days aren’t as smart as they were in my day; Women can’t change a plug; recent immigrants are stupid and lazy; and, of course, 90% of science fiction is crap.

None of this conflicts with Bach’s observation about his fellow testers at Apple or his advice that, with just little effort, you can be better than 90% of your co-workers. But it should make you pause before you decide that your group is better, in some way, than some other group.

One other observation and then I am done with buccaneering for a while.

Bach describes a strategy for learning that is very similar to my own. He talks about building a schema for a new topic before he goes about learning the details. I do that too.

When I am learning a new subject, I want to have a theory for what it’s about as a whole before I start learning the particulars. It’s a bit more iterative than that, of course: particulars help me understand the whole and the whole helps me understand the particulars; but my initial goal is to develop a theory for how everything hangs together rather than learn any particular detail.

I sometimes wonder if the people who study for exams miss this.

Having never studied for an exam (except my Latin O Level – I didn’t have a good theory of Latin), I don’t quite know how studying works. But I suspect that the studiers are trying to fill their heads with facts rather than build a skeleton understanding of the subject. It’s inevitable that they’ll forget everything almost immediately because the soft tissue of facts has no bones to cling to. If you have understanding, you can’t help but learn the facts as an accidental bi-product.

I have been trying to teach this to my son but, since he doesn’t study for exams either, he probably knows it already. I hope so. I expect he’ll turn out to be a buccaneer scholar too, even if he doesn’t know it yet.

Buccaneer Scholars Unite!

Posted on August 7th, 2010

I just started reading James Bach’s Secrets of a Buccaneer Scholar. Buccaneer scholar is Bach’s term for someone who takes responsibility for their own education rather than having it handed to them by the authorities.

The book is an odd mix of autobiography and How To guide. The autobiographical bits have remarkable parallels with my own life right down to our reasons for learning harmonica and the kids we saved from certain death (I came across mine floating face down at midnight in the pool at Corton’s Holiday Camp with not another soul around).

A sampling of coincidences …

We both learned to program in BASIC from a book before we even had a computer to type them into. I used to write programs during French classes in a book under my desk and then type them in when I got home. I typed mine into a Zx81; James into an Apple II. I graduated to Z80; James to 68000.

James left home and school at 15. I waited until I was 16. We left for about the same reason – school was boring and we felt we weren’t learning anything. It took me several years though before I bluffed my way into my first programming job. I would’ve done it much earlier except I didn’t know it was an option.

Unlike James, I loved taking exams as a kid. It was a chance for me to excel at school without actually doing any work. In England, at that time, the only thing that counted towards your final grade was the exam at the end of the year, so I was pretty much able to do zero work for the rest of the year and still come top of my class. Sadly for them, American kids don’t have that option.

I should clarify what I mean by zero work. Like James, I was incredibly driven to learn. Apart from teaching myself to write software, I read lot of books – just not the ones my teachers wanted me to read. My dad got me a college textbook on organic chemistry for my 14th birthday. I read that several times.

Also like James, I excelled at antagonizing my teachers and was constantly in trouble at school. I also had an episode of failing exams on purpose.

The Navy had a very strict policy on throwing people out if they weren’t able to keep up academically. We had an exam every week or two for the four years of my apprenticeship. If you failed one, you were put on a Commander’s Warning; two got you a Captain’s Warning and so on as you worked your way up the hierarchy of shame. Each warning came with ever increasing ceremony (picture a military court and you’ll have the setting about right) and ever more impressive certificates of failure.

I got very good at getting exactly 49% (50% was a pass) but, on a surprising number of occasions, when I got my paper back, it had been altered to give me a couple of extra points and a passing grade.

When I received the final warning signed by the Commander in Chief himself, my Divisional Officer scribbled on a note “this beautiful certificate is even more impressive than the one you’ll get when you graduate”.

One more failure and I was out. But I blew it. I was so disenchanted with how low the academic standards were in the navy that I wanted to know if I could still pass a proper exam. A friend of mine was taking A-Level Maths and I went and asked if I could take it too.

The education officer explained how it was a two year course and no one had passed it in ten years and failures reflected badly on him and it was a waste of his time and blah blah. Somehow, I conned him into letting me take the exam without taking the classes.

A couple of days after I got my CinC Warning, I was pulled out of class and told to go see the Captain. I was not told why, but I assumed that I had failed my fifth and final exam and that the end of my career in the navy was imminent. Imagine my surprise when I learned that the Captain had called me out of class to give me my A-Level result personally. I had got an A.

It took them a couple of days to figure out that I was the same dude who had been failing all those exams. When they did, I was told in very plain terms that I would not fail any more exams or there would be serious consequences. In a couple days, I had hatched my new scheme: I would become an officer and exercise an officer’s option to resign…but that’s a story for another day.

Back to the book.

I am about three quarters through it already. I’m enjoying it immensely but it’s hard for me to recommend it.

If you are the kind of person to quit school at 16, you probably did that already. And you probably don’t need James’s lessons on how to learn.

If you are not that kind of person, you probably think of people like us as reckless fools. You are probably better off taking the establishment path to an education anyway.

Star-Strolling

Posted on July 15th, 2010

After working much too late last night, I sat out on my newly-laid patio in my new Adirondack made from freshly-chopped-down, endangered, rain-forest hardwood wrapped in a scarcely-needed blanket with my daughter on my lap sipping rum and milk respectively.

We gazed up at the heavens – something we do all too rarely – looking for planets. I confidently pointed out Mars and my daughter asked me what that other fuzzy clump was.

Trusty iPad to the rescue!

If you point an iPad (or iPhone) filled with Star Walk at a star, it tells you its name. Turns out that the fuzzy clump was M5 and that Mars was actually Arcturus. I had been lying about Mars for years!

It also draws in all the constellations as you wave your iPad across the sky. It’s like Orion has OnMouseOver.

Star Walk – it’s what iPad was invented for. That sound you hear is Matt, clicking on the Apple Store right now.

Solar Flexus

Posted on January 1st, 2010

I’ve wanted to write a physics engine for years and messing with Squeak made me want to try it in Flex. It wasn’t quite as easy as Squeak but it wasn’t too hard.

(It probably needs flash 10 to work)

So far I have gravity and collisions for circular objects. Up next: drag.

Here’s the main loop:

    public function tick() :void {
      for each(var body :Body in bodies) {
        var force :P oint = calculateForceOn(body);
        body.apply(force);
        body.move(1);

        checkForCollision(body);
      }
    }

Inverse Square Law to calculate gravity:

    public function calculateForceOn(body :Body) :P oint {
      var force :P oint= new Point(0, 0);

      for each(var other :Body in bodies) {
        if(body != other) {
          var distance :Number = Point.distance(body.position,other.position);

          var magnitude :Number = (body.mass+other.mass) /(distance*distance);

          var direction :P oint = other.position.subtract(body.position);

          var additionalForce :P oint = new Point(direction.x*magnitude/distance, direction.y*magnitude/distance);

          force = force.add(additionalForce);
        }
      }

      return force;
    }

Look for collisions and calculate the impulsive forces:

    public function checkForCollision(body :Body) :void {
      for each(var other :Body in bodies) {
        if(body != other && body.intersects(other)) {
          var normal :P oint = body.findCollisionNormalTo(other);

          var relativeVelocity :P oint= body.findVelocityRelativeTo(other);

          var relativeNormalVelocity :Number = dotProduct(relativeVelocity, normal);

          if(relativeNormalVelocity < 0) {
            var impulse :Number = -dotProduct(normal,relativeVelocity) *(coefficientOfRestitution+1) /(1/body.mass+1/other.mass); 

            body.applyImpulse(impulse, normal);
            other.applyImpulse(-impulse, normal);
          }
        }
      }
    }

And some heavenly bodies:

      var sun :Body = new Body("Sun", World.Origin);
      sun.radius = 60;
      sun.mass = 50000;
      sun.color = 0x26393D;

      var earth :Body = new Body("Earth", new Point(0,500));
      earth.radius = 40;
      earth.mass = 4;
      earth.velocity = new Point(5,0);
      earth.color = 0xE8E595;

      world.add(sun);
      world.add(earth);
      world.add(moon);
      world.add(mars);

I am still not sure whether I like Flex. The libraries are fantastic but the language – ActionScript – is super-annoying. It makes me wish for C#. It’s allegedly a dynamic language but the compiler makes you declare every type anyway in that wacky syntax that I can never quite remember. Simulating solar systems is fun though.

My Drawing Table Squeaks

Posted on December 30th, 2009

Took the kids to the Exploratorium today. It’s currently my favourite museum. Better even than OMSI (although they don’t serve beer at The Exploratorium. How come that hasn’t caught on outside Portland?)  I wish San Jose had a decent museum. The Tech sucks worse than possibly any museum in the world except Morwelham Quay.

I couldn’t find my favourite exhibit – Ladle Rat Rotten Hut. There are so many great exhibits that I have never actually seen them all.

Wan moaning, Rat Rotten Hut’s murder colder inset, “Ladle Rat Rotten Hut, heresy ladle basking winsome burden barter an shirker cockles. Tick disk ladle basking tutor cordage offer groin-murder hoe lifts honor udder site offer florist. Shaker lake! Dun stopper laundry wrote! An yonder nor sorghum-stenches, dun stopper torque wet strainers!”

Jazz fell in love with the drawing board and watched it for about 90 minutes. It’s basically a table hung from four ropes like a pendulum and a pen that draws patterns on a piece of paper as the table swings and twists.  There is a weight that makes it swing eccentrically to make the patterns more interesting.

I promised to make her a real one but I wanted to see if I could do it in Alan Kay’s excellent Squeak first. It was pretty easy and quite effective.

Here’s the program. I messed around with the constants to get different effects.

squeak program

and here’s a picture I made with it:

drawingtable

Today: simulation. Tomorrow: the real thing.

Wish us luck!

UPDATE:

I just downloaded the latest version of Squeak (now called etoys). It’s MUCH better than it used to be. All the bugs are gone and it doesn’t look like it was made in 1983 any more. Go get it from http://www.squeakland.org/ then you can play with my project – Squeak: Drawing Table

etoys

Maybe you could add damping for me.

Teach Your Kids to Argue

Posted on August 26th, 2009

They teach way too much english and history in school and not nearly enough physics (but that’s the topic of my next blog).

But the one subject that they really need to teach more of is rhetoric. Jay Heinrich believes that every parent should teach their children to argue and I agree.

To disagree reasonably, a child must learn the three basic tools of argument. I got them straight from Aristotle, hence the Greek labels: logos, ethos, and pathos.

Heinrich goes through each of the elements of rhetoric in turn and illustrates it with examples from arguments with his children.

Logos is argument by logic. If arguments were children, logos would be the brainy one, the big sister who gets top grades in high school. Forcing my kids to be logical forced them to connect what they wanted with the reasons they gave.

“Mary won’t let me play with the car.”

“Why should she?”

“Because she’s a pig.”

“So Mary should give you the car because she’s a pig?”

Repeat the kid’s premise (she’s being a pig) with her conclusion (therefore she should let me play with the car), and she has to think logically.

Logos is the one that gets technical people in trouble with their non-technical wives. Ethos is what gets them out of trouble. Sometimes.

Ethos, or argument by character, employs the persuader’s personality, reputation, and ability to look trustworthy. (While logos sweats over its GPA, ethos gets elected class president.) My kids learned early on that a sterling reputation is more than just good; it’s persuasive. In rhetoric, lying isn’t just a foul because it’s wrong, it’s a foul because it’s unpersuasive. A parent is more likely to believe a trustworthy kid and to accept her argument. For example, if both children — the entire list of suspects — deny having eaten the last cookie, ethos becomes important.

Me: “One of you took the cookie.”

Dorothy: “Have I ever stolen cookies before?”

Me: “Good point. George?”

Careful with pathos. Especially if you have a daughter.

Then there’s pathos, argument by emotion. It’s the sibling who gets away with everything by skillfully playing on heartstrings. hen a kid learns to read your emotions and play them like an instrument, you’re raising a good persuader.

Dorothy: “Dad, you look tired. Want to sit down?”

Me: “Thanks. Where did you have in mind?”

Dorothy: “Ben & Jerry’s.”

The article was even better the second time I read it. You should read it too.

Integration is just a better multiplication

Posted on July 16th, 2009

Regular readers know that I am a big fan of Better Explained in which Kalid makes mathematical ideas accessible.

Today’s installment:

Integration is just multiplication when one of the operands is changing.

Most people grok integration as area under a curve but, as Kalid explains, area is just one convenient way of visualizing multiplication…but we don’t need to visualize multiplication as multiplication is already pretty straightforward – in the simplest case, it’s just repeated addition.

Many ideas in maths start out simple like that and then gradually generalize to a more complex idea. In Kalid’s words:

Our understanding of multiplication changed over time:

  • With integers (3 × 4), multiplication is repeated addition
  • With real numbers (3.12 x sqrt(2)), multiplication is scaling
  • With negative numbers (-2.3 * 4.3), multiplication is flipping and scaling
  • With complex numbers (3 * 3i), multiplication is rotating and scaling

We’re evolving towards a general notion of “applying” one number to another, and the properties we apply (repeated counting, scaling, flipping or rotating) can vary. Integration is another step along this path.

In other words,

Integration is just a better multiplication

or, conversely,

Multiplication is a special case of integration when the values are static.

You say Cwawfee, I say…

Posted on March 30th, 2009

One problem with bringing up kids in countries where they speak funny is when you get homework like this:

Underline the words that have a schwa sound and circle the vowel that makes the sound.

A schwa sound? What the bloody hell is a schwa sound?

Luckily they gave us …erm…the kids…some clues:

  • (A)bout
  • Less(o)n

I doubt there are two words in the English words that have less vowel sounds in common except maybe orange and Aardvark. Neither of those words, as best as I can tell, have a schwa sound. An oo sound maybe, if you are Canadian, but no schwa sound.

Luckily, we have an older child – he talks funny too – to help. Eldest rattled off the first few words:

  • Again – Ok, I can see that. If About has schwa then maybe again has one too.
  • We argued about silent. It seemed to have something in common with Lesson but, ultimately, we decided against.
  • Problem – Wait! What?

If I squint my ears I can be persuaded that the em syllable vaguely resembles a schwa but both my oddly-spoken offspring assured me that the schwa-like syllable was the first one.

We went through a whole verbal dance.

Funny-speaking son: prwaaablem

Normal-speaking father: problem. It’s a short o. Like pot.

Funny-speaking son: Not pr’bliiim….prwaaaawblem!

Funny-speaking daughter: prwwwaaaaaawwblem!!

Eventually, I was dismissed and the funny-speaking ones decided among themselves.

Need a parent to derive the quadratic formula for you? I am there! Got a  theorem that needs proving? I can do that! Need help locating the fallacies in America’s founding mythology? That’s my strong point…but locating schwa sounds…?….

You’ll need to find a different parent or a funny-speaking sibling.

What is education for?

Posted on March 23rd, 2009

I just got through reading Charles Murray Real Education so I was perfectly primed to be alarmed by this passage

Teachers can’t prepare for the content of the tests and so they substitute practice exams and countless hours of instruction in comprehension strategies like “finding the main idea.” Yet despite this intensive test preparation, reading scores have paradoxically stagnated or declined in the later grades.

This is because the schools have imagined that reading is merely a “skill” that can be transferred from one passage to another, and that reading scores can be raised by having young students endlessly practice strategies on trivial stories. Tragic amounts of time have been wasted that could have been devoted to enhancing knowledge and vocabulary, which would actually raise reading comprehension scores.

in today’s New York Times.

Murray covers this idea at length and I was already predisposed to accept the book’s four simple truths

  1. Ability varies
  2. Half of the children are below average.
  3. Too many people are going to college.
  4. America’s future depends on how we educate the academically gifted.

and its central thesis – that a liberal arts education is inappropriate for most kids – so it was quite a feat for the author to persuade me that he is an out-of-touch reactionary.

I still agree with the author’s basic premise that the students on the left of the bell curve are ill served by the education system because it optimizes for an outcome that they don’t desire and cannot achieve. The system is also not optimized for students on the right of the bell curve because it has to accommodate those on the left of it. The result is mediocrity.

Those are unfashionable views, but I share them.

More controversially, Murray claims that much of the investment in the education system since the early sixties has been intended to narrow the gap between the high achievers and the low achievers and that all of the programs intended to achieve this aim – from Head Start to No Child Left Behind – have utterly failed. More than that, Murray claims that they could not possibly have succeeded because there are some ideas that some kids are not smart enough to learn. Shocking, but I agree with that too.

I was still with him when he repeated his claim, from The Bell Curve, that society has conspired to turn a college degree as a simple marker for ‘sufficiently intelligent’ for most jobs – a marker that would be reliably visible after a third grade IQ test.

I have long believed that most kids would have a more successful outcome if they were steered towards a vocational curriculum. It would be kinder, sneers one reviewerto teach them to fix cars rather than ask them to read novels. I do believe – with Murray – that some kids would thrive if they were allowed to succeed at subjects that they enjoy rather than fail at subjects that they resent. But it doesn’t have to be fixing cars. I am thinking of my brother who is a successful carpenter, my father who was a successful butcher and my nephew who was a bank manager at age 21. A liberal arts degree would not have made any of them more successful and designing the whole system of education around the idea that most kids should shoot for one helps neither the kids nor the system.

But he lost me in his prescription for the syllabus for a degree. While he concedes that scientists, engineers and architects might also benefit from a rigorous tertiary education, according to Murray the main purpose of a Batchelor’s degree is to provide a well-rounded education for the future leaders of America. This well-rounded education would cover Aristotle, Decartes, Kant, Dante, Milton, Mozart, Beethoven, Michelangelo – not a single scientist among them. Meanwhile, those not smart enough to complete a four year degree can take vocational courses in software design or accounting.

While I agree with Murray that a Batchelor’s degree should be sufficiently difficult to deter all but the brightest of kids, I’d rather students got to specialize early so that student with a talent and a desire for, say, maths can get to the interesting stuff more quickly rather than taking required courses in music or art. We probably also disagree about which knowledge is academically challenging to acquire and which is relevant to the 21st century.

Ever expanding circles

Posted on March 14th, 2009

Whereas the constitution sets clear boundaries on the authority of the Federal government.

Whereas the federal government has no business defining basic mathematical constants.

Resolved, that theories, definitions and celebrations of Π should be left to the various States.