In the early 1960's, Stanley Milgram, a professor at Yale, conducted a group of experiments which basically tested how obedient people would be to authority. I won't go in depth (please read the excellent Wikipedia Article), but the basic conclusion was 65% of test subjects performed an act they believed was lethal to another human being because the guy in the lab coat told them to do so. The authority figure told them everything was fine, and though many were uncomfortable, about two thirds did it anyway.
My concern this season is about programmed obedience, but not to a guy in a lab coat, but rather to a machine.
See, in the first instance, the Lovely Jennifer and I were shopping at Target. The nice young cashier told her the bill was $85.65. She handed him $85, missing the last part. He counted the money twice, and Jennifer realized before he did that she had shorted him, and apologetically handed him another dollar.
The young man must have punched the sum of $86 into the register wrong, because he tried to give her $10 dollars change. Despite the fact he had just counted the money, the computer TOLD HIM TO GIVE HER $10, and by God, he was going to do it. Once the situation was explained, we all know all he had to do is put the ten dollar bill back in the register, and his books would be even; however, he couldn't allow the computer to be wrong. He voided the sale and started over. He was extremely apologetic and polite, but see, the Computer could not be wrong.
Our very next stop brought us to Hastings, where they were having a sale, five used books for $20. Music to my ears. The Lovely Jennifer and I confirmed with the nearest book associate that the sale was valid, and picked out some books. Then, I breezed through their comics, and picked out some stuff I wanted to sample; five issues at about $4.00 a piece. See the nice neat pattern? A $40 ticket, with $20 worth of books and $20 worth of comics.
Once we reached the register, the books began to ring up full price, but the young lady behind the computer assured us the computer would adjust the price. She punched in the coupon code, and proceeded to ask us for $70. I told her she was wrong, and there was a sale. She insisted she had punched in the code. I pointed out the two very even sums of $20 that the merchandise cost, and pointed out as well that the five comics I handed her were not $10 a piece, and did not themselves add up to $50. She insisted I must be wrong, because the computer told her this was a $70 purchase.
Those who know me understand I have a tendency to get irate when people don't listen to well reasoned arguments, particularly when just a moment of common sense thought would show there was a problem. Explaining it to her again, she tells me I shouldn't get mad at her, IF there's a problem, she did everything she was supposed to do.
Once the book manager, and then the STORE manager were involved, they realized that one of the used books was mis-marked, and therefore showing up as a new book, and not triggering the proper coupon code. Once this was dealt with, the total was indeed $40 plus tax. The manager apologized, the sales girl did not; it never occurred to her I was not angry not to get my discount, but because she was listening to the computer over her own ability to count. She was so convinced of the computer's infallibility, she forgot the main rule of customer service: the customer is always right, even when he isn't. In this case, I was right, and she never understood why. She didn't have to, because the computer would tell her what to do.
Think this is isolated? Ask the clerk at Starbucks to count your change back to you the next time you buy a latte. Take an online class and have the program score a correct answer on a test as incorrect when it is correct, then convince the instructor of that. Tell someone the forwarded email they got about Mars looking as big as the Moon, contrary to ALL COMMON SENSE is wrong. You will be met with confusion, ambivalence, and downright scorn, hopefully in that order. We don't need to worry about a government going astray, we need to worry about our machines leading us down a primrose path.
Now, am I talking Cylon-like, Terminator take over? No, because our machines are only as good as their users (the Target kid punched in the wrong data; the Hastings book had been tagged improperly). We are asking devices we believe capable of outthinking us to do so with the same information we give them in the first place. That will end in disaster.
Do your world a favor; learn to count. Think critically the next time something looks wrong. Bother to care.
And ignore the irony of it being a guy on a computer asking you to do these things.
Merry Christmas.
