[PREV - SUPERFORECASTING_ISSUES]    [TOP]

QUESTIONING_DEBIASING

                                             November 23, 2015

When Tetlock was scheduled to do a Long
Now talk, I took the opportunity to order
a copy of his book first.  I went through    Typically, if something occurs
it throughly beforehand to try to craft a    to me at all it's very late in
decent question to ask.                      talk when it's difficult to get
                                             the question passed in in time.
I considered a few possibilities,
eventually settling on this question:
                                            
   "You've compared your 'debiasing         I talked about this here:
   training' to no-training, but
   shouldn't you also do a control             SUPERFORECASTING
   comparing it to some other training?
   You might compare your 'ten
   commandments for superforcasters'
   to Neustadt and May's 'Thinking in
   Time', or for that matter, to
   reading the Lord's Prayer?"
                                                                   
This didn't make the cut, in fact it probably looked               
completely off-topic to the first-line reviewers, because it         
turned out that Tetlock neglected to mention anything at all       
about this in his talk. The debiasing training and the 10%                 
improvement result only came up in passing in the Q&A                  
period-- Stewart Brand had never heard of this before and                  
suddenly remarked "That's great!".                                         
                                                                           
   Another question I considered and skipped:                             

      Would encouraging pundits to make more precise,      
      testable predictions actually help with our          
      collective decision-making?  We're living in a       
      world where even the IPCC's conclusions are all      
      but ignored: The magnitude of "identity              
      protective reasoning" would seem to be strong        
      enough to dismiss *any* contrary opinion, no         
      matter how strong the track record or impressive     
      the credentials.                                     
                                                           
      My prediction: if you could get Niall Ferguson to     
      make inflation predictions without hedging, he       
      would have no trouble walking away from them when    
      they turn out to be wrong.                           
                                                           
                                                          
   Another potential question was:                     

      Making a fine-grained probability estimate down
      to a percentage point runs against the grain of
      anyone with scientific training-- if you say
      something is 23% you're supposed to have data to
      back up that it's not 24 or 22%.

      What is the breakdown of your superforecasters
      by academic and professional background?  Are
      there any correlations?




--------
[NEXT - SCATTER_THE_NOISE]