Sports: The final frontier for nerds to takeover

The word nerd has negative connotations and is applied derisively to describe people who focus on academics then on oh so important social matters like partying, drinking and everything that is wasteful. The Wikipedia search for Nerd  suggests "...the word is derived from "knurd" ("drunk" spelled backwards), which was used to describe people who studied rather than partied.". For better definition of nerd than what is on Wikipedia, read this on NPR. While nerd calling exists widely in the west, it is not so outwardly manifested in the eastern culture where academic success is given due respect. But nerd calling does exist in the east. I know because I have been called one in both the worlds.

During high school and beyond, I was a kid who was picked the last when spitting into two teams for a game of cricket. During my Bachelors days, I had a friend who had an issue with my academic pursuits(he almost always lost). He would suggestively say that more than academic skills, practical skills were necessary to survive in the world. Few years later, I was in the USA and my supervisor at work thought it(I) was weird that I could add two random three digit numbers in few seconds without using a calculator. I was surprised at her behavior because I wasn't even the best among my friends back home when it came to skills related to basic arithmetic without the use of calculator.

With introduction and personal anecdotes out of the way, let me get to the main part. I recently read a book called "Money Ball" by Michael Lewis. I liked the authors other works(books and magazine articles on Economy of US and the world) and bought this book. It was a good read as most of his works are. Coincidentally, the movie starring Brad Pitt by the same name based on the book was released at about the time I was reading the book. Moneyball, published in 2003, is The story of Oakland A's general manager Billy Beane's successful attempt to put together a baseball club on a budget by employing computer-generated analysis to draft his players.

In this book, he describes how the traditional scouts are enamored by the looks of the athletes and judge them by what they felt were good players. The traditional scouts are very bad at picking good players. Billy Beane used computers, spreadsheets and statistics to identify undervalued players, sign them on to play for his team and was very successful in winning baseball games.

The nerds have done well later in life in almost every field, like science, engineering, technology, medicine. It is in entertainment and sports they failed to make inroads for a long time. Other than being the lead men/women, the entertainment industry has slowly but surely been taken over by professional writers, graphics artists, production assistants, technicians etc. Now shows are made with leads portraying nerds. Some sitcoms that come to my mind are '30 Rock'(Tina Fey plays Liz Lemon, a head writer), 'How I met your mother'(the lead Ted Mosby by Josh Radnor is an architect and professor), 'The Big Bang Theory'(my favorite and all 4 men are well stereotypical nerds).

The only field left was sports. Just like in entertainment, the nerds and geeks are making inroads into sports. Nerds have found a way to fit analytics((in baseball they are called sabermetrics) ,what they can do without a blink, to change sports. In baseball, other teams have tried to emulate what Billy Beane did with the Oakland As. Other sports are quickly following suite. NBA is one such sport.  The current champions, Dallas Mavericks, employed analytics that helped them win the championship. They have a Basketball Analytics team headed by Roland Beech and is credited for helping the team win the championship. Here is an ESPN article on Dallas Mavericks employing data geekery to win championship. Analytics have been part of NFL for some time now. Here is short history of it. The same is the case for NHL and a good site for hockey analytics is http://hockeyanalytics.com/.

Then there is the Sloan Sports Analytics Conference that is held every year in Boston by MIT Sloan School of Business. In 2011, into it's fifth year since the start in 2007, the conference has grown to 1500 attendees with 300 in the waiting list. In 2007 about 175 attended the conference. That's a leap of  850% in 5 years and is widely covered by the print and television media now. Guest speakers for 2011 include Micheal Lewis, Malcolm Gladwell, Bill Simmons, Mark Cuban etc. This is just to say how popular this conference has become. The research paper grand prize winner for 2011 was an Indian guy Arup Sen(He is a PhD candidate in Economics at Boston University). In this paper Arup Sen provides evidence of moral hazard in long term guaranteed contracts using data from NBA. Of course this is not the first time that ‘contract-year effect’ was suggested, excepting Arup Sen showed evidence, with statistical analysis build in, that players productivity peaks in the last year of the contract and drops in the first year of their long term contract. Another favorite subject of mine is the 'hot hands' and 'crunch-time' lore. There is lot of debate on hot hand in basketball. Analytics say that it does not exists while Athletes vow that it exists. Whatever side you are on, how do counter this - Illusion of Hot Hand makes athletes take bad shots. This is supported in another research paper that professional basketball players may overgeneralize from their very recent experience to their expected future performance. Similarly it is being shown that the widely held belief that some start are better in crunch time is not so true. Kobe Bryant will surely disagree with that analysis.

There are other aspects of sports, other than the actual players playing the game, that is important at the professional level. Some of these aspects are scouting, picking talent in the draft, trading talent, managing the team, coaching etc. I don't want to cover every aspect of sports here as it is beyond the scope of my knowledge. However, there is a general perception that an ex-athlete is better at picking talent and managing the team. Back in the days, whenever India lost a match or series in the game of cricket(which was most of the time unlike now), the reason discussed was that retired athletes are not given enough role in picking the talent and managing the team. On the face of it, this argument is simple. Who better to gauge how good a player is, than one who played the sport at a high level for a long time? However, there is evidence that it is not to be so. For instance, Michael Jordan who is considered to be the best ever to play basketball, has proven so far highly ineffective at picking talent and at making the team he owns, the Charlotte Bobcats, successful. The scouts are mostly players who did not make it big and most of their picks turn out to be duds. This is true in every sport. Here is a video from the Sloan conference that discusses the Draft and how most of the draft picks are busts. The problem is scouts and retired athletes are all looking at a prospect who has the looks. They are carried away by the physical aspects and the recent good showing of the player than the actual metrics of the players athletic abilities in the sport. Analytics can chafe that information out and with technology better analytics can be produced in sports where it is lacking now.

The other part of sports that geekery is making inroads is from the efficiency standpoint of referees. Previously referees could commit mistakes(still do) and get away(still do) with it. The professional leagues levy fines on the players for expressing dissent and reduce accountability on part of the referees for their mistakes. Analysts now are trying to quantify the efficiency of referees and thereby are bringing changes to how sports are regulated on the field. Referee Analytics is now a part of what is discussed at the Sloan Conference. Here is one article quantifying referee bias in Soccer. The introduction of replay in many sports is a welcome change to minimize errors(blunders in some cases) that have direct relation to the outcome of the game. In cricket, a constant problem with referees(called umpires) is with regards to run outs, LBW and nicks. The on field umpires were making and continue to make bad calls with those cases and the television instant replay has fixed the first problem of calling run-outs. All they have to do, is make a call to the third umpire when in doubt. Instant replay is widely used in almost every sport in US. Virtual Eye ball tracking technology and Hot Spot technology address the latter two issues with umpiring in cricket and are now used in the Umpire Decision Review System introduced in international cricket last year.There was a loud outcry and the Indian Cricket management did not accept it as the technologies were not perfect. I would rather take the side of technology than humans when it comes to these close calls in cricket.

With all this hyperbole on the geek adventures in sports, a note of caution is in order. Analytics can only supplement and not take over human decisions. When taken to extremes it has proven to cause trouble. Let us take an example of Long Term Capital Management, from the world of financial markets, that was too Too Big To Fail in 1998 and was bailed out by the Fed. LTCM was a hedge fund started by John Meriwether. It had on its board Myron Scholes and Robert C. Merton who won Nobel Prize in 1997 for their work on creating a model to discover the price of an option. Their work was built upon an earlier work by Louis Bachelier - He is credited with being the first person to model the stochastic process now called Brownian motion.

Scholes and Merton in collaboration with then deceased Black developed a mathematical model for the financial market called The Black–Scholes model or Black–Scholes-Merton  or simply Black Model. The mathematical model they derived was a partial differential equation, now called the Black–Scholes equation, which gives one the price of the option with time variable. The problem however was with the assumptions in the model. The Brownian movement assumes that the movements are small, random devoid of external influences or biases. Plotting these movements will generate the familiar bell curve also known as normal or Gaussian distribution. However the stock markets with its risks and frequent wild movements does not behave like a gas molecule. In fact other mathematical models have been proposed to represent the financial markets. One such model supported by Nassim Tabeb of the Black Swan fame is Power Law. LTCM could not in it's mathematical models anticipate the rare events and incurred heavy losses from the 1997 Asian Financial crisis and Russian default in 1998 and deadly combo with leverage(at the height of euphoria they were levered 25:1). It had to be bailed out by a consortium of banks and the fed.


There is great debate still being waged on the caused of the recent financial crisis dubbed the great recession. Of the many reasons some blame has been levied on the excessive reliance on flawed mathematical models VaR(derived by the risk management team at JPM)  and the Gaussian Copula Function by David Li. You can read more on how the Var and Copula Function contributed to the crisis here and here respectively. The weird thing is that both these models use the Gaussian Distribution as the core and  are widely popular of their simplicity. They spent much time developing a simple model for a complex system with frequent occurring big risks. Both models underestimated the potential risk that they were supposed to measure.

What does mathematical model that over simplifies a financial market got to do with sports and nerds getting into it. Nothing I suppose, but I wanted to caution the excessive reliance on technology and statistical models on measuring players and the sports in general. There is a definite need for advanced statistics that measures the value of players better than the current popular metrics used. For eg., average ppg used in NBA is skewed towards players that play lot more minutes and take lot more shots than say a defensive expert whose contributions to the game are largely overshadowed with this metric. One simple metric that measures other contributions in NBA is Efficiency. A little more complex number called PER(Player Efficiency Rating) was created by John Hollinger of ESPN. PER takes into contributions almost all aspects of basketball including the minutes and pace of the game played. It is a step in a right direction and is getting accepted as a measure of player's current value. However this number has problems - it rewards inefficient shooting and can produce a distorted picture of some defensive specialists who produce few blocks and steals. Please find here a good analysis of problems with PER. I personally think that both efficiency and PER are both skewed towards offensive abilities of a player and are highly correlated with the ppg of players. Clearly a better model is still wanted in NBA.

Thankfully it does not use Gaussian distribution to predict the future value of the player so team managements can load up on a single player. But wait, some teams have already done that without any mathematical model. Eg., Eddy Curry with the Knicks till 2010, Gilbert Arenas with the Magic and Rashard Lewis with the Wizards are just examples of bad contracts. Thankfully there is the amnesty clause that came of the new CBA. Bad Contracts have one advantage though. It gives the owners a good excuse to come back to the table to discuss CBA and demand bigger piece of the revenue pie as reward for their own mistakes. I digress.

We already discussed baseball and how moneyball highlighted the use of analytics by a certain team in the sport. The book highlights the undervalued usage of the metric On-base Percentage. Please listen to this podcast that throws more light on it as I am not a big follower of baseball to provide good analysis.

Lets jump to cricket. A lot of data is collected in cricket, but most of it is centered around the batsmen or the offensive side of the game. Most Man of the Match (equivalent to MVP in American sports) awards go to batsmen. A bowler rarely gets the MVP award unless a player takes at least 4 wickets and the match is won, this too when there is no other big contribution by a batsmen. The value of a batsmen is usually measured by his batting average and the number of 100's and 50s he has scored. This to me is a very skewed metric. A batsmen that comes in the first three positions has a chance to play lot more  balls and hence a chance to have better batting average and better chance at scoring more 100s and 50s. What we consider great batsmen come onto the crease at the first three positions. This however skews other important metrics like, how many wins did he contribute to, how many partnerships did he generate, how well did he rotate the bat, how many dot balls did he have, how many chances did he give away(how lucky he was on any given day) that the opposition failed to capitalize on. Similarly for a bowler, the metrics that are used are wickets taken and average runs per over given away. This again fails to measure the contribution to the win, the conditions in which he was playing, how many dot balls did he have(the more the better), how many good length balls did he bowl, did the bowler in tandem get any wickets etc. Fielding is another contribution that is given scant respect. No metrics exist on how much a fielder contributes to the win of a team. Other than who caught the ball when a player gets out, nothing about fielding is measured. A good thorough review of some concerns related to cricket stats can be found here.

With all this I have to conclude with the saying from a superhero nerd aka Spider Man comic strip - 'With Great powers come great responsibility'.

While nerds are making inroads into sports, they are making inroads into the fashion business with their looks. Look for athletes like Dwyane Wade, Lebron James, Kevin Durant who sport the nerdy look quite often. Kevin Durant has taken it a step further with a back pack in tango. Grantland has a feature on this new trend in NBA - The rise of the NBA nerd. More power to my peeps.


A few more links of interest can be found below.
Redistribution: Blocking the Revenge of the Nerds?
Who’s a Nerd, Anyway?
The beauty of the geek

Sloan Sports Analytics Conference Recap
Will Kuntz '06, Real Life MoneyBall "Scout"
Shane Battier - The No-Stats All-Star

Prof. Daniel Kahneman - Nobel prize winner for prospect theory
Reinforcement Learning and Investor Behavior
The Gambler's and Hot-Hand Fallacies: Theory and Applications
Recency: Hot-hands and the Gambler's fallacy
The hot hand belief and the gambler’s fallacy in investment decisions under risk
Success Rates of Traders
What Makes a Rogue Trader Tick?
Hot Hand in Sports
Option Traders Use (very) Sophisticated Heuristics, Never the Black–Scholes–Merton Formula
Warren Buffett on LTCM
Eight Days - The battle to save the American financial system

Fat fingering, flash crash and all the fun stuff called investing

I am not a stock trader, but I occasionally burn my fingers attempting to do just that. This post is a look into the rearview of my experience with trading in the stock market. It has been a while since I bought or sold into the market. I incurred severe losses in my attempts to make money in the stock markets during the years 2008 and 2009. I was long in 2008 and short in 2009. Yes, stupid right. Needless to say, I had better success at trading in a flea market. I wouldn't be writing this post if I made money in those years.

Date: May 6th, 2010.
Subject: Fat fingering could save a trade.
By Noon that day, the markets have lost about 5%. I had entered a trade the previous night to execute at $22/share. I don't remember the security nor do I remember the number of shares. So anyway, when I was taking a coffee at around 2:30 PM, a colleague mentioned that the market was tanking pretty bad. I also recieved a phone call from my friend to deliver the good news. I was worried that the trade I had entered was executed and deeply in red from the broad market tanking. One of the reasons for the markets to crash, I heard during that day, was that an unidentified futures trader had fat fingered a sell order for $5 Billion worth of securities while he meant only to palce $5 million worth of them. An extra zero never meant so much to so many people around the world like it did on that day.  I went home and eagerly logged into my trading account to find how much loss I had incurred on my trade. Surprisingly, I did not see the security in my account. I checked the order status and it showed that the order was cancelled. I checked the trading range for the security to see if for some reason the secruity did not hit my buy price. The security did go to as low as 13 bucks. I reviewed my cancelled order to find that I had set my buy limit order at $12.00 and not $22.00 as I had intended. This casual mistake of mine kind of saved my day. However I did feel a tinge of disappointment because, had the security gone down to $12 and my order been executed, I would have made 33% on that day for my fat fingering error. How is that possible? When right after what is now called flash crash the market rebounded, recovering most of the losses between 2:30 PM to 3:00 PM. The security I was interested in buying rebounded to $16.00. While I was happy, I did not suffer a loss of about 30%, I was also unhappy that my mistake did not make me rich by 33%. Well the way my trades were going, the only way I was going to make money was by mistakes. Alas it was not meant to be.

Note: It was later revealed that the cause of the crash was not fat finger, but a sell algorithm that executed the sell order of S&P 500 futures. The report by SEC was be viewed here -http://www.sec.gov/news/studies/2010/marketevents-report.pdf.

So with that preview in mind, I have to go a bit earlier into my life of trading. Why did I get into this mess to begin with? It was that one odd year when for the first time in my life I discovered I may have to payback Uncle Sam on my tax returns. It turns out that I was being withdrawn less taxes in my paycheck and hence would have to pay back those taxes. One of the ways to avoid that was to open a traditional IRA account. So I did just that. This was 2008 and the market was tanking. Now opening an IRA account opened a whole new oppurtunity to prove to myself that I could make more money outside my profession.

I began reading books, a lots of them, by Benjamin Graham, John Bogle, Warren Buffett, even Suzy Orman. Since I began with books by value investors, I took a long only approach and stocks for life approach. Besides the word value in value investing kind of appealed to the way I approach life. So I did some screens for low P/E ratios, high dividends etc and put money in those. It was a time when commodities were bullish. I did not buy any when the market was going up as I felt they were expensive(read - I tricked myself into thinking like a value investor). So when the market began to crash, I began to buy them assuming there was capitulation and that I need to go in when the market was falling. I also did cost averaging as the securities were losing more and more. In 2008, the pit was bottomless. All the theories I had read were becoming useless and fear took over. I realized that times have changed and every recession is different. I also recognized that the market way ripe for shorting.

However I could not short since the account I was trading was an IRA account. I could not buy put options(which was the one way to short a market in an IRA) as my account was not setup for options trading. I could buy inverse etfs(that shorted the market), but since most inverse etfs are shorted on a daily basis, the gains from shorting the market were lost by the inherent defects of the etf. I had applied to change my account to allow option trading and opened a regular trading account with another broker. (Yes, more money and less restriction would fix everything and now I could go on to be a millionnaire shorting the market. The irony is that easy access to more money, hence leverage, and lax regulation were the causes of the current recession.)

I realized the lossess in my IRA account about the end of 2009 and began shorting the market with option puts. I also bought leveraged inverse etfs. With these moves and the economy still tanking, things turned around a bit in my account for some time though the losses were not completely recovered. From losses in thousands to have recovered within hundreds was a major achievement. I was euphoric and once on a phone call to a friend of mine said this "value investing has no place now. In this age investors have to be nimble". Clearly I was overconfident and not following my own statements.

The market turned around and I did not realize the gains in time. I failed to stagger my profits. Instead I was still stuck in the bear mode. I said to myself that the market was irrational, the fundamentals have not changed and hence it was only a matter of time before the market begins to tank again. It wasn't meant to be.

Weeks become months and market takes an ascent I havent seem before. I begin looking into technical analysis as everyone on the web was talking about it. It was also widely said that Hedge funds with their HFTs use largely technical analysis to make their trades. The S&P500 index hung around the 200MA for a few days and it gave me hope that my thesis could be right and the bear market was set to begin. Well, ever a contrarion indicator, the market broke through the 200 moving average and began it ascent with greater vigor. That was it for me; I took my losses, which were now more than when I began to short the market.

I stayed away from trading the market for about 6 months. Infact, I did not get back into it until 2010. Well the rally was slow in 2010 compared to 2009 and 2011 was deja vu all over again. Only the player now is Europe and not US and I am long the market only this time in my 401K accounts. I cant do much there which probably is a good thing.

In all this, I tried almost everyt trick. Well! not everything; I havent developed an algorithm for automatic trading with back testing. Other than that I have done everything and for every recommendation I have proof that they wont work. I attended webinars, earnings reports(live as well as transcripts), did screeners for etfs and mutual funds. Bought etfs, straight and leveraged, long and short. I did market orders, limit orders, stop loss orders, trailing limit orders. I did options both calls and puts. I did not go into covered calls, spreads and other exotic option trading. Nor did I deal with forex trading and futures trading, well not directly(but did so through etfs that use forex and futures trading). In all this the only thing that I learned was that, there is no formula for successful trading. A recommendation by a successful trader/investor is no guarantee for success. Infact it can be guarantee for failure as was in my case.

With so much rumination on things that have failed, I feel obligated to disclose what worked for me.
1. Buy No Load, No Transaction Fee Mutual Funds only. Avoid transaction fees in anyway you can.
2. Avoid futures derived commodity etfs, leveraged etfs, short etfs, options unless you know clearly how these are valued and how they work.
3. Buy Big Cap dividend players.
4. Buy index funds or index based etfs.
5. If you can pick a bottom or if you are sure the bottom has passed, buy securities that are oversold due to certain events. For Eg., BP was a good pick a few months after the oil spill. In these cases, the markets sells more not knowing what the impact would be of the event.
6. Realize your gains when they are. There is merit to the statement 'ride your winners', but it is only a matter of time before your winners become losers. So try layering your sell orders when you have profits.
7. Cut your losses at the earliest. Have some percentage and time based rules for getting out. Follow them strictly.

Of the above, the last rule is most difficult to implement. I have most trouble implementing this rule. I still have a few securities that violate this rule and are sitting blood red in my account as of now. The problem for me is that, the securities that I have sold have done a volte face as soon as I have sold. Also losses are difficult to realize. You hold on to the losers in a hope that things may change as soon as you sell them. Sometimes you are waiting for the market to turn around to minimize the losses. You say to yourself, this security is oversold. Let it turn around a bit and I will sell. When they do, you dont know when to sell. Your mind plays tricks. Do you let your mind rule your trades or do you let the rules mind your trades. Experienced traders say it should be your rules.