Leveraging Behavioural Finance for Better Investment Outcomes
Updated: Jun 17
Behavioral finance is often criticized for being a messy conglomeration of biases and anomalies. The field is often positioned as an adversary to traditional finance, as if the two fields have radically different implications. In this article, I will examine the Efficient Markets Hypothesis (EMH) and compare it to the worldview embodied by Behavioral Finance. These two theories offer up the same practical advice on intelligent investing, although their reasons differ. Behavioral finance cannot help (most) investors to beat the market consistently, but it can improve investment outcomes in other, important, ways. By synthesizing some of the most important work on behavioral finance, I provide some of the most effective, actionable insights that the field provides. Most investment professionals have not created systems that make use of their insights, despite apparently widespread knowledge of their importance. This article hopes to provide some practical examples of how to implement important academic findings in behavioral finance, thus embedding this knowledge in the real world.
How do markets work? Efficient Markets versus Behavioral Finance
There seems to be an important difference in worldviews between efficient-markets-theorists and behavioral economists. Professor Eugene F. Fama of the University of Chicago, in 1966, expanding on work by Paul A. Sammuelson (1965), developed the Efficient Markets Hypothesis (EMH), which states that security prices reflect all available information. In 2007/2008, after the world financial system was decimated, the EMH took a hit. After all, if current prices are the best approximation of intrinsic value, how could they be so wrong, as they were with regards to housing prices in the United States of America (U.S.) in 2007?
Behavioral finance – pioneered by thinkers like Daniel Kahneman, Amos Tversky and Richard Thaler – asserts that market prices reflect two things. First, like the EMH, information is contained in market prices. However, behavioral finance proposes that market prices also reflect the collective cognitive bias of all investors. That is, investors can be unreasonably optimistic, pessimistic, or otherwise irrational, causing security prices that are not tied to fundamental value. Richard Thaler, in a discussion with Eugene Fama (2016), gives an example of this irrational behavior. Thaler (2016) gives an example of a closed-end mutual fund, ticker symbol: CUBA. These mutual funds can have share prices that deviate from the collective value of the assets owned by the fund. This mutual fund had no holdings in Cuba (the country), since there were no securities available for purchase in Cuba, and it was illegal to do so. On the day that the then-president, President Obama, announced a relaxation in trade relations with Cuba, the fund nearly doubled in price from $90 to $170. This $170 price represented a significant 70% premium compared to the value of the assets in the fund. Thaler argues that this represents a bubble: a case where the market is valuing a security at a price measurably higher than the security’s rational (or fundamental) value.
What do these theoretical concepts have to do with wealth management? To some financial practitioners, academic theory appears to be lost in minutia, caught up in a realm disconnected from the real world, real results and real clients. It is my belief that behavioral finance and the EMH are players on the same team. Together, they help us win the investing game. How? Well, in this discussion, I will ask the major questions that these theories invite. Then, using evidence-based research and analysis, I will provide answers that will empower the decision-making process and capacity of investment professionals to maximize the value they create for their clients.
To what extent are financial markets efficient?
The Efficient Market Hypothesis is an old idea. Adam Smith, in An Inquiry into the Nature and Causes of the Wealth of Nations (written 1776), described how the industrialized capitalist system could outperform the mercantilist system of the time. In a free market, Smith argues, with intelligently designed institutions, competition regulates the economy with its (in)famous “invisible hand” to balance supply and demand through prices (Britannica 2019).
It took almost 200 years for Smith’s ideas to be scientifically examined in modern financial markets. Eugene Fama and Paul Samuelson, working in the 1960s, developed the EMH from two different research agendas. Samuelson (1965) showed that prices cannot be forecast, implying that any changes in price are the result of new, unexpected information. Eugene Fama (1965) used empirical analysis to show that stock prices are random and unpredictable. This result is caused by the actions of many active market participants all attempting to profit from their information. Even small changes in information are used by investors to realize profits, quickly eliminating the opportunity that such information can provide. The faster information moves, and the cheaper trading is, the more market prices reflect all available information. Fama’s world is one where “investors form expectations rationally, markets aggregate information efficiently, and equilibrium prices incorporate all available information instantaneously” (Lo 2007).
The EMH is a simple proposition, i.e. asset prices fully reflect available information. The EMH has three different forms: Strong, semi-strong and weak. Only the strong form asserts that asset prices fully reflect all available information. The semi-strong form suggests that all relevant publicly available information is included in security prices. The weak form of the EMH holds that security prices reflect all information about past-prices, implying that technical analysis cannot reliably add value to an investing strategy in the long-term (Fama 1970).
If the EMH is true, the only way to make profits is by taking on a risk that is priced into the market. In other words, risk explains differences in returns, not skill or informational advantages. The trouble with the EMH is that it is a difficult hypothesis to test rigorously. In addition, the approach that one takes to investing is determined by one’s views on market efficiency. If markets are efficient, then investing is a simple matter. Asset allocation should be based purely on the risk profiles of the investor, and active trading should not add value. If one believes that the market is not efficient, then an investor will seek to outperform the market by using prediction-based security selection and market timing. An investor who takes this approach, the approach taken by traditional active management, assumes that she has skill or information that gives her an edge.
I have proposed two different modes of thinking above. Which one should a given investor or investment professional subscribe to? In other words, what hypotheses can we examine that would allow us to choose an evidence-based investing philosophy? I will propose two hypotheses which, if true, validate the EMH.
Hypothesis 1: Short-term stock prices follow a random walk
In an efficient market, we would expect short-term market price fluctuations to be random. If short-term market price fluctuations are random, then price changes are the result of new, unexpected, information.
Conclusion 1: We accept our hypothesis and conclude that short-term stock prices do follow a random walk.
There is strong evidence for the weak-form EMH according to numerous empirical studies (Fama 1970), including recent studies with more recent techniques and data (Durusu-Ciftci, Ispir and Kok 2019).
Hypothesis 2: Luck, not skill, determines success among active traders
In an efficient market, we would expect that any differences in return among active traders is the result of luck. In other words, those who outperform the market do so because they are outliers. In a world of millions of investors, probability theory would predict some outperformers. To test this hypothesis, Fama and French (2010), conducted a study to measure the alpha (α) – market-beating returns not explained by risk – of mutual funds in the U.S.
Conclusion 2: A small percentage of investors can outperform the market, but it is impossible to predict who they will be.
Fama and French (2010) find that from 1984 to 2006, mutual fund investors tended to underperform comparable benchmarks by the costs they charged. In other words, mutual fund investors performed like the market, but charged higher fees for this service, leaving investors with the market return less fees. Fama and French (2010) also examined 3,156 individual funds and used statistical techniques to account for luck. The authors found that some managers did have sufficient skill to cover their costs, but these a few and far between. In a study by Dimensional Fund Advisers (2019), 3,097 equity mutual funds were examined. The study reported that of the original 3,097 funds, after 20 years only 42% were still in business, and only 23% had beaten the market over 20 years. Further, past performance of mutual funds gave no indication of future success. When selecting mutual funds who had returns in the top quartile of the previous quarter, only 21% of them (on average) outperformed again for the following five years. If anything, previous outperformance was correlated with future underperformance. Exhibit 1 below (Dimensional Fund Advisers 2019) indicates this.
Source: (Dimensional Fund Advisers 2019)
The EMH appears to be a good, if imperfect model, of financial markets. Can the imperfections of this model, exemplified by behavioral finance, be exploited to make profits in financial markets? Behavioral finance points to market anomalies, often referred to as “bubbles”, which appear to be systematic mispricing of assets (an indication of irrational investing activity). Robert Shiller, Yale economist, developed a historical housing price index in the United States. This data was presented in the second edition of Shiller’s book (2005) Irrational Exuberance, where he showed that home prices were steadily rising in real terms, and without good explanation (at the time). Shiller predicted that rising housing prices could lead to a collapse, which they did after 2007. However, if the cause of this collapse was indeed “Irrational Exuberance”, it has not gone away. Exhibit 2 illustrates data for real US home prices, from 1880 until the end of 2019. The exhibit illustrates that real home prices in the U.S. are nearing their 2008 highs, having never reached “normalized” levels (indicated by 100 on the index) . Does this mean that there is another bubble? Or does it mean that housing prices have reached a new, much higher equilibrium? If the first is true, how can we predict when it will end? If the latter is true, then the 2007 housing crash was not, in fact, a bubble, or if it was, it was much smaller than we thought.
Source: (Yale 2020)
Why illustrate this? Well, if bubbles exist, it is impossible to know how to tell they were bubbles before they burst. In other words, a bubble is only a bubble in hindsight. Therefore, timing the market by shorting bubbles is dangerous and difficult. If bubbles represent a market inefficiency, and it is difficult to say if they do, then profiting from them is nearly impossible.
In Berkshire Hathaway’s annual letter to its shareholders in 2019 (p. 11), Warren Buffet reiterated one of his classic folksy anecdotes: “A venerable caution will forever be true when advice from Wall Street is contemplated: Don’t ask the barber whether you need a haircut.” Investment professionals are paid for their expertise and, having studied for years to obtain them, are understandably reluctant to accept that they cannot beat the market. It requires more than a small dose of humility for the so-called “smart money” to step back from the complex analysis, research and honed instincts for which investors pay large fees. Of course, these active professionals set prices in markets with their trading, an invaluable service, and there will always be a place for the enterprising to profit in financial markets. However, investment professionals still need to come to terms with the likeliness of this outcome.
I have asked: to what extent are financial markets efficient? Markets are not perfectly efficient. Eugene Fama himself has long held that this is the case. The EMH is a model. Like any model, it is not perfect and does not perfectly reflect the real world. However, markets are efficient enough (especially where information flows quickly, and trading costs are low) that arbitraging any opportunities in mispricing is extremely difficult. Ray Dalio, a lauded U.S. hedge fund manager commented in an online TED talk (2020), commented that “most investors will not be able to play the game [of investing] well…being successful in the markets is more difficult than getting a gold medal in the Olympics…We put hundreds of millions of dollars into the game every year and its tough.” Markets are therefore efficient enough that time spent trying to outperform the average market return is not well spent. In fact, behavioral finance and efficient market theorists largely agree on the best investing strategy for almost all investors. Richard Thaler and Daniel Kahneman, behemoths in the behavioral finance world, both recommend buying and holding index funds as an intelligent investment strategy. This may be disappointing for some investors, but behavioral finance still has much left to offer.
Leveraging behavioral finance for better investment outcomes
I hope that I have argued that behavioral finance should not be used as a tool to time the market. Despite the EMH being only partly true, the data illustrate that the EMH is a good working hypothesis for any investor. The question then becomes, how can behavioral finance and behavioral economics (the two fields are largely related) improve the way investment professionals make decisions to improve outcomes for their clients?
Behavioral approaches to investing have gained traction with popular publications – including Nudge (Thaler and Sunstein 2008) and Thinking, Fast and Slow (Kahneman 2011) – bringing behavioral reasoning to a wider audience. Despite this, there is significant resistance by investment professionals to fully embrace behavioral thinking in their work. In fact, investors exhibit some significant behavioral biases themselves, and eliminating these as far as possible should be a good start in improving client outcomes. Investment professionals are subject to the same cognitive biases that all people experience, although scholars differ on the effects of experience in reducing some biases including overconfidence (Bailey, Kumar and Ng 2011 & Zahera and Bansal 2018).
Khaneman & Tversky (1979,1986) and Kahneman (2003,2011) present compelling evidence for the numerous cognitive shortcomings of the human being. In Kahneman’s popular 2011 book, Thinking, Fast and Slow, Kahneman presents two modes of thinking. The first, our ‘fast brain’, uses evolutionary traits to create useful heuristics, snap-judgments, and intuitive answers. Our fast thinking, or system 1 thinking, is typically subconscious. Mostly, this type of thinking serves us well. When you hear a ‘quack’, you assume it is a duck. When you are approached by a smiling salesman at the mall, that sinking feeling in your stomach lets you know you should decline the advances or end up paying too much for dead-sea salt products. These cognitive shortcuts reduce our mental load and help us focus on more important tasks.
Unfortunately, these system 1 cognitive processes can be hijacked by marketers or simply malfunction, leaving us with less-than-desirable outcomes. They say that the worst thing that can happen to a new stock market investor is winning big on their first trade. Why? Well, we all tend to assume that recent trends will continue. This recency bias causes us to systematically overweight the recent past and systematically underweight long-term trends. Professionals correct for this to some extent, but it is difficult to control. Kahneman himself is skeptical of our ability to train our system 1 cognitive processes. In a talk he gave at Google in 2011, he noted, “I do not know of a lot of evidence that people can change their system 1 thinking, unless you have acquired specific skills through reinforced practice. What you can clearly do is educate your system 2 and you can learn to recognize situations where system 2 can take-over your reactions. Thereby you can avoid some mistakes, but it cannot be done too much. If I do not sound too optimistic about training system 1, it’s because I am not.”
Mullainathan & Thaler (2000) and Thaler & Benartzi (2004) illustrate other ways in which we can change behavior for better outcomes by using nudges. A nudge is an intervention that aims to change behavior without changing their economic incentives. One clear example is the setting of default options. Suppose you are president of the country (Nudgeland) and believe that scientific research and medical training has much to gain from post-mortem examination. One could embark on a costly education initiative to increase the rate at which people donate their bodies to science, which would be the traditional way of tackling the problem. In Nudgeland, in order for your body to be donated to science, you must fill in a body donation form at the local government office. The process is relatively easy and takes about an hour. Traditional methods of tackling this problem involve education the public about the importance of donating for this purpose, speeding up the process of opting-in or even offering a financial incentive. A more cost-efficient, and likely more effective, approach could be to require every person not wishing their bodies to be donated to opt out rather than opt in. Notwithstanding any ethical implications, such a system is elegant and brilliant, and significantly changes outcomes. Such is the power of the nudge.
Theoretically, this is all ok, but how could an investment professional take these important concepts and apply them to improve investment outcomes, client relationships or business operations?
1. Take the train in Tokyo
Japan has a rail system that is world-renowned. Moving 12 billion people in a year takes serious work, though, and serious safety precautions. Shisa kanko, pointing-and-calling is Japanese innovation for industrial safety. The practice apparently reduces workplace errors by up to 85%. White gloved, blue-hatted, straight-armed conductors pointing and calling to thin air might seem crazy to the untrained eye, but it is highly effective. Rather than relying on a worker’s habits alone, reinforcing a task physically with an audible cue raises the safety check from the unconscious to the conscious. What this practice reveals is that training employees in a systematized way, and increasing the level of consciousness at which an activity is done can drastically reduce mistakes. So many mistakes occur because we allow them to, we do not make sure they do not happen. It is possible to drastically reduce bad decisions by putting barriers in place that prevent them (Richarz 2017).
Charlie Munger is vice chairman of Berkshire Hathaway, and notes that “It is remarkable how much long-term advantage we have gotten by trying to be consistently not stupid, instead of trying to be very intelligent”. The same kind of thinking is going on here. Control what can be controlled by limiting mistakes and good outcomes will follow (CNBC 2017).
Investment professionals can implement this kind of thinking by establishing a ritual for every trade. For example, portfolio managers should have a clear goal and investment thesis in mind for their portfolios. Every time a security is bought or sold, the manager should stop and ask out loud “does this security match my investment philosophy, in terms or risk, expected return and/or ethical standards?”. She might also ask “Am I making this trade based on recent media I have consumed?” or “Have I run my thinking past a trusted colleague?”. Practices like this may seem silly, but they will help to clarify decision-making. The key is to set up structures like this before you need them, so that they are ready to catch you when you fall – as, inevitably, you will at times.
2. Trust in the algorithm
Kahneman (2016), noted in an interview that even seasoned professionals can have widely diverging assessments of a situation if given too much discretion. In his consulting work, he worked with large financial institutions, where loan approvals and insurance judgements are made frequently. Kahneman asked team leaders how much they expected valuations from two different professionals would vary. The answer most frequently received was 5-10%. In reality, valuations differed between 40-60%, an order of magnitude more. These valuations were conducted with the same data. In a word, people are noisy. We receive the same data and interpret it differently, so we come up with different answers. Algorithms work differently, if you plug the same data into an algorithm twice, you get the same answer. By using algorithms, even if you don’t have a ton of data, you can narrow your range of outcomes and pursue a given strategy more effectively.
Implementing this kind of strategy is not as difficult as it may first appear. If the choice is between completely non-systematic, gut-based, decision-making and an imperfect algorithmic process, behavioral economics favors the latter. Kahneman (2016) gives an example of how he implemented a 6-criteria rating process for hiring in the Israeli army. The selection process pre-Kahneman was completely intuitive and hiring was done based on the gut feel of the hiring staff. Kahneman suggested rating 6 attributes, punctuality, and sociability among them, on a scale of 1-5. Adding up these scores gave an objective way to enlist better Israeli soldiers.
3. Take the long, broad view
Samuel Johnson was an English writer, essayist, poet, and playwright. He lived in the 18th century and is credited with the quote “People need to be reminded more often than they need to be instructed”. This is true of investors especially. In a study conducted by Charles Schwab Investment Management, more than 300 financial advisers were surveyed (2019). In a report entitled “The Role of Behavioral Finance in Advising Clients” the most effective behavioral bias mitigation technique was reported to be “taking a long-term view”. Exhibit 3 below illustrates the survey responses when the financial advisers were asked what they thought the most effective techniques were for mitigating behavioral biases among their clients. These techniques could apply equally well to the advisers themselves; I suspect.
Source: (Charles Schwab Investment Management 2019)
What exactly does taking a long, broad view mean in this context? These are really two different pieces of advice, but they are related. I will begin by describing what taking a ‘broad view’, or implementing ‘broad framing’ (McCaffrey 2018) means in this context. Consider an example: I offer you a bet. I will flip a coin, and if it comes up heads, I win and you lose $1,000. If it comes up tails, I lose and I give you $1,100. In Kahneman’s (1979) famous experiment, most people do not take this bet. Rational choice theory would predict that people should take this bet. The expected outcome is: (0.5)*1,100 – (0.5)*(1,000) = 50. In other words, the expected payoff from such a bet is $50 dollars. Most people, however, do not take this bet when offered it for real. Why? People weigh losses more heavily than equal gains (loss aversion). This is dangerous in the long run, because life is a continuous number of small bets. If you don’t take bets with positive expected returns, you will lose out. Broad framing helps here.
Let’s suppose I make you the same bet, but now I offer to flip the coin 100 times. Would you take the bet now? Most people will. The law of large numbers is in your favor now, whereas on a once off basis, you were less certain. To practice broad framing, “See each decision as a member of a class of decisions that you’ll probably have to take” (McCaffrey 2018). When discussing potential risks, especially in market turmoil such as we are experiencing now, frame things broadly to clients.
Implore clients to see each day, week or month of the stock market like a coin toss with the odds I illustrated above. You might lose one toss, or two, or 5 in a row if you are unlucky. But if you keep tossing that coin, based on the information you have now, you expect to come out ahead in the long-run, and that’s the best decision you can make now.
This brings us to the second piece of related advice, sticking it out for the long run. Framing like this can help to implement this long run approach. A Charles Schwab Investment Management report (2019, p. 6) found that “Particularly in periods of volatility, reminding clients of their investment goals and ensuring they adhere to a sensible financial plan can help them reduce emotional reactions and avoid making poor investment decisions.”.
Evidence suggests that markets are very, though imperfectly, efficient. Experts in behavioral finance and classical finance agree that beating the market is not an intelligent strategy for the vast majority of investors. Behavioral finance is therefore not a useful tool for outperforming financial markets. However, behavioral finance provides right insights which can improve investment outcomes in other ways. By synthesizing work in behavioral finance from influential authors, I demonstrate three practical ways that academic work in behavioral finance can be applied by investment professionals in their everyday work. These insights include: creating structured routines to avoid common mistakes, creating algorithms for better decision-making and framing events broadly so as to capitalize on frequently occurring bets with positive outcomes.
Bailey, Warren, Alok Kumar, and David Ng. 2011. "Behavioral biases of mutual fund investors." Journal of Financial Economics 1-27.
Britannica. 2019. The Wealth. August 12. Accessed May 19, 2020. https://www.britannica.com/biography/Adam-Smith/The-Wealth-of-Nations.
Charles Schwab Investment Management. 2019. The Role of Behavioral Finance in Advising Clients. Research Report, Charles Schwab Investment Management.
Chicago Booth Review. 2016. Are markets efficient? June 30. Accessed May 19, 2020. https://review.chicagobooth.edu/economics/2016/video/are-markets-efficient.
CNBC. 2017. Charlie Munger: The power of not making stupid decisions. August 4. Accessed May 19, 2017. https://www.cnbc.com/2017/08/04/charlie-munger-the-power-of-not-making-stupid-decisions.html.
Dimensional Fund Advisers. 2019. MUTUAL FUND LANDSCAPE. Research Report, Dimensional Fund Advisers.
Durusu-Ciftci, Dilek, Serdar M Ispir, and Dundar Kok. 2019. "Do stock markets follow a random walk? New evidence for an old question." International Review of Economics & Finance 165-175.
Fama, Eugene. 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work." The Journal of Finance 383-417.
Fama, Eugene. 1965. "Random walks in stock market prices." Financial Analysts Journal 55-59.
Kahneman, Daniel. 2003. "Maps of Bounded Rationality: Psychology for Behavioral Economics ."
The American Economic Review 93 (5): 1449-1475. —. 2011. Thinking, Fast and Slow. Farrar, Straus and Giroux.
Kahneman, Daniel, and Amos Tversky. 1979. "Prospect Theory: An Analysis of Decision under Risk." Econometrica 263-291.
Knowledge at Wharton. 2016. Daniel Kahneman’s Strategy for How Your Firm Can Think Smarter. June 8. Accessed May 19, 2020. https://knowledge.wharton.upenn.edu/article/nobel-winner-daniel-kahnemans-strategy-firm-can-think-smarter/.
Lo, Andrew W. 2007. "Efficient Markets Hypothesis." In The New Palgrave: A Dictionary of Economics, by L Blume and S Durlauf. Palgrave Macmillan Ltd.
McCaffrey, Paul. 2018. Daniel Kahneman: Four Keys to Better Decision Making. June 8. Accessed May 19, 2020. https://blogs.cfainstitute.org/investor/2018/06/08/daniel-kahneman-four-keys-to-better-decision-making/.
Mullainathan, Sendhil, and Richard H Thaler. 2000. Behavioral Economics. Cambridge: National Bureau of Economic Reseach.
Richarz, Allan. 2017. Why Japan’s Rail Workers Can’t Stop Pointing at Things. March 29. Accessed May 19, 2020. https://www.atlasobscura.com/articles/pointing-and-calling-japan-trains.
Samuelson, P A. 1965. "Proof That Properly Anticipated Prices Fluctuate Randomly." Industrial Management Review 41-49.
Shiller, Robert J. 2005. Irrational Exuberance. New Jersey: Princeton University Press.
Talks at Google. 2011. Daniel Kahneman: "Thinking, Fast and Slow" | Talks at Google. November 11. Accessed May 15, 2020. https://www.youtube.com/watch?v=CjVQJdIrDJ0.
TED. 2020. What coronavirus means for the global economy | Ray Dalio. April 9. Accessed May 19, 2020. https://www.youtube.com/watch?v=yrxYhv2O3wU.
Thaler, RH, and S Benartzi. 2004 . "Save MoreTomorrow™: using behavioral economics to increase employee saving." Journal of Political Economy 112 (1): 164-187.
Tversky, Amos, and Daniel Kahneman. 1986. Rational Choice and The Framing of Decisions. Stanford: Stanford University.
Yale. 2020. ONLINE DATA ROBERT SHILLER. March 1. Accessed May 19, 2020. http://www.econ.yale.edu/~shiller/data.htm.
Zahera, Syed Aliya, and Rohit Bansal. 2018. "Do investors exhibit behavioral biases in investment decision making? A systematic review." Qualitative Research in Financial Markets 210-251.
 “Irrational” in the behavioral finance context simply means a judgement of value that is not based on a security’s underlying value and prospects.