Predicting Daily S&P 500 Returns - Part i

*This post is an occasional digression from the Stock Market Winners project.  This is still in the vein of looking for opportunities to exploit.  In particular, this post is the first of several to see if future returns of the S&P are predictable. This post looks at some data.  It falls short of making a prediction, but it does suggest that predictions might be possible.*

When I talk about predictions being possible I have two different meanings in mind.  One meaning is economic - can we make predictions that will allow us to enjoy better performance (more return with less risk) than a strategy without predictions (generally buy and hold).  The second is statistical.  One could always guess the future return will be some average return from the past.  Such a method would clearly create a lot of error.  The statistical measure seeks to find predictions which generate significantly less error than just predicting the average.

Let’s look at daily returns of the S&P 500. Here we have daily returns on the S&P 500 from January 03, 1950 through November 30, 2015. Note that the early data effectively only has closing prices. More recent data has open/high/low and close. There are 16585 prices (and 16584 returns)

head(spx)

##             Open  High   Low Close  Volume Adjusted
## 1950-01-03 16.66 16.66 16.66 16.66 1260000    16.66
## 1950-01-04 16.85 16.85 16.85 16.85 1890000    16.85
## 1950-01-05 16.93 16.93 16.93 16.93 2550000    16.93
## 1950-01-06 16.98 16.98 16.98 16.98 2010000    16.98
## 1950-01-09 17.08 17.08 17.08 17.08 2520000    17.08
## 1950-01-10 17.03 17.03 17.03 17.03 2160000    17.03

tail(spx)

##               Open    High     Low   Close     Volume Adjusted
## 2015-11-20 2082.82 2097.06 2082.82 2089.17 3929600000  2089.17
## 2015-11-23 2089.41 2095.61 2081.39 2086.59 3587980000  2086.59
## 2015-11-24 2084.42 2094.12 2070.29 2089.14 3884930000  2089.14
## 2015-11-25 2089.30 2093.00 2086.30 2088.87 2852940000  2088.87
## 2015-11-27 2088.82 2093.29 2084.13 2090.11 1466840000  2090.11
## 2015-11-30 2090.95 2093.81 2080.41 2080.41 4245030000  2080.41

Let’s look at the distribution of daily returns. We see that the worst daily return was -20.4669%. This occurred on October 19, 1987. The best return was 11.58% which occurred on October 13, 2008.

##      Index            daily.returns      
##  Min.   :1950-01-03   Min.   :-20.46693  
##  1st Qu.:1966-06-30   1st Qu.: -0.41141  
##  Median :1983-01-12   Median :  0.04649  
##  Mean   :1982-12-29   Mean   :  0.03382  
##  3rd Qu.:1999-06-09   3rd Qu.:  0.49767  
##  Max.   :2015-11-30   Max.   : 11.58004  
##                       NA's   :1

A histogram shows the vast majority are clustered near 0%, but the scale indicates there are rare, large outliers. Recall there are 16584 daily returns in our dataset.

The boxplots show the dispersion as well. The blue boxes contain one-half of the observations. The whiskers (horizontal lines outside the blue) are 1.5x the width of the boxes. Points beyond the whiskers are outliers. About 5.6% of the data are outliers using this definition.

Can we do better than average?

Again the average and median daily returns are 0.033824% and 0.04649% respectively. An investment of $1 would have grown to $124.87 ignoring taxes and dividends. The question is can we identify type of days which might be better than others?

Let’s start with an example called the turn of the month anomaly. With this we will only invest for the 3 trading days before and after the turn of the month.

The average return around the turn of the month is significantly different from the average daily return. In fact it is about 2.9x higher. That said, these only account for 23.8% of the observations.

Calendar Month

Some months appear better (worse) than others, but only September and December appear significant at a 5% level. September, October and November seem to have the least dispersion of monthly returns.

Trading Day of Month

Consistent with the earlier comments on turn of the month, the first few trading days of the month appear better than average.

Trading Days Left in Month

Consistent with the earlier comments on turn of the month, the first few trading days of the month appear better than average.

Day of Week

The market seems to like Wednesday’s, Fridays but not Tuesdays.

Days since Holiday

Days until Holiday

The market seems to get into the holiday spirit (any holiday) for a few days before a holiday.

A lot of attention is given to the market when it crosses the average value of some number of previous days. Here we look at several moving averages including 2, 5, 10, 20, 50, and 200 days. Here we take the last price of the market and divide by the moving average. Caveat, since the last price is used in the calculation, this can be misleading due to a look ahead bias. In a 2 day moving average, if the price has gone up, the price/SMA2 will tend to be higher and the return will be positive. Thus, this is coincident and not predictive. Even though the price is only 1 of 200 observations in the average, there is a relationship because when prices are trending up, returns will be good.

SMA2 - Price / 2 day simple moving average

SMA5 - Price / 5 day simple moving average

(due to technical issues, this will be continued in the next post)