In this post, we look at out-of-sample performance for some support vector machine (SVM) models. The purpose here is not an exhaustive analysis. We are only hoping to get an indication that our efforts might bear fruit.
Each model we build uses predictor (x) data from one month (e.g. Dec 2002) and response (y) data from the subsequent month (e.g. Jan 2003). We refer to a model here by xmonth/ymonth (e.g, Dec02/Jan03) to indicate the months of the x and y data. The y variable is a company’s excess return defined as the return of company for the month less the average return of all companies for that month.
The first model is Dec02/Jan03. The first set of predictions was for Jan 2004 created by averaging the 12 predicted returns for each company by feeding in the Dec03 x data into the Dec02/Jan03 through the Nov03/Dec03 models. The last prediction is for Oct 2015. We have 142 months of predicted return, a bit shy of 12 years.
For each month with a predicted return, we calculate three values: LongEx, ShortEx and Hedge. Respectively, these roughly represent buying the 50 stocks with the top predicted returns, shorting the bottom 50, and doing both which can be thought of as long, short and hedged portfolios. However, since our y variable is an excess return over the average stock, the values for LongEx is the return over the average. The values for ShortEx are the returns under the index. Hedge represents the long minus the short. In all cases, positive values are desirable.
Below is a summary of the 142 observations for each.
## LongEx ShortEX Hedge
## Min. :-22.5302 Min. :-32.275 Min. :-46.755
## 1st Qu.: -1.6911 1st Qu.: -1.435 1st Qu.: -2.763
## Median : 0.8873 Median : 1.280 Median : 2.070
## Mean : 0.6054 Mean : 1.349 Mean : 1.954
## 3rd Qu.: 2.8920 3rd Qu.: 4.568 3rd Qu.: 7.004
## Max. : 19.3902 Max. : 21.453 Max. : 38.014
The means and medians are all positive, but there is substantial variation. Below we look to see if the means are significantly different from zero. The standard error is the standard deviation divided by the square root of 142. That is that standard deviation of the sample mean used to determine the signicance of a sample mean. The z score is the mean divided by the standard error.
## LongEx ShortEX Hedge
## Mean 0.6053754 1.3486556 1.9540310
## SD 5.0889217 6.5421246 10.2368647
## StdErr 0.4270528 0.5490029 0.8590586
## z 1.4175657 2.4565545 2.2746190
It appears that the model is better at finding short candidates than long. For fun, the average annual return for the hedged portfolio (without transaction costs or taxes) would have been 18.15% which sounds nice but it had an annualized standard deviation of 35.46 which is much higher than a stock index.
The data in graphical and tabular forms follows.
## LongEx ShortEX Hedge R2 RMSE
## 20040102 4.6985561 2.694902012 7.3934581 1.379280e-02 11.580118
## 20040130 -3.2886064 -0.529383004 -3.8179894 1.540982e-02 9.832418
## 20040227 1.1691026 0.453155712 1.6222583 3.306673e-05 10.107648
## 20040402 -9.4197351 -5.162604176 -14.5823392 4.068867e-02 11.785238
## 20040430 -2.4486500 -2.728383848 -5.1770338 1.016575e-02 11.720396
## 20040604 2.0121839 1.505728151 3.5179121 1.343413e-02 9.551383
## 20040702 8.0205122 6.147703657 14.1682158 2.246450e-02 10.503473
## 20040730 -3.9090475 1.550578868 -2.3584686 3.820769e-03 10.185049
## 20040903 7.7911959 -5.382921111 2.4082748 9.784061e-04 9.686064
## 20040930 -1.1508347 -5.031083623 -6.1819183 1.032160e-02 10.649651
## 20041029 -5.3518646 -4.301093109 -9.6529577 1.363384e-01 4.523042
## 20041203 -6.2159879 -1.086511442 -7.3024994 5.084027e-03 10.082328
## 20041231 4.9204895 11.510903842 16.4313933 8.262045e-02 9.426351
## 20050131 10.1772604 5.080044931 15.2573053 4.480866e-02 9.867702
## 20050228 -1.1684236 5.278319163 4.1098956 2.176766e-03 9.475941
## 20050331 -0.5282040 2.610445860 2.0822419 1.266045e-02 9.545485
## 20050429 0.6247247 -2.256541253 -1.6318166 2.394808e-02 11.260544
## 20050531 5.0352092 -0.913236628 4.1219726 3.004012e-03 9.146220
## 20050630 2.7664134 -2.203862367 0.5625510 6.369541e-03 9.789016
## 20050729 5.0818389 0.397479009 5.4793179 9.274058e-03 9.841564
## 20050831 5.9381303 1.452395046 7.3905253 2.970105e-02 10.021611
## 20050930 -5.7560095 2.696531632 -3.0594778 3.815124e-03 9.652464
## 20051031 -2.0949916 -0.039789787 -2.1347814 6.963174e-04 9.563784
## 20051130 3.4136096 4.647954537 8.0615641 9.050108e-03 8.172802
## 20051230 8.4483295 -0.381620938 8.0667085 2.714860e-02 11.340995
## 20060131 -12.1742304 2.631561766 -9.5426687 1.421208e-02 9.780551
## 20060228 2.1173905 5.101000525 7.2183910 1.119775e-02 9.763087
## 20060331 4.8802301 1.889904626 6.7701347 1.327410e-02 9.601770
## 20060428 -1.2667677 -3.126810393 -4.3935781 2.321707e-02 9.831850
## 20060531 3.2954483 1.719109982 5.0145583 6.174438e-03 8.873465
## 20060630 -3.0001241 2.121893309 -0.8782308 1.204514e-03 9.915648
## 20060731 -2.9519613 -2.424451377 -5.3764127 1.623641e-02 10.395490
## 20060831 -5.8639552 0.963745152 -4.9002101 1.228122e-03 8.582910
## 20060929 0.7407814 0.885725566 1.6265070 2.870203e-04 10.038357
## 20061031 2.3024105 -1.300622627 1.0017878 3.010293e-04 9.440758
## 20061130 1.0249685 1.466223683 2.4911922 9.155474e-03 8.116759
## 20061229 2.5375217 0.005573975 2.5430956 7.218869e-03 8.558668
## 20070131 -0.6019387 3.151324637 2.5493860 1.399973e-04 8.643099
## 20070228 2.4756421 4.120796129 6.5964382 6.368336e-03 8.774652
## 20070330 2.8922754 2.497992920 5.3902683 2.757660e-03 8.303947
## 20070430 1.0951503 5.437779536 6.5329298 2.968128e-03 9.952062
## 20070531 2.7275424 1.457198392 4.1847408 7.304716e-04 8.325163
## 20070629 2.7571032 5.163734031 7.9208372 2.878221e-02 10.235410
## 20070731 -0.8270457 -5.128787655 -5.9558334 2.054421e-03 10.876710
## 20070831 9.9080912 8.096688020 18.0047792 4.729174e-02 9.946572
## 20070928 12.2245405 7.519328688 19.7438692 6.453732e-02 12.212728
## 20071031 0.8746712 5.989696081 6.8643673 1.260344e-02 11.005084
## 20071130 3.0809504 3.500391305 6.5813417 2.679592e-02 11.351225
## 20071231 -3.4125767 -16.088007679 -19.5005843 8.960132e-02 13.364583
## 20080131 14.4102077 6.581377521 20.9915852 6.625561e-02 12.387180
## 20080229 -1.4101853 6.552416407 5.1422311 3.064980e-04 11.772409
## 20080331 5.1874578 3.972513329 9.1599711 4.541956e-03 12.856980
## 20080430 8.4525058 10.209135146 18.6616410 2.898336e-02 11.754873
## 20080530 19.3902135 18.623526192 38.0137397 1.985966e-01 12.599652
## 20080630 -22.5302026 -6.662501689 -29.1927043 7.068278e-02 15.545567
## 20080731 -4.4282657 -3.324941758 -7.7532075 3.306981e-02 12.125835
## 20080829 -11.7722720 0.263123137 -11.5091489 2.532377e-03 14.980266
## 20080930 6.4162058 11.366816239 17.7830220 7.446795e-02 16.508013
## 20081031 6.5469029 10.902521769 17.4494247 6.309035e-02 16.765770
## 20081128 -4.2175975 -5.877812827 -10.0954103 2.277161e-02 16.406279
## 20081231 -6.7720346 7.445946414 0.6739118 5.081219e-04 15.932738
## 20090130 0.4117665 14.864446576 15.2762131 4.632900e-02 13.488844
## 20090227 -4.3974121 -4.918795213 -9.3162073 2.335671e-02 13.990093
## 20090331 -14.4797172 -32.275294291 -46.7550115 2.234701e-01 21.347236
## 20090430 -4.7057675 -9.717338123 -14.4231056 6.171574e-02 15.892092
## 20090529 4.6972006 2.818316135 7.5155168 2.968280e-02 11.827252
## 20090630 -0.1121372 -1.993525191 -2.1056624 8.520777e-03 12.328718
## 20090731 3.1016299 -4.183537480 -1.0819076 3.564415e-03 12.362953
## 20090831 0.1524758 -2.029120718 -1.8766449 2.133177e-03 10.820495
## 20090930 -2.7514725 0.686879790 -2.0645927 2.335912e-05 9.813227
## 20091030 0.7103931 -0.428367589 0.2820255 1.162558e-04 10.777253
## 20091130 1.9437377 2.305396547 4.2491342 1.612888e-02 9.549518
## 20091231 -0.7367724 -1.217666948 -1.9544394 9.613093e-06 9.421133
## 20100129 0.1045668 1.952638201 2.0572050 6.257510e-03 9.237443
## 20100226 4.3245460 2.726498883 7.0510448 1.817045e-02 10.484150
## 20100331 2.7296818 1.662756620 4.3924384 2.719764e-03 10.063540
## 20100430 -3.8015262 -0.452613701 -4.2541399 6.562637e-03 8.420164
## 20100528 -7.2575387 -0.911258113 -8.1687968 3.653815e-02 9.435504
## 20100630 1.6027876 2.679733175 4.2825207 6.390563e-03 9.809369
## 20100730 -2.7221437 1.621680830 -1.1004628 1.388381e-03 10.516280
## 20100831 -0.6089243 0.432357443 -0.1765669 1.076919e-06 9.545242
## 20100930 0.9420235 0.129855493 1.0718790 3.161409e-03 8.932311
## 20101029 0.8097786 4.326887549 5.1366661 1.251836e-02 9.596541
## 20101130 -2.8945491 -1.777852727 -4.6724018 6.277160e-03 9.693821
## 20101231 -0.2060415 2.475000124 2.2689586 1.200716e-03 9.093803
## 20110131 6.6128069 0.591296109 7.2041030 1.262807e-02 9.385216
## 20110228 3.7905835 6.240848831 10.0314323 1.710657e-02 9.171467
## 20110331 -2.3158950 3.987703908 1.6718089 3.226784e-04 9.182035
## 20110429 -0.9262357 0.115318542 -0.8109172 2.168133e-06 8.487364
## 20110531 1.5646763 -0.537999759 1.0266766 1.002818e-02 7.642411
## 20110630 0.9411761 -0.626953795 0.3142223 9.817661e-05 9.037727
## 20110729 -2.9199226 3.488792014 0.5688694 4.493216e-07 10.814098
## 20110831 -3.1865255 3.563159650 0.3766341 5.535636e-03 10.539890
## 20110930 -7.5664193 -12.528642700 -20.0950620 1.259601e-01 12.653926
## 20111031 1.9263643 10.756965392 12.6833297 4.244968e-02 10.356395
## 20111130 0.8999014 5.184810862 6.0847123 1.128724e-02 8.384039
## 20111230 -6.7221313 -9.691540645 -16.4136719 1.240757e-01 11.469275
## 20120131 0.2687574 -1.479615073 -1.2108576 2.920551e-03 9.030588
## 20120229 2.4354056 7.816017713 10.2514233 3.065025e-02 8.175265
## 20120330 3.6834006 0.680030588 4.3634312 2.224765e-02 8.490556
## 20120430 4.1638102 12.253976739 16.4177870 1.114420e-01 9.606406
## 20120531 -0.4048295 3.228558339 2.8237288 7.092628e-04 8.509756
## 20120629 1.6456021 -0.482056819 1.1635453 5.121104e-03 10.150198
## 20120731 -1.7659512 -0.316094792 -2.0820460 9.635983e-03 9.902787
## 20120831 0.6468250 -4.337629552 -3.6908045 6.746121e-03 7.296174
## 20120928 2.5129265 -2.047623802 0.4653027 1.488669e-04 9.060010
## 20121031 0.2621510 1.669420998 1.9315720 2.253398e-03 8.367610
## 20121130 2.7323326 -0.931846104 1.8004865 3.679373e-03 6.968976
## 20121231 2.6522951 -1.142367414 1.5099276 5.552424e-03 8.066738
## 20130131 -1.1875919 3.770420963 2.5828291 1.182282e-02 8.174054
## 20130228 4.0820997 0.755573156 4.8376729 1.802635e-02 7.285747
## 20130329 0.4781656 5.345692993 5.8238586 8.559810e-04 8.581097
## 20130430 -0.7071254 4.850713604 4.1435882 5.453313e-04 10.534415
## 20130531 0.2197082 6.878535056 7.0982432 1.310865e-03 8.509980
## 20130628 4.2246477 3.844475843 8.0691236 5.207451e-03 9.641167
## 20130731 0.1271500 -3.061753216 -2.9346032 3.533501e-04 8.744177
## 20130830 1.5263991 7.970015378 9.4964145 1.807064e-02 9.166737
## 20130930 2.4794686 1.106906480 3.5863751 3.344829e-04 10.090434
## 20131031 2.5477359 8.175574683 10.7233105 3.481279e-02 9.558321
## 20131129 -0.6981850 3.909423640 3.2112387 6.510337e-03 9.428769
## 20131231 -0.1352843 -5.486022703 -5.6213070 1.089779e-02 12.359731
## 20140131 2.6096398 -0.342498973 2.2671408 4.779181e-03 9.792065
## 20140228 -4.0514968 0.211724646 -3.8397721 1.127459e-02 8.945963
## 20140331 -0.5357733 -4.585307983 -5.1210813 2.257639e-02 9.503867
## 20140430 2.1111778 0.800018464 2.9111963 2.909139e-03 8.540904
## 20140530 -1.4664814 -3.362621251 -4.8291026 1.476260e-02 9.937716
## 20140630 1.3885452 -6.075509073 -4.6869639 2.689076e-04 9.304842
## 20140731 1.2757093 -1.584550258 -0.3088410 9.702190e-05 8.754481
## 20140829 0.9920803 -0.051360438 0.9407199 1.415707e-03 9.050361
## 20140930 -1.0384572 -1.755297434 -2.7937546 2.408312e-03 11.661520
## 20141031 5.7042581 7.826509440 13.5307676 4.991654e-02 9.954761
## 20141128 1.3398425 7.233493338 8.5733358 2.048029e-02 10.208432
## 20141231 2.8913374 3.170057740 6.0613951 2.774182e-02 10.458340
## 20150130 -6.3852028 -5.201137102 -11.5863399 2.696450e-02 10.245243
## 20150227 1.9768171 11.943729396 13.9205465 2.212988e-02 9.219389
## 20150331 -2.3002665 -22.693066046 -24.9933326 1.522786e-01 10.921062
## 20150430 7.2972803 13.304049165 20.6013294 7.528878e-02 10.049901
## 20150529 3.1343957 6.583574212 9.7179699 4.454163e-02 8.842418
## 20150630 6.0152399 21.452527715 27.4677676 1.538402e-01 10.712322
## 20150731 -2.0159471 -0.652881651 -2.6688288 2.088631e-03 9.964481
## 20150831 3.6665013 15.511893059 19.1783944 1.135260e-01 10.511436
## 20150930 -0.2650218 -4.227637311 -4.4926591 1.391680e-02 12.880863
No comments:
Post a Comment