Jump to content

Welcome to the new Traders Laboratory! Please bear with us as we finish the migration over the next few days. If you find any issues, want to leave feedback, get in touch with us, or offer suggestions please post to the Support forum here.

  • Welcome Guests

    Welcome. You are currently viewing the forum as a guest which does not give you access to all the great features at Traders Laboratory such as interacting with members, access to all forums, downloading attachments, and eligibility to win free giveaways. Registration is fast, simple and absolutely free. Create a FREE Traders Laboratory account here.

jperl

Trading with Market Statistics XI. HUP

Recommended Posts

I wonder if the fact that "The skewness of a random variable X is sometimes denoted Skew[X]. If Y is the sum of n independent random variables, all with the same distribution as X, then it can be shown that Skew[Y] = Skew[X] / √n." can be exploited to come up with a non iterative approach?

 

I am having trouble understanding this. Doesnt it mean that you can simply calculate the skew for the first element (which will be zero right?) and then extrapolate from that. I don't understand how that formula can be correct.

 

 

 

Its from http://en.wikipedia.org/wiki/Skewness

Share this post


Link to post
Share on other sites
I wonder if the fact that "The skewness of a random variable X is sometimes denoted Skew[X]. If Y is the sum of n independent random variables, all with the same distribution as X, then it can be shown that Skew[Y] = Skew[X] / √n." can be exploited to come up with a non iterative approach?

 

I am having trouble understanding this. Doesnt it mean that you can simply calculate the skew for the first element (which will be zero right?) and then extrapolate from that. I don't understand how that formula can be correct.

 

 

 

Its from http://en.wikipedia.org/wiki/Skewness

I don't think this shows some non-iterative approach. As I said I am not educated in statistics so I am merely guessing, but I think this random variable X is not a single number but actually a discretionary data set or a continous distribution function. So if you want to determine Skew[X] you need to iterate or integrate, anyway.

Share this post


Link to post
Share on other sites
Ondrej,

 

Would you mind posting the bit of code that calculates the Skew? I find it easier to understand algorithms than formulas on the whole. :)

I am not sure how familiar you are with AmiBroker Formula Language, but anyway, this is the iterative part of my code. Some parts within the iteration are ommited.

i cycle goes through bars, j cycles through price levels. This iteration computes Volume per price (VUcount), PVP, VWAP, Variance and the 3rd central moment (CentMom3). BoxesInRange is the number of price levels in the Volume Distribution function.

 

for(i = iStart; i < BarCount; i++)

{

SumPjVj = 0;

TotalVolume = TotalVolume + Volume;

if((BoxesInRange < BarCount))

for(j = 0; j < BoxesInRange; j++)

{

BoxPrice = Bot + j * TickSize;

if((Low <= (BoxPrice + TickSize/2)) AND (High >= (BoxPrice - TickSize/2)))

{

VUcount[j] = VUcount[j] + VolUnit;

MaxVUcount = Max(MaxVUcount, VUcount[j]);

}

SumPjVj = SumPjVj + (BoxPrice * VUcount[j]);

}

if((frac(i / CalcInt) == 0) OR (i == BarCount - 1))

{

VWAP = SumPjVj / TotalVolume;

for(j = 0; j < BoxesInRange; j++)

{

BoxPrice = Bot + j * TickSize;

Prob = VUcount[j] / TotalVolume;

if(MaxVUcount == VUcount[j])

PVP = BoxPrice;

Variance = Variance + (Prob * (BoxPrice - VWAP) * (BoxPrice - VWAP));

CentMom3 = CentMom3 + (Prob * ((BoxPrice - VWAP) * (BoxPrice - VWAP) * (BoxPrice - VWAP)));

}

}

}

}

Share this post


Link to post
Share on other sites
I don't think this shows some non-iterative approach. As I said I am not educated in statistics so I am merely guessing, but I think this random variable X is not a single number but actually a discretionary data set or a continous distribution function. So if you want to determine Skew[X] you need to iterate or integrate, anyway.

 

The way I read this was if you know the skew of the set of variables you can determine the scew of any particular variable (like a newly arriving bar). I'd wager my stats are weaker than yours though!

 

Thanks for the algorithm by the way. I am not particularly familiar with ambibroker but syntax dosen't look too dificult :)

 

This might interest you http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance this was what inspired me to look for non iterative approaches to VWAP and it's SD's

Share this post


Link to post
Share on other sites
The way I read this was if you know the skew of the set of variables you can determine the scew of any particular variable (like a newly arriving bar). I'd wager my stats are weaker than yours though!

 

Thanks for the algorithm by the way. I am not particularly familiar with ambibroker but syntax dosen't look too dificult :)

 

This might interest you http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance this was what inspired me to look for non iterative approaches to VWAP and it's SD's

The way I read this is that if you know the skew of a variable X (variable meant as a function), you can somehow calculate the skew of a varaible Y (again meant as a function). But I don't understand why. Reading Wiki I was also a bit confused, then I realized that in statistics they often use whole functions as variables. I noticed that if a variable is a function it is usually marked with a capital letter, such as X or Y. If a variable is a number (a value of the function at some point) it is usually marked with a small letter, such as x or y.

Thanks for the link, but my indicator for AmiBroker works fast enough to be used in real time on 15sec database for 1 day. I am a newbie and I need to learn a lot so my priorities lie elsewhere. By the way, that condition in the code "if ( frac( i / CalcInt == 0) ... ) sets an interval of calculation (frac is fractional part). When I recalculate VWAP and everything only once per four 15sec intervals, the speed of the indicator almost doubles.

Share this post


Link to post
Share on other sites

Slowly trying to get my tiny mind round this. Do you Divide this value CentMom3 by SD^3? I see you add it to VWAP and multiply by the SD for plotting.

 

CentMom3 = CentMom3 + (Prob * ((BoxPrice - VWAP) * (BoxPrice - VWAP) * (BoxPrice - VWAP)));

 

Seems like you should to me. I finally had one moment of clarity (if you pardon the pun) still lots that's hazy. Anyone know of a better place than Wiki to properly learn this stuff?

Share this post


Link to post
Share on other sites
Slowly trying to get my tiny mind round this. Do you Divide this value CentMom3 by SD^3? I see you add it to VWAP and multiply by the SD for plotting.

 

CentMom3 = CentMom3 + (Prob * ((BoxPrice - VWAP) * (BoxPrice - VWAP) * (BoxPrice - VWAP)));

 

Seems like you should to me. I finally had one moment of clarity (if you pardon the pun) still lots that's hazy. Anyone know of a better place than Wiki to properly learn this stuff?

Yes I do sqrt(Variance) to calculate SD and I do CentMom3 / SD^3 to calculate Skew(3CM). But these operations I do not do within the iteration loop, because AmiBroker AFL is primarily an array processing language. So I do these computations on arrays outside of the iterative loop I posted.

Share this post


Link to post
Share on other sites
Yes I do sqrt(Variance) to calculate SD and I do CentMom3 / SD^3 to calculate Skew(3CM). But these operations I do not do within the iteration loop, because AmiBroker AFL is primarily an array processing language. So I do these computations on arrays outside of the iterative loop I posted.

 

Gotcha, thanks. From what I hear AmiBrokers script language is pretty efficient in execution so I guess thats a plus too.

Share this post


Link to post
Share on other sites
Gotcha, thanks. From what I hear AmiBrokers script language is pretty efficient in execution so I guess thats a plus too.
Today I improved my code so when it updates VUcount (volume per price) the j-loop doesnt go through all prices seeking where is the actual bar, but first I find the bar borders and then the loop goes only within the bar. With this improvement I can compute the indicator on sevreal days of data without any noticable slowdown, so now it is fully usable for real time even for longer periods of high detailed data.

I dont know if AFL is slow or fast compared to other softwares, because I am a beginner and AmiBroker is the first charting software I use :) But I find it very flexible, so far I could program everything I wanted there.

Share this post


Link to post
Share on other sites

Well it sounds like you are doing great with it. Good job! Funnily enough that was exactly the improvement that I made to DBtina's code for PVP. Getting stuff out of loops (or eliminating them altogether) is one of the arts of efficient code. The code I have for VWAP SD and PVP does not have a single loop in it (except to initialise arrays to zero on bar 0).

 

I am now pretty certain that this can be done in a non-iterative manner its just a bit beyond me from the maths/stats end of things.

Share this post


Link to post
Share on other sites

Well I have made some progress by completely changing my variance algorithm and basing the 3rd central moment stuff on that. Ondrej could you post the odd chart with the whole days action squashed up? I guess really I am going to have to do a long hand algorithm to test against.

Share this post


Link to post
Share on other sites

In this post of mine

http://www.traderslaboratory.com/forums/6/trading-with-market-statistics-xi-hup-2735-7.html#post43133

there is the Skew Line displayed for ES 9:30 - 16:15 in the first two of the three attachments. Filenames show dates.

If you need more, just give me a symbol, date and times (from to) and I will make a screenshot. The date should be within the last two months.

 

However, I observed the Skew Line a bit more and I recognize that sometimes PVP to VWAP relationship is better for description of the price action. Sure the Skew(3CM) is the real statistical skew, yet one has to think about what the skew is and what it means.

 

Skew is "missing expected data" compared to normal distribution. For the skew(3CM) the base of the distribution is formed with whole price range for the examined period (e.g. daily range). In this range we expect normal distribution and skew tells us the deviation from normal. Yet one has to ask if the chosen examination interval is relevant. I attached a chart for ES from the last Friday (08/01). Apparently the opening price at 9:30 was quickly rejected and the market never returned to that area again. This action created missing volme in the top of the daily range and thus a skewed distribution. Notice the thin light blue line (the Skew Line = VWAP + Skew(3CM) * SD) wrapped around the 1st SD for the whole day. Looking back one can recognize that setting the start of the examined interval after the opening price rejection would provide a better insight.

 

On the other hand you can notice that the PVP to VWAP relationship described the real situation much better. Thats because skew(PVP) is not so dependent on the daily range, it smooths low volume excesses. It is only as sensitive to daily range as VWAP is.

 

So each Skew tells you a bit different kind of information.

 

EDIT: Also notice the thick violet line showing the daily range middle. Range middle is mostly a good approximation of the skew line.

ES080801.thumb.jpg.7fd0446836e67be17f34eeb904051f43.jpg

Share this post


Link to post
Share on other sites

Thanks,

 

I do think the PVP presents some problems also, I'll discuss them later. I am really pleased with the new algorithm as it does not seem prone to numerical instability and overflow errors even when cumulating ^3's. Having said that the skew calculation looks like it needs a tweak compared to yours. I enclose my chart for the same day. I'm really not quite sure what needs doing to be honest.

 

Edit: Didn't Pearson suggest 3*(mean-median)/SD? I wonder if VWAP + 3(mean-median) might be useful?

Vwap7.thumb.png.2f031236ab822070049874c4203ee63b.png

Share this post


Link to post
Share on other sites

If your Skew Line is the blue one you have it wrong. I will recap the procedure.

 

Asume you have an i-loop going through all bars since the start of the examination period.

 

For each i you need to do the following:

 

1. You need to determine Volume per price (the volume distribution function). Lets say you will have an array VD, where VD[j] will be volume per j-th price. This VD array will change for every bar (every i).

 

2. You need to go through this array in a j-loop and calculate the following:

 

a) Probability of j-th price: Prob[j] = VD[j] / TotalVolume. TotalVolume is the cumulative volume since the start of the examination period.

 

b) You need to calculate the 3rd central moment for every bar (every i). You again need to do it in a j-loop.

CM3 = sum( (Price[j] - VWAP)^3 ). The sum goes through all j's.

 

c) You calculate the Skew = CM3 / SD^3. This you dont have to do in a j-loop.

 

The j-loops dont have to be separate, you can do everything in two. I use three j-loops for every i. In the first one I determine the VD. I do this in a separate loop because I dont go through all j's but only through those representing the actual bar. To VD array I add volume cumulatively from the start of the examination, so I dont need to recalculate it whole every bar, I need only to add the current bar volume to it.

The second loop I use for VWAP calculation and in this one I need to go through all prices.

The third j-loop I use for Prob and CM3 calculation and in this one I need to go through all prices. For this calculation I must know VWAP, so I cannot integrate the first and second loop. But if your software calculates VWAP for you and you dont need to calculate VWAP yourself, you can use only 2 loops.

Share this post


Link to post
Share on other sites

The chart for today until present time.

 

EDIT: Btw it was an awful creepy movement today before FOMC, I am glad that after 2hrs of wtching this I decided not to trade at all today.

ES080805.thumb.jpg.59332eb339063ad2184b2cbca1506898.jpg

Edited by Head2k

Share this post


Link to post
Share on other sites

Thanks for your continued dialogue! Yeah its wrong :( It is a continuous algorithm though so it doesn't use loops at all. It is based on an algorithm by West the weighted incremental algorithm here http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance. I think its real close now but something is a wee bit out. It will run on months of ES tick data though! Its a pretty neat and not too difficult to understand. I think the logic is pretty good as everything is OK except the skew. My hunch is that its the last bit where I divide (n-1) * sumweights. Not sure why though :(

 

n = 0

foreach x in the data:

if n=0 then

n = 1

mean = x

S = 0

C=0

sumweight = weight

else

n = n + 1

temp = weight + sumweight

S = S + sumweight*weight*(x-mean)^2 / temp // sum of weighted ^2's

C = C + sumweight*weight*(x-mean)^3 / temp // sum of weighted ^3's

mean = mean + (x-mean)*weight / temp

sumweight = temp

end if

end for

 

Variance = S * n / ((n-1) * sumweight) // if n = 1, omit n/(n-1)

SD = Sqrt(Variance)

Skew = C * n/((n-1) * sumweight) / cube (SD)

Share this post


Link to post
Share on other sites

I am not a programmer. To programe in AmiBroker I needed to read a manual first. So I am afraid understaning your code would cost me too much effort. Anyway, if C is the sum of weighted ^3's, why don't you calculate the skew just as Skew = C / cube(SD) ?

Maybe I missed something...

Share this post


Link to post
Share on other sites

Well another day another algorithm this one based on Terriberry's paper 'Computing Higher Order Moments Online". This one is realy neat but I don't fully understand how to weight it properly.

 

I wonder if you would mind posting todays ES Ondrej? This is what I have but I know (well am pretty sure) the 3CM isn't weighted correctly. Its the magenta line. At least its the right shape now (I think).

 

Cheers,

Nick.

Vwap10.thumb.png.526c6bc9b35ac33991793c36cc85743c.png

Share this post


Link to post
Share on other sites

Not so far away from what I have.

 

As a side note, this discussion merely belongs to the Skew thread, not this one. I posted my first post here as a reaction on a post when the 3rd central moment was first mentioned. But now we spent several pages discussing skew in a HUP thread... Maybe some moderator could move it, if it is not too difficult?

ES080806.thumb.jpg.8f40f206b6883382a10281664b410f0f.jpg

Share this post


Link to post
Share on other sites
Not so far away from what I have.

 

As a side note, this discussion merely belongs to the Skew thread, not this one. I posted my first post here as a reaction on a post when the 3rd central moment was first mentioned. But now we spent several pages discussing skew in a HUP thread... Maybe some moderator could move it, if it is not too difficult?

 

Yes I was thinking the same thing if a mod wouldn't mind?

Share this post


Link to post
Share on other sites

Hi Jerry,

Thanks for the great threads!

 

I was thinking about two things you mentioned in the treads:

1. Risk tolerance vs. Stop-Loss: If I am using risk-tolerance of 2% of account of 100K (i.e. my risk tolerance is 2K) and I am trading a specific market (e.g. YM), then actually the risk tolerance can be translated to 2K/5 = 400 points for every trade.

Do you enter a trade with default SL of 400 points away?

 

2. After examining the VWAP/PVP/SDs and their relationship you decide to enter a trade Long. Your risk tolerance is still far away, but the market conditions changed (e.g. vwap and price are now below PVP) - would you exit the trade? Stay in the game till your risk-tolerance is hit?

 

Thanks,

 

moon.

Share this post


Link to post
Share on other sites
Hi Jerry,

Thanks for the great threads!

 

I was thinking about two things you mentioned in the treads:

1. Risk tolerance vs. Stop-Loss: If I am using risk-tolerance of 2% of account of 100K (i.e. my risk tolerance is 2K) and I am trading a specific market (e.g. YM), then actually the risk tolerance can be translated to 2K/5 = 400 points for every trade.

Do you enter a trade with default SL of 400 points away?

 

2. After examining the VWAP/PVP/SDs and their relationship you decide to enter a trade Long. Your risk tolerance is still far away, but the market conditions changed (e.g. vwap and price are now below PVP) - would you exit the trade? Stay in the game till your risk-tolerance is hit?

 

Thanks,

 

moon.

 

Moon, your calculation is correct for YM. This does not mean that you simply enter a trade without managing it. If, as you point out the price action is such that the trade no longer makes sense to you, then you simply exit it or reverse it. The point about using risk tolerance, is that you are not locked in to a course of action which takes you out of the market too early because a stop was hit. Rather, it gives you the opportunity to adjust the trade by either adding size to it or reversing it without the fear that a pre-set stoploss creates.

Share this post


Link to post
Share on other sites

Hi Jerry

Thanks a lot for your answer.

 

Main reason for my previous question comes from what you wrote few pages back at this thread (as an answer to Trader333)

 

“Also think about what you would have done if the trade at the VWAP had not gone your way. Would you have exited the trade at the PVP or done something else?”

 

Trader333 answered:

I would have exited at either the PVP line or 1st Standard Deviation away from entry whichever was nearer at the time.

 

And you wrote:

That is what most traders would do and in my opinion leads to a slow bleed of your capital

 

By this answer I assume you would have stayed in the trade although market conditions changed. (?)

Can you please advise what would you have done?

 

Thanks!

 

moon

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.