Don't Be Fooled By Randomness

[Don't Be Fooled By Randomness]

and finally, what NEVER changes is the way that professional interests act on the market with respect to inventory bias....they ALWAYS look to squeeze whichever side is out of balance....the result? You can actually use HFT activity to get a nice boost toward your profit targets (assuming you have a reasonably decent system and can figure out where the longer time frame targets are)....

 

What amuses me is that so many here want to debate this at length...without taking the time to really think about what is realistic in terms of usable info....

 

Hi Steve,

 

The first paragraph above seems useful info - please could you clarify and expand upon it a little?

 

Thanks,

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

I can see random orders being placed and pulled, but it is hard for me to understand why a firm would place random executions unless that firm controlled the bulk of the order flow of that instrument or was part of group of market makers who together controlled the market place. Otherwise, I don't see it and would like to know how or why they would do it.

 

Hi MightyMouse,

 

Some orders get placed miles away from the last traded price and aren't expected to trade, just intended to spoof the order book, I guess. And some firms probably do control most of the order flow in some thinly traded equities. And some institutions seemingly use iceberg type algos for execution that have randomising elements exactly as TheDude describes.

 

All of this is, of course, completely anecdotal - I have no first hand experience of how a market-maker or large institution operates.

 

Just to progress the discussion, I think it is perfectly plausible to have a profitable strategy that has randomly generated entry orders . . .

 

And I think that people are continuing to confuse the question of whether price is random with the question of whether price is predictable (by which I mean conforms to probabilistic models). The two need not be mutually exclusive.

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

Are they really doing this or is this just a possibility that someone thought of?

 

Developers have acknowledged the use of random cycle generators, it seems, but I think this particular application is just informed speculation (it came from an Eric Hunsader article). The idea seems to be to try and screw other participants' read of market depth.

 

Any suggestions as to why else certain firms would spit random orders (and also orders arranged in pretty but pointless geometric patterns) into the market?

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

No one inputs orders 'at random'.

 

Hello,

 

That's not true.

 

Consider the following time series:

 

2,792,34,4,7,2,567,114,9,1,8,8,441,16,93,4

 

What's the pattern?

 

The pattern is that the number doubles - 2,4,8,16 . . . But you can't see the pattern because I have disrupted it with randomly generated numbers. I know what the numbers are, so I can simply discount them to see the underlying sequence, but you don't know which numbers are random and which are pattern generated.

 

I have added noise to the time series.

 

It has been widely reported that certain market making firms insert orders of random size, at random price, and with random cycle frequency, into the markets to add noise that they can decode but that will disrupt the way that price and volume data is perceived by other particpants.

 

Furthermore, consider that random order placement and random markets are not synonymous.

 

BlueHorseshoe

[Don't Be fooled by Randomness Part2 - Day Trading Vs Swing Trading - Who Makes More?]

But the reason i started this discussion was to get a feel for who is doing really well day trading

 

Jim Simmons.

 

and what kind of objective methods they are using.

 

Hidden Markov Models, allegedly.

 

But that's not a great deal of use to you or me. Nor is knowing whether anyone posting here uses TA successfully (the answer is "yes" - law of large numbers!).

 

Here is a possible alternative question - 'Day Trading vs Swing Trading - Who Expends Least Effort for Proportionally Maximal Gain?' Just a thought . . .

 

BlueHorseshoe

[Don't Be fooled by Randomness Part2 - Day Trading Vs Swing Trading - Who Makes More?]

Sure, Albnd. I first figure the Kelly number for each system (and sometimes for both longs and shorts separately). Kelly was originally developed as a gambling approach for bet size. It tells you the % of your total capital you should risk on each bet (or trade) to maximize your total gain without going bust, assuming you know the Win % and the Win/Loss ratio, and that they do not change going forward (which of course they will in trading).

 

I then take a percentage of that Kelly number (say 15% of what Kelly says I should risk - this is usually determined by modeling a given system and seeing what the drawdowns would be given certain percentages of Kelly). Then I set the maximum risk of my total capital to that amount on every trade that particular system places.

 

For example:

 

1. I first caclulate the Kelly for the system, as shown below. Let's say it works out to be 30.2%.
   
2. Then, I pick a pecentage of that number to test. 15% of 30.2% Kelly would be a 4.53% risk per trade (.15*.302=.0453). I keep testing different percentage of Kelly until I find one that gives me a balance of profitability and drawdowns that I like. (To give you an idea, my total portfolio of systems gives me drawdowns of >40%, but I'm comfortable with that. So choose a drawdown that you can handle and be conservative in this estimate because the drawdown will be larger than in testing or history.)
   
3. So, then I take total capital times percent risk to get my total $ risk per trade. Say, $100,000 total capital * 4.53% = $4,530 total risk per trade.
   
4. Then I find my stop loss per contract on the next trade. I always make my stop loss dynamic, rather than a set amount, by setting my stops a certain percentage of recent volatility away from my entry. ATR (or higher timeframe ATR) works well. Let's say in this example that it's $500 per contact.
   
5. Then I just divide total capital risked by stop per contract and round down: $4,530/$500 = 9 contracts.
   

 

So, what happens is you continue to risk the same amount of your capital on your system on every trade. If you make profits, i.e. - more capital, then your quanitity will rise; if you lose money then it goes down. I try to always think of profits and losses as percentages rather than dollar amounts. That makes all trades (regardless of equtiy) comparable. Hope that helps.

 

From Investopedia:

There are two basic components to the Kelly Criterion:

• Win probability - The probability that any given trade you make will return a positive amount.

• Win/loss ratio - The total positive trade amounts divided by the total negative trade amounts.

 

These two factors are then put into Kelly's equation:

Kelly % = W – [(1 – W) / R]

 

Where:

W = Winning probability

R = Win/loss ratio

 

It's worth noting that the two components, win probability and win:loss ratio, must each be calculated over a large enough sample size to be statistically significant - this means lots of trade data, and not simply minimal trade data derived from lots of price data.

 

Kelly is supposedly what Larry Williams used.

 

BlueHorseshoe

[Position Sizing and Predicted MAE]

If ES volatility is at or slighlty below avgV and NQ v is above nq avgV then which one do you buy? Do you ever buy less than account size? If you intend to go all in and use your entire buying power everytime you take a postion then why is position size part of the equation?

 

Certainly, less than the entire account size can be bought.

 

The idea is to be fully invested for the typical trade. If a trade is deemed likely to be atypical - a likely outlier in terms of dollar excursion from the entry price (due to what I am calling "volatility"), then the idea is to decrease the position size.

 

"Fully invested" means only for the portion of total equity that is allocated to that class of instruments. So the "equity" in my pseudocode refers only to a portion of total equity (in the EL code I have sent, I have plugged in the figure of 5k to avoid the confusion of further variables).

 

Priority of allocation within a class of instruments (such as equity indices - the ES and the NQ in your example) is based on a different selection metric altogether, so their relative volatilities would not be considered by the strategy.

 

The purpose of the code, taking (equity/close) as a base scenario, is to try and make all trade outcomes as much like the average trade outcome as possible, by accounting for the predicted price behaviour (MAE/MFE) relative to average price behaviour. The typical trade provides the average outcome, and the average outcome is aligned with (equity/close), with everything else scaled around this.

 

BlueHorseshoe

[Position Sizing and Predicted MAE]

 

You're probably sorting the net profit column of the optimization report. Export to excel, add a column that measures deviation of returns, sort that column, that's the optimal f that will satisfy your smoothing requirements, while forsaking net, unless your fitness function strikes a balance between net and smoothing.

 

Besides optimal F, ... R is the only other moving part. I was confident i had thought this through when I wrote the paragraph below here but after thinking for too long I'm unable to determine if increasing R increases trade size. If it does then it's backwards. Substituting hard numbers in place of the variablies on line 4 is too much for me at the moment but that should be easy to resolve. If R is wired wrong it will need to be fixed before F will work. If R is wired correctly then you will still need the custom excel function to optimize F.

 

As R increases it simultaneously increases trade size. R normalizes individual trades to a variable scale that causes fluctation in trade net and Equity Curve. Fixed size normally returns a smoother EC than variable size. Varying size is normally associated with increasing gain. Varying size can be used to smooth an ec but if you use range as the input it needs to work backwards from the way it's written. When anticipated range increases forgo the opportunity for increasing profit. Decreasing size will maintain the fixed net profit for that trade where the net per contract is increased for that trade.

 

Hi Onesmith,

 

Good to see you around the forum again and thanks for your reply.

 

I can't be certain whether you've understood what I'm trying to do or not - this is not Ralph Vince's Optimal F, and I'm not trying to optimise anything. Here's an explanation of my thought process and goal:

 

1. The base strategy uses zero leverage and whenever it is not flat it is fully invested. So the position size for that is simply Equity/Close.
   
2. The problem is that two positions with identical equity available and the same entry price can have markedly different outcomes depending on "volatility" after entry.
   
3. In my formula f is just a simple multiplicative function derived from past trade data (average-trade-length) and past price data (highest(h,average-trade-length) - lowest(l,average-trade-length)) that has on average been the best predictor of "volatility" average-trade-length periods into the future. In every case I have examined f has been close to 1 (ie the best predictor of future "volatility" is current "volatility"), so b*f is practically identical to b (my intention is that f could theoretically be replaced with some kind of higher order polynomial extrapolation)
   
4. c is the average "volatility" for the entire data sample (because this takes a while to compute and the sample size quickly becomes too large for the max bars to reference in TS I have replaced it with a totally recursive LQE).
   
5. Average "volatility" or c is then aligned with the base scenario (equity/close).
   
6. Position sizing when b*f is higher than average should therefore be proportionally smaller, and when b*f is lower, proportionally larger, according to the following: positionsize = (equity/close) * (c/[ b*f ]);
   
7. When "volatility" is higher than average, the formula calls for more units to be purchased than the equity allows with leverage; lower than average volatility is therefore simply ignored by capping position size at a maximum of equity/close.
   

 

I would expect all this to result in reduced net gains but also reduced deviation of returns and a smoother equity curve. Somehow, it isn't doing that . . .

 

Any suggestions as to where the issue may be?

 

Thanks,

 

BlueHorseshoe

[Are There Any Professional Parties Using Easylanguage Instead of C ++ Language?]

Studying Easylanguage myself, however I notice all the professional parties request developpers with C ++ knowledge.

 

Anyone know of a profesional party using Easylanguage or do they all use C + or C++ ( .net) or other languages?

 

Hi MMS,

 

EasyLanguage is adapted from Pascal (thanks to ZDO for telling me that), and I'm sure that this is/has been used by professionals in the industry at some time . . .

 

C++ is certainly not ubiquitous - take a look at a few profiles of regulars over at quant.stackexchange.com. Here's a list of recognized languages from an application form for a quantitative fund:

 

Java

C

C++

PHP

Visual Basic

Perl

Python

C#

JavaScript

Ruby

Common Lisp

MATLAB

 

I think a lot probably depends on the suitability of the language for the particular task, and what everyone else in the environment is using. Ultimately the underlying concept value has got to be more important than the language it's coded in, right?

 

Nobody is ever going to try and run an HFT operation using EasyLanguage (I hope!), but if you're essentially end of day trading then it should be able to do everything you need (in fact, you can probably do everything you need with paper and a calculator).

 

Hope that helps,

 

BlueHorseshoe

[Position Sizing and Predicted MAE]

Hello,

 

Assuming that the best predictor of b at time t=-a is b * f, why isn't the following position sizing formula helping me to reduce the deviation of returns?

 

a = average trade length;

b = highest high ( a ) - lowest low ( a );

c = average b ( population size );

position size = ( equity / price ) * ( c / [ b * f ] );

 

Any help very much appreciated!

 

BlueHorseshoe

[Don't Be fooled by Randomness Part2 - Day Trading Vs Swing Trading - Who Makes More?]

There is a lot of information in those time frames. It's going to be tough seeing it until you change that belief.

 

There's a lot of information about everything everywhere. The volume of information doesn't seem important to me - the information's utility is all that matters.

 

The information contained in any timeframe is only as valuable as its utility permits, and that depends on the strategy being deployed.

 

Information from one timeframe can, of course, have utility in other timeframes.

 

It is sensible to focus on the information that provides the greatest utility for the strategy in question, wherever that information comes from.

 

BlueHorseshoe

[Don't Be fooled by Randomness Part2 - Day Trading Vs Swing Trading - Who Makes More?]

None of that shows on a chart.

 

The point is that whether a daytrader or swing trader "makes more" has absolutely nothing to do with whether the trader is daytrading or swing trading but rather with the competence of the trader.

 

Successfully exploiting what is visually/graphically an identical price movement on a daily chart will result in a greater net gain than exploiting the same price movement on an intraday chart, but the fixed costs will remain the same for both.

 

Take the left hand chart you have posted above. Select an entry and exit point that would result in a profitable trade. If that chart were daily, you might have made 100 ticks per contract. If the chart were one minute, you might have made 2 ticks per contract. Both from what, graphically, is an identical price movement. If your fixed costs are 1 tick then you make 99 ticks in the first instance, and 1 in the second.

 

The question is, can you perform the second procedure 99 times for every one time you perform the first? Or, is your risk sufficiently limited in the second for you to perform it once with 99x the position size of the first? Or 10 times with 9.9x the position size of the first etc? If you can do this then your returns will probably be less volatile (great for money management, and perhaps the main benefit of successful daytrading), and your "edge" will have more opportunity to play itself out.

 

Expectancy x Opportunity is the key concept in understanding this.

 

I agree though - visually, I would doubt that anyone can consistently distinguish between a daily and an intraday chart and, unless we're talking about execution-specific strategies, that there would be any merit to trading the one differently from the other.

 

And of course, the more competent trader makes the most, as you say.

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

A record of randomly generated price and volume points can from time to time appear to be non-random. It would appear to be non-random when it mimics a market generated record of a similarly priced instrument. On the other hand, a market generated record of price and volume, occasionally, but only rarely, appears to be randomly generated. A methodology which trades only random behavior will lose more often than the methodology which exploits non-random behavior if both the former and latter are true.

 

I wasn't claiming that prices are random (or that they are non-random); I was saying that it may make sense to trade as though they are always one or the other. If your statements above are true, then you could trade as though markets are always non-random, including through those brief periods when they are random.

 

In short, I was arguing for simplicity in a strategy, rather than one which tries to distinguish between two different types of price behaviour (and all the complications that involves).

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

BHS, ...and when you (happily) explain could you include an explaination in a little bit more depth why you "think it makes better sense to assume that the markets are always either random or non-random" . Thanks. zdo

 

Hi ZDO,

 

Please see my rambling reply above . . .

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

Yes horse i would like to hear you explain that happily.Don't do it grumpily (it's not your style anyway)

 

Do you think it's time to find an alternative to the word "random" ?

 

Hi Mitsubishi,

 

I think of it in the following way . . .

 

Assume that you have a "perfect" methodology for exploiting both random and non-random behaviour. Assume that I have a perfect method for exploiting only random behaviour.

 

Price is behaving randomly, so we're both making money. As price behaviour begins to change to non-random, we both start to lose money. Eventually (unless you can also be assumed to have some sort of perfect zero-lag regime switching model - one assumption too many for me!), you will start to make money once more as you shift to your non-random methodology.

 

You seem to have stolen a gain on me, until . . .

 

The market begins to behave randomly again. From the very instant this happens I am perfectly placed to take advantage of it. You, meanwhile, are waiting for your model to recognise the shift. If the shift is not clean then your model gets whipsawed. My model would never get whipsawed - it's just 'right' or 'wrong'. Also, consider that even if we both make exactly the same, your switching model has expended more 'effort' than mine. So it's less efficient.

 

Even if you have a sound method for identifying such shifts in something like real time, then other problems may arise:

 

1) Consistencyof returns is key to successful money management/position-sizing. In an ideal world, you want all trades to be as similar as possible - the closer every trade is to your avaerage trade, the better (this is why an outlier strategy that has higher single contract average returns as a second strategy may not perform as well as the second when both are optimally position-sized). How likely is it that your returns from each of your methodologies (random and non-random) are truly afine?

 

2) As per my previous post, the rules governing shifts from random to non-random may change, should they even exist in the first place. Most likely, such cycles are also random, at least some of the time . . . Whatever the case, introducing a reliance upon regime recognition into your approach is introducing another potential failure point (I'm sure engineers must have a term for such things). Put simply, the more things you try and respond to, the more chance you have of getting something wrong. If you just position yourself to exploit one type of behaviour, then you have no chance of getting it wrong when that type of behaviour prevails.

 

I hope that wasn't too grumpy, and hope it makes some kind of sense, even to those who disagree!

 

BlueHorseshoe

[Don't Be fooled by Randomness Part2 - Day Trading Vs Swing Trading - Who Makes More?]

Remove the X and Y axes and you'd be hard-pressed to tell the difference.

 

Have a look at your commissions and spread (fixed, whether you're holding for a thousandth of a second or 3 years), and you'll be hard pressed not to

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

and for the record (again) markets cycle from random to non-random,

 

I agree, they do seem to. But are these cycles you describe random, non-random, or does that depend on some further 'meta-cycle' of randomness/non-randomness?

 

I find the idea of a strategy that discriminates between random and non-random market behaviour uncomfortable, but that's just me. I think it makes better sense to assume that the markets are always either random or non-random (a clearly false assumption), and then to find approaches that balance losses and gains favourably during those periods when this assumption is false. I'll happily explain why if anyone is interested.

 

BlueHorseshoe

[Don't Be Fooled By Randomness]

 

Someone please explain that to me

 

Hello,

 

It's fairly simple really: perfectly random behaviour is highly predictable. The probability of a long sequence of similar events is very small. You can bet against the continuation of such sequences as they unfold.

 

Please do note that I am saying the explanation to your question about randomness is simple, and NOT that trading is simple!

 

Regards,

 

BlueHorseshoe

[EasyLanguage Assistance]

Hi Deej,

 

In very simple form your code needs to include the Darvas Box code, and would then look something very roughly like this:

 

If condition1 then
Buy next bar at btt {btb} stop;

 

And in the case of a shooting star, defined as condition2, then like this:

 

If condition2 and H[1]>=btt[1] and btt=btt[1] then
Buy next bar at btt stop;

 

Hope that helps, and if you have any questions I'll check back tomorrow.

 

Kind regards,

 

BlueHorseshoe

[Looking for Fx Advice (intraday Trends)]

I have a decent fundamental understanding of the fx market. however.. I learned.. (learning) how to trade futures..not doing well currently w my live acct. anyhow thats beside the point.

 

I would like to know is the fx market better suited for a trend trader than emini index futures and CL futures? I would think that it is.. due to the massive daily volume compared to futures daily volume.

 

basically..will I see more trend signal entry opportunities (intraday) in fx market? less whipsaws? because lately Ive been trading morning session eminis and and crude.. and I see little to no entry signals and the ones I take are mostly all whipsaws! I'll be up 10-12 ticks..then price will go against me and I get stopped out. plus the commissions are killing me at over $5 per trade.

 

one exception that I saw recently was a huge uptrend in CL early last week..of course I could not trade that day..it was like a 90 tick move up! but for the most part CL has a lot of whipsaws..and the risk is too much for me due to the quick price moves and range.. I set my stop at 10 ticks..and this seems to be too small. anyway..back to the topic at hand:

 

which currency pairs are more suited for the intraday trend trader?

 

I know this topic is highly subjective, but I am looking into other options..just trying to get some good input..thanks in advance

 

Hi Shooly,

 

You've probably answered your own question above . . . I have no experience with CL, but as far as currencies versus an index e-mini goes, then it's probably a reasonable assumption to suggest that currencies trend (or 'breakout') more than indices do.

 

I assume this is because a currency (or frozen orange juice, or lean hogs, or crude oil etc) represents the actual price of something, whereas an index doesn't - it represents the aggregate value of a whole bunch of things which might have something in common (they might all be US stocks, say), but also have seperate and unique influences upon their price.

 

Take a look at the behaviour of a sector index such as XLV, for example, which represents stocks that have a lot in common, versus something like VTI, which is based on the value of a whole bunch of arguably dissimilar stocks. Which trends more?

 

Try combining a whole bunch of completely un-related instruments into one big, hypothetical index - what do you see? Lots of noise, most likely, as the index is not representative of any real, concrete thing. If you wanted to trade mean-reversion then a basket tracking such an index might be an idea, but for trend trading it would probably be poison.

 

You might find it useful to define some measure for how breakout-prone an instrument is in your chosen timeframe (the percentage of the range of each bar that occurs outside the range of the prior bar is one simple measure that I like), and then compare each market using this metric.

 

Finally, remember that the FX vs Futures dichotomy is misleading: a currency future will trend in a very similar manner to the relevant cash pair.

 

Hope that helps,

 

BlueHorseshoe

[Introduce Yourself Here - Don't Be Shy!!]

Hi,

I'm trading futures (CL, QM, ES, C) since 2010, but traded also forex and stocks for many years. I'm a 100% mechanical/quantitative trader, and I used to develop my systems on my own. I traded many strategies on TradeStation and NinjaTrader platforms. I'm currently interested in sharing ideas and projects on this nice forum.

 

Hi Albnd,

 

Welcome to TL - mechanical traders are a little under-represented here, so I look forward to seeing you around the forums!

 

Kind regards,

 

BlueHorseshoe

[Regime Changing]

Hi Suby,

 

It might be worth searching through some of Jeff Swanson's articles on this site - he has posted useful and well tested ideas for regime switching models incorporating concepts like hysteresis.

 

If you just want to know whether to be long or short only, then you can do a hell of a lot worse than a simple moving average - the problem is always arriving at a solution that works and is not curve-fitted.

 

Regards,

 

BlueHorseshoe

[Print (EasyLanguage)]

Hello all, I am new here.

 

I have a few questions about print().

 

1) If I am trying to get strategy/indicator data out of tradestation en mass into csv files in order to analyze it with other programs (excel, SAS etc) is print() the most efficient way to do this? Or is there some better way I can get all of my charts to print data automatically?

 

Welcome to TL, Investor!

 

I have come across more efficient methods in older books (eg. Thomas Stridsman's 'Trading Systems that Work'), in which the data is exported wholesale from tradestation, but have never been able to get them to work. Currently, I do almost exactly what you're doing, as I too have only ever needed end of day equity data. Still, it's clunky, and wide open to human error . . .

 

Hopefully Tams or someone else might have some suggestions?

 

BlueHorseshoe

[Making a Living Off of Trading 1 Contract a Week]

This guy has some ideas that could help,he calls it "harmonic rotation"I guess you either keep your stop away from those zones or you aim for them ie enter while others are covering.

futurestrader71 - $NQ_F http://stks.co/2SsS NQ Harmonic rotations study result... | StockTwits

 

February 7th Chat: Harmonic Rotations | FuturesTrader71

 

I got some useful ideas from that - thanks!

 

BlueHorseshoe

[Technical Analysis Software in C++ or Perl]

If you have any difficulty getting a free trial, you could always sign up for a trial with Infinity Futures and gain access to Sierra Charts and plenty of live feeds that way.

 

Hope that's helpful . . .

 

BlueHorseshoe