[taotree]

Originally Posted by paolfili »

If you are using the Zenfire' s API consider also the well know "UDP" problem of the feed...which invalidates many timestamp.
Anyway I think that the "correct" sequence (send by the Exchange) can be preserved only by a "Ticker Plant" design platform.
Zenfire doesn' t offer this...
:-(

All that has nothing to do with my comment--it's all still valid. The only thing you pointed out is that Zenfire does not satisfy the requirement in my last statement.

And to be accurate... Zenfire's out of order problem doesn't invalidate the timestamp--it's just passing that along from the exchange--it invalidates the sequence of arrival.

[paolfili]

The thing is a stop is just a market order sitting there waiting to execute at a certain price or better. If a limit comes in and crosses that order it will execute and never display on the order book.


Are you sure about it?
Is this the correct Exchanges behaviour?
Where can I find some (CME,EUREX,etc) explanation on the market dynamics you suggest? (order on trade never displayed on the book)

Thanks

Originally Posted by BlowFish »

This I think we can agree on For me sequencing is what is important. particularly preserving sequence across bid ask and last data.

I don't want to get hung up on the trades between best bid and ask but they can and do occur. Maybe the Lee & Ready algorithm papers might explain things better than me. The thing is a stop is just a market order sitting there waiting to execute at a certain price or better. If a limit comes in and crosses that order it will execute and never display on the order book. It is also quite possible for this price to be between the current best bid and ask. Just look at a data run particularly around news or a busy open - it happens even on the ES.

I agree It is bothersome not knowing if a string of events happened at the same time or in sequence but slightly apart. More often than not they do occur at the same time. (I have a hunch we will have differing opinions on that ) If you see price tick above the days high and 500 orders execute at the same time there is a pretty good chance that they really did. above price extremeties there will be 100's of orders on both side of the book (stops and limits) just waiting to be filled. You would not want to see them with different time stamps nor should you. Of course with aggregation (and we dont really know who is an by how much) things get very hard to slice and dice.

[paolfili]

Sorry,but the "real" problem of Zenfire' s API is not the "out of order" (as you have noted in the Zenfire' s forum).You can solve it by re-sorting the timestamps.
Sadly to say,but the "real" problem is that UDP gives no guarantee that any packet Zenfire send will arrive.

Originally Posted by taotree »

All that has nothing to do with my comment--it's all still valid. The only thing you pointed out is that Zenfire does not satisfy the requirement in my last statement.

And to be accurate... Zenfire's out of order problem doesn't invalidate the timestamp--it's just passing that along from the exchange--it invalidates the sequence of arrival.

[AgeKay]

Originally Posted by RichardTodd »

Contracts bought and sold are always equal amounts, obviously, so... all you are really saying here is that you don't know how to interpret a delta indicator. In general, price moving down while delta runs up indicates not much size on the bid. And lo and behold, your example has the smaller trader on the bid. See?

I know how to interpret the delta indicator. What you don't see though is that after taking out the bid, 800 were left on the offer. He might even offer an additional 1000 and some long traders will be hitting the bid to get out of their losing position since they won't get filled on the offer with a huge offer like that. You always need to interpret whatever you are looking at. My point is that indicators like this leave out a lot of information.

Originally Posted by RichardTodd »

On the other hand I have seen trades matched inside the spread countless times on the NYSE, and I think it's fairly common knowledge that the specialist has a certain amount of time to hold orders for matching (which effectively makes all batched orders "simultaneous"). Or at least they used to.. it's been a while and maybe the hybrid system changed that up.

I'd never trade a market with specialists. You have an information disadvantage. In Futures everyone sees the same information.

Originally Posted by BlowFish »

Maybe the Lee & Ready algorithm papers might explain things better than me.

You are comparing apples with oranges. These papers use historical data from the exchanges that includes spread trades and OTC trades that are not reported in real-time so you do not see them in a live-feed any way. So yes they do happen, but you won't see that live-trading. I was on the phone with TT and Eurex yesterday with regards to another issue and they confirmed that you won't see spread trades and OTC trades taking place in the data feed. I've actually gotten filled several times without seeing a trade being reported at that price, which must have been a spread trade.

Originally Posted by BlowFish »

The thing is a stop is just a market order sitting there waiting to execute at a certain price or better. If a limit comes in and crosses that order it will execute and never display on the order book. It is also quite possible for this price to be between the current best bid and ask. Just look at a data run particularly around news or a busy open - it happens even on the ES.

Of course, it does. You see it in Time & Sales, don't you? Well if it's there, then it was in the order book, you just couldn't see it with your eyes because it happened to quickly for your eyes to see. I can actually prove this because I have developed a way of visualizing all changes in the order book. And I do look at it during news.

Originally Posted by BlowFish »

I agree It is bothersome not knowing if a string of events happened at the same time or in sequence but slightly apart. More often than not they do occur at the same time. (I have a hunch we will have differing opinions on that ) If you see price tick above the days high and 500 orders execute at the same time there is a pretty good chance that they really did. above price extremeties there will be 100's of orders on both side of the book (stops and limits) just waiting to be filled. You would not want to see them with different time stamps nor should you. Of course with aggregation (and we dont really know who is an by how much) things get very hard to slice and dice.

The exchange still executes these market orders sequentially (it's software after all) and if it's sequential then there is a small time difference. If you don't understand this, then I don't see the point of continuing this discussion. You just say stuff that isn't true and makes no sense.

Originally Posted by taotree »

Now, that assumes that the data flow from exchange through feed provider all preserve the sequence of events as they send it to you

Well, they don't. At least in none of the data feeds I've worked with can you get sequential order book updates. Instead you get the state of entire order book at one point in time.

Originally Posted by paolfili »

Is this the correct Exchanges behaviour?
Where can I find some (CME,EUREX,etc) explanation on the market dynamics you suggest? (order on trade never displayed on the book)

You won't find it because it's not true. I think before believe anything on an anonymous forum you should use common sense and/or call the exchange and just ask them if it is important to you to know how that works.

[taotree]

Well, they don't. At least in none of the data feeds I've worked with can you get sequential order book updates. Instead you get the state of entire order book at one point in time.

Which data feeds have you worked with? Aside from the drop/sequence issues, Zenfire does give you sequential order book updates. And from what FulcrumTrader has said, I get the impression that DTN does give you that without the issues.

[BlowFish]

AgeKay I really can't understand why you can not see this. You are obviously a smart guy and a poster I respect. You seem to be coming at this from a completely blinkered direction. Your original argument was an order can not execute at 'half a tick' (showing an assumption that the spread can never be more than a tick wide) this suggests to me you have really not considered this properly.

Incidentally most of the papers on Lee & Readys algorithm either exclude data that there is not a complete audit trail for or present results with and without it. OTC trades certainly do not account for trades between best bid and ask. In fact as you can see them in a live data stream by your definition they can not be OTC trades.

Maybe if we go through things one step at a time we can see where we have a disconnect.
Would you agree a limit order improving best bid or ask can arrive and be elected immediately if there are enough stop orders currently on the book to fill it?

[RichardTodd]

Originally Posted by AgeKay »

I know how to interpret the delta indicator. What you don't see though is that after taking out the bid, 800 were left on the offer. He might even offer an additional 1000 and some long traders will be hitting the bid to get out of their losing position since they won't get filled on the offer with a huge offer like that. You always need to interpret whatever you are looking at. My point is that indicators like this leave out a lot of information.

Delta's only there to show you one thing, and if that's not the thing you are wanting to see, don't blame delta. That's like complaining that a hammer can't turn bread into toast. And if you are trying to pull data out of a delta indicator that it's not designed to show you, then I would argue that you actually don't know how to interpret it. Sorry...

As I'm sure you know, the whole point of most indicators is to leave out some data in order to highlight other features. Very few are isomorphic to the data they start with. I could go around complaining that my low-pass filters leave out a lot of high-frequency data, I guess. But would I be making some kind of "point"?

[AgeKay]

Originally Posted by taotree »

Which data feeds have you worked with? Aside from the drop/sequence issues, Zenfire does give you sequential order book updates. And from what FulcrumTrader has said, I get the impression that DTN does give you that without the issues.

You're right taotree. I need to take back my statement. I was wrong. I've worked with Ninja Trader, IQFeed's API (DTN), X_Trader API and another one that was tied to a specific broker. I've worked so long with the X_Trader API that I thought all of them didn't give me sequential order book updates while I just checked the API documentation of each data feed again and it looks like X_Trader API is the only one that doesn't. I stopped working with Ninja Trader and IQFeed pretty quickly because Ninja Trader didn't allow access to its data from external applications and I had latency issues with the IQFeed.

Originally Posted by BlowFish »

Would you agree a limit order improving best bid or ask can arrive and be elected immediately if there are enough stop orders currently on the book to fill it?

A limit order that improves the best bid or ask will by definition be the new best bid or ask. Do you agree? If so, then you have to agree that a trade cannot occur between the best bid or ask, right?

I don't understand "if there are enough stop orders currently on the book to fill it". Stop orders are triggered by trades not limit orders improving market prices. And even if they were, they would still trade on that bid or ask and not some magical "mid-point". All I have been saying is that a trade always occurs at the best bid or ask.