cancel
Showing results for 
Search instead for 
Did you mean: 

VMBU showing erroneous data

Community Veteran
Posts: 19,100
Thanks: 437
Fixes: 21
Registered: 31-08-2007

VMBU showing erroneous data

On visiting VMBU recently I decided to have a look at the 'View Internet Activities', something I don't do very often.
I observed usage showing for 'Gaming' at 2%. I don't do gaming.
Today, it was showing as 1%, presumeably because I've been doing browsing only and that % of total has increased, thereby reducing the other. BUT just looked again and it's back at 2%. I don't do gaming!
On previous visits, this and other months , one observation was there that there was 0% showing for e-mail, I do do e-mail but not a lot so assumed that it was such a small % it wasn't registering.
I have upto date virus checking software and antispyware which is run and updated regularly as well as firewall. The PN firewall is on. My Firewall Icon in the notification area does not show traffic when I'm not expecting it, so I am confident I don't have any virus or other problems. I do not have a wireless connection.
From the Help and Support Pages, I found this.
QUOTE:
You may see records of activities that don't match your usage, such as gaming usage, even though you don't play online games. This is background traffic that our monitor has classified incorrectly. You should only be concerned if you see a lot of usage of an unexpected type............................ END QUOTE
I don't know if anyone else has seen similar problems. I know others have reports problems with VMBU not being updated etc.
So all in all I am not having a lot of confidence as to whether VMBU is even showing my other/total usage correctly.
10 REPLIES
James
Grafter
Posts: 21,036
Registered: 04-04-2007

Re: VMBU showing erroneous data

Hiya,
There's a massive difference between our reporting on the amount of usage you and the proportional breakdown of your traffic.  Because of all of the different protocols, ports and so on that are used, the breakdown of traffic is just a rough guide, so see some unexpected traffic shouldn't be a concern.
The total usage is completely separate and is recorded in a different manner and is extremely reliable.
Community Veteran
Posts: 19,100
Thanks: 437
Fixes: 21
Registered: 31-08-2007

Re: VMBU showing erroneous data

Sorry James, that's a slightly woolly answer. I will come back in a moment when I've finished dabbling with my calculator, but it's not looking good!
Community Veteran
Posts: 3,789
Registered: 08-06-2007

Re: VMBU showing erroneous data

As I understand it, the VMBU totals are received directly from BT via their RADIUS logs.
The VMBU breakdown is calculated from data exported from the Ellacoyas.
Whilst theoretically they should total exactly the same amount, it is likely that there will be discrepancies.  The main reason I can put forward as an example is the peak/off peak transition period.
BT regularly post updates to the data transfer usage.  Taking an example that they send data every 10 minutes, Plusnet will discard any data-usage that occurs over the transition to peak or off-peak time.
eg:
23.51 -> BT transmit usage, Plusnet log it as 'peak'
00:01 -> BT transmit usage.  Plusnet discard this packet as they cannot reliably tell how much usage is peak and how much is off-peak
00:11 -> BT transmit usage, Plusnet log as 'off-peak'
This means that there is potentially a chunk of usage that isn't recorded in the VMBU 'totals'.  However, the Ellacoyas are likely to record at least some, if not all, of this usage so the VMBU breakdown will end up totalling more than the VMBU totals (if that makes sense).
I'm sure there are other circumstances where the figures will not tally up directly.  However, the BT totals sent through from RADIUS are the ones used to calculate whether you exceed your allowance or not, and I believe these are fairly applied.
B.
prichardson
Grafter
Posts: 1,503
Registered: 05-04-2007

Re: VMBU showing erroneous data

Think of it like an electricity meter.
The VMBU total records the total number of unit used within your property, and we could leave it there if we wanted to. But why bother when we can break it down, allowing customers to understand usage and control it.
The breakdown would be like classifying what types of device use what ammounts of the electrical units you use, say split between entertainment, lighting, heating, cooling and other (to give random categories).
The overall total is indeed correct and always will be, breaking down how it is used is not so simple, and as per the statement on the support pages, some will appear in incorrect categories. It's when you are seeing more than you currently are that there should be a concern.
Community Veteran
Posts: 19,100
Thanks: 437
Fixes: 21
Registered: 31-08-2007

Re: VMBU showing erroneous data

Sorry for not responding quicker, got side-tracked elsewhere, and stomach was rumbling.
Hi Barry,
Quote from: Barry
As I understand it, the VMBU totals are received directly from BT via their RADIUS logs.........................the BT totals sent through from RADIUS are the ones used to calculate whether you exceed your allowance or not, and I believe these are fairly applied.

Thanks for that explanation, that at least restores confidence in the VMBU totals that show on the main page Smiley,  assuming that BT haven't got any exchange or other problems Huh, I'll have to keep a careful eye on their clock Wink
Quote from: Barry
..............................there is potentially a chunk of usage that isn't recorded in the VMBU 'totals'.  However, the Ellacoyas are likely to record at least some, if not all, of this usage so the VMBU breakdown will end up totalling more than the VMBU totals (if that makes sense).

Makes perfect sense. So considering the way we are charged then that all seems perfectly reasonable and fair.
As the VMBU breakdown is only shown in %, it's a bit difficult to make any exact calculations. But as my free usage this month has been low so far, taking the 24% from the breakdown, of my VMBU main page total gave a figure some 26% different from the VMBU free total Shocked so James's use of the word 'massive' would seem appropriate here  Roll eyes .
Whilst I'm not going to get over-excited about that as the actual Mb values aren't huge, taking eg. 2% of say a 1GB monthly allowance, some 20MB, I would like to know what type of usage would get mistaken for that amount of gaming? I don't use any unusual protocols, mainly browsing, a bit of e-mail (insignificant this month so far), saving a few web pages and some pdf files, one file download so far (38MB), and a very odd bit of video streaming (usually off-peak).
@Phil Richardson, whilst I appreciate you are trying to be helpful, your very simple analogy may help those who couldn't follow Barry's explanation, but you don't address the primary issue -
Quote from: P
......................................................... breaking down how it is used is not so simple, and as per the statement on the support pages, some will appear in incorrect categories. It's when you are seeing more than you currently are that there should be a concern.

If it's large enough to appear as a % on the breakdown, then it is of concern.
I've seen reference to the Ellacoyas elsewhere, in relation to VMBU but if I'm not mistaken, also for traffic shaping, but I've skimmed over a lot and not retained much, so I will have to do some more background reading before I can make further (constructive I hope) comment, but I have a feeling that as this issue has obviously been around for some time (why else is it mentioned in the support pages) then perhaps there should be a few red faces at PN towers for not getting this addressed, after all the average user would expect typical usage to be shown relatively accurately.
Community Veteran
Posts: 3,789
Registered: 08-06-2007

Re: VMBU showing erroneous data

Quote from: Anotherone
I've seen reference to the Ellacoyas elsewhere, in relation to VMBU but if I'm not mistaken, also for traffic shaping, but I've skimmed over a lot and not retained much,

The Ellacoyas are technically 'accounting' servers.  Their main function is to analyze a packet of data (or a datastream) and identify what sort of traffic it is. 
Once they have identified some data, there are several avenues open to them.
a)  Record the type of traffic against the customer, to display the VMBU data (the true 'accounting' purpose)
b)  Set the QoS (Quality of Service) field in the packet so that all of the Plusnet hardware knows how to prioritise that particular packet across the network
The actual traffic shaping isn't done by the Ellacoyas, I don't believe.  That job is handled by another part of the network (I would hazard at guess at the routers that connect the centrals to the network).  However, they do work on the premise that the Ellacoyas identify the traffic first.
Prioritising the traffic is actually more complicated than it seems.  Of course, it's based on identifying the packet in the first place, but that information then has to be combined with the user and their particular product (i.e. they're on BBYW Pro). They also need to check whether that user is restricted (i.e. they've used up all their included bandwidth).  It also has to be combined with the various hourly schedules (i.e peak time), and also the network condition (Plusnet has the ability to divert 'spare' network capacity to allow protocols that are usually low-priority to soak up any leftover allocation after all the gold/platinum traffic has been accounted for).  There is also an 'overall' network status which can be brought into effect (for example if they have a central drop) which can lower the priority of some traffic on an 'emergency' basis.
So, all of this is calculated in real-time as the packets flow across the Ellacoyas. 
As to why you're getting a 'significant' amount of traffic identified as Gaming - I suspect it's straight-off misidentification on behalf of the Ellacoyas.  it may even be 'Internet Background Noise' such as port scans - some of which may be identified as Gaming purely because they scan *all* available ports.
Obviously, there are gaps in the ruleset that feed the Ellacoyas.   Dave Tomlinson posted here a brief overview of how rules get added.  The important thing to bear in mind that the more rules that the Ellacoyas have to process, the longer it will take (or the more likely that the resource limits on the devices themselves will start being a problem).  Whilst they're operating in sub-millisecond times at the moment, it would cause massive problems if they started inducing lag on the very traffic they're trying to prioritise as titanium!
Anyhow, I think I meandered around the topic enough already, so I'll stop there and wait for Tommo to come and correct where I'm wrong Wink
B.
EDIT:  I forgot to add that it's probably likely that the Ellacoya's don't work on a strictly 'store-and-forward' mechanism.  I wouldn't be surprised if they let the first few packets of data through either unidentified, or prioritised until the particular stream has been identified and placed into the correct prioritisation queue.
ChrisL
Grafter
Posts: 733
Thanks: 2
Registered: 13-12-2007

Re: VMBU showing erroneous data

Quote from: P

The VMBU total records the total number of unit used within your property, and we could leave it there if we wanted to. But why bother when we can break it down, allowing customers to understand usage and control it.

This seems to me to be the main point. Last month I exceeded my usage allowance by 180Mb. In the first week of this month VMBU showed 63Mb of Gaming usage. I don't do gaming either.
At this rate I could easily exceed my allowance again WITHOUT KNOWING WHY. I can make guesses about Skype server activity being misconstrued, or whatever, but it doesn't really help me decide what action to take if I want to maximise my allowance for surfing, streaming, or whatever.
Community Veteran
Posts: 19,100
Thanks: 437
Fixes: 21
Registered: 31-08-2007

Re: VMBU showing erroneous data

@ChrisL
You've hit the nail on the head.
@Barry Z
Thanks for a very clear and concise insight into the job of the Ellacoyas. It's clarified what was a fuzzy recollection at the back of my brain somewhere, not having yet had the chance to do that background reading. The link was also useful.
I shall add your post to my favourites for future reference (sorry should I say 'Bookmark'  Roll eyes ).
The only bit I couldn't really get to grips with was the bit about why traffic may be mistakenly identified as Gaming
Quote from: Barry
................... it may even be 'Internet Background Noise' such as port scans - some of which may be identified as Gaming purely because they scan *all* available ports.

I don't use any port scanning software (that I know of!), my firewall is set to only allow specific programs access (IE, OE, AVG update etc), everything else has to ask. So are you saying that in effect (DS) port scanning traffic, even if it doesn't get past the PN firewall, gets counted as usage?
pierre_pierre
Grafter
Posts: 19,757
Registered: 30-07-2007

Re: VMBU showing erroneous data

probably, as usage, as far as PN is concerned, is both up and down stream, it all takes bandwidth
Community Veteran
Posts: 3,789
Registered: 08-06-2007

Re: VMBU showing erroneous data

If you're using the Plusnet firewall (another thing the Ellacoyas have to take into account Wink) then your downstream traffic should be blocked before it passes the Centrals. 
This really does leave very little that could be identified as Gaming traffic.  However, it could still happen. For example a single IP could be classified as a gaming host, but may also host websites, of which you visited one.
Or, some content (ie a video) on a website may have been dynamically linked from another site using a strange port like 29000, which is often identified as a gaming port.
Plusnet do tread a fairly fine line using Deep Packet Inspection (DPI) to identify packets, and not incur the wrath of privacy watchdogs and consumers.  Already there have been questions raised as to the "legality" of DPI so anything that could be considered too invasive may be passed over in favour of a signature which may be incorrect 1% of the time.
Also, the Ellacoya database is being updated all the time so a misclassification should be fairly quickly identified and fixed.
It's not a perfect system - especially when consumers may also be trying to 'game' the system to increase priority on protocols.  However, I'm happy that 90% of the VMBU breakdown is correct and I'm also happy that the VMBU totals are correct so I'm not concerned enough to investigate any small amounts of misclassified usage.
HTH
B.