cancel
Showing results for 
Search instead for 
Did you mean: 

Don’t Make Me Wait!

Dan
Grafter
Posts: 121
Registered: 05-04-2007

Don’t Make Me Wait!

I’m Dan Kirkland, the Head of Software Design within PlusNet. I head up the design team and technical architects for our Portals and internal business systems. One of the major programmes of work we are currently undertaking involves optimising the responsiveness of our systems. This posting aims to give you an insight into what we are trying to achieve.

Customers have been telling us that our website has slowed down in recent months and right now our Portal can take anything up to 15 seconds to load a page. Therefore, we are now focusing our attention on major upgrades and enhancements to improve the responsiveness of our website. This programme of work is called “Don’t Make Me Wait!” and aims to significantly increase the responsiveness of our Portals and, quite literally, stop making people wait for their Portal pages.

Currently it takes anywhere between 5 and 15 seconds to load a Portal page, but averaging around 6.5 seconds. This programme aims to reduce the time to about 3 seconds or less. In order to do this we will be undertaking some major enhancements and upgrades to our platform. The work involved includes:

- Database server upgrades and optimisation
- Designating servers for specific functionality
- Upgrading the PHP service
- Upgrading our page caching systems
- Reducing the size of our PHP source files
- Improving our DNS solution

Collectively, these have the aim of making our Portal respond from a “click to a page” within 3 seconds. This doesn't include reports but it does include most web pages that we host all of the data for.

Below describes these initiatives in more detail:


Database server upgrade and optimisation
This is the first thread of the programme and we are pleased to say that it was completed a couple of weeks ago on 2nd November. These alone will give us a great performance improvement by hosting our core databases on new high-end servers.

Another important database optimisation was the splitting of our database files. To explain what this did, consider having a spreadsheet with 10,000 sheets in it. Think of how long it would take to open if each sheet had thousands or millions of rows in. Also, think about how difficult it would be to manage. The optimisation split resulted in lots of smaller files being created that will be much easier to manage.


Designating servers for specific functionality
As a younger company, one of the things we liked most about our Platform was that all of our staff shared the same infrastructure as our customers. That way, if customers felt pain then so did we; if a server or service went down then we’d know about it at the same time (or before) our customers. It worked well until we grew as a business and started using the servers more than our customers, and as a result we were slowing down our customers and affecting their Portal experience.

Because of this we are now separating our business functions from the Portals and believe that this will allow our servers to work more for our customers rather than for ourselves. We are aiming for this work to be complete by the beginning of December.


Upgrading the PHP service
You may or may not know that our Portals are built using PHP, the most popular web scripting language. We now believe that it is time for us to upgrade to the most recent stable version, which will not only allow our developers to make use of the most up-to-date features, it will also allow us to use more advanced tools to further speed up our website. We expect this upgrade to be complete by mid-December.


Upgrading our page caching systems
When a web page is first requested it will be read from a server’s disk and then processed for the customer, and this can be relatively slow to complete. However, once the server has finished processing the page it will try and store it in memory ready for any future requests. This means the next time the page is requested, the server will not have to read it from disk and process it, but instead just read it from memory. This process is known as caching.

At the same time as upgrading PHP we will also be upgrading our caching tools. If the tests that we have completed are anything to go by then this alone will make a significant difference. As with PHP, we aim for this upgrade to be complete by mid-December.


Reducing the size of our PHP source files
As mentioned earlier, most of our Portal is written using PHP. With time, some of our PHP files have become quite large and could benefit from being split. Whenever a page is requested our servers have to process these large PHP files, but most of the time only a small portion of each file is needed. By intelligently splitting these files, the servers will no longer have to process large files with each request but instead only have to read small ones. Furthermore, the servers will only read in the files that it needs instead of large files from which only a small portion is required.

This work is also planned to complete alongside the PHP upgrade and completed by mid-December.


Improving our DNS solution
We have identified that certain other DNS solutions may be performing better than our current one. This has caused us to start a full review of our system and any resultant work could yield huge speed gains for our Portals. The review and proposals are due to complete by the end of November so we will keep you posted regarding progress.


So, how are we going to monitor / present performance of the platform?

We are in the process of implementing new Quality Assurance tools that will automatically test and monitor the responsiveness of our website. These tools will run through most of the actions possible on our website, including a sign-up, upgrade and downgrade, log-in, VMBU and nearly everything else that you’ll find on our Portals. If our tools fail to complete these actions within our acceptable time limits then alerts will automatically be raised to inform someone.

Furthermore, when these tools are fully functional we will not allow any changes to our Platform that result in any of the time limits mentioned above being breached. For example, if a developer makes a change to our sign-up process and as a result our tools detect that the time limits have been breached, then the change will be rejected. Therefore, these tools will help us maintain consistent performance levels for our Portals.

I will keep you informed over the coming weeks regarding progress, and look forward to seeing and experiencing the effects of the improvements.

Regards,

Dan Kirkland
Head of Software Design
20 REPLIES
N/A

Don’t Make Me Wait!

Hi Dan

Will these proposed new changes/upgrades, be fully tested before they become operational, or are we, the customer, once more to be guinea pigs for BT this time round?
csogilvie
Grafter
Posts: 5,852
Registered: 04-04-2007

Don’t Make Me Wait!

* Colin wonders what BT have to do with anything...

Hopefully these changes should mean that using the portal is a much quicker experience.
N/A

Don’t Make Me Wait!

Because, if I'm not mistaken, BT are about to complete on a takeover of PlusNet.

I may have dreamt all this though or would that be a nightmare?

Perhaps you would confirm that that is what is happening Colin?
csogilvie
Grafter
Posts: 5,852
Registered: 04-04-2007

Don’t Make Me Wait!

BT have made a bid to buy PlusNet, and the directors have recommended it's accepted. There's still something like 40% of the shareholders to agree to sell (giving them 50% control) before it's likely to happen.

However, that doesn't have anything to do with "Don't Make Me Wait" - since some of the items included in Dan's list have been on the cards since the 15th August, which was before the rumours started.
N/A

Don’t Make Me Wait!

Just checking out of the window to see if there really are pigs flying out there! Wink

Of course BT will buy PlusNet, if they didn't then PlusNet's shares will go south. PlusNet know this and so do all of the other shareholders.

I can't believe that BT are buying PlusNet for the 200,000 customers and I can't believe they are buying them for their expertise.

I can believe that BT are buying PlusNet as a test bed and that they are using the customers as guinea pigs. A bit like the puggers are used by PlusNet.
csogilvie
Grafter
Posts: 5,852
Registered: 04-04-2007

Don’t Make Me Wait!

What do your comments relating to BT have to do with the upgrades Dan is talking about in his post? As I said - they have been planned for a while before the BT announcement (or even the rumour!) came about so they are irrelevant.

If you wish to discuss the BT stuff, I recommend doing it in another thread and leaving this for purely discussing Dan's post.
N/A

Don’t Make Me Wait!

The portal has been running ever since I can remember, so it seems a bit odd to embark upon major project work like this now that Plusnet are within striking distance of being taken over.

Wouldn't it be better to wait and see what the new owners want, rather than run the risk of having the whole thing ditched or changed, which is normally what happens with most Plusnet projects and decisions? Tickets/questions, Comms Team/Voice of the Customer etc.
N/A

Don’t Make Me Wait!

I asked a simple question:

Quote
Hi Dan

Will these proposed new changes/upgrades, be fully tested before they become operational...


I added this because I was concerned and wanted to know the answer from a PlusNet representative, not someone who is intent on being picky for the sake of it.
Quote

or are we, the customer, once more to be guinea pigs for BT this time round?


I was being neither disrespectful or posting outside of the forum rules. I asked a serious question, or is that not allowed anymore?
Dan
Grafter
Posts: 121
Registered: 05-04-2007

Don’t Make Me Wait!

Quote
Hi Dan

Will these proposed new changes/upgrades, be fully tested before they become operational, or are we, the customer, once more to be guinea pigs for BT this time round?


All of these upgrades and enhancements have significant testing times alloocated to them. In fact, I'm making sure that PlusNet staff feel any pain internally first by performing the upgrade on all of our internal servers before upgrading our Portals. This way, major pain should be reserved for us rather than customers.

Of course, before I unleash the changes internally I'm also making sure that we do adequate testing - I don't want to cause internal pain if I can help it.

Hope this helps,

Dan
N/A

Don’t Make Me Wait!

Quote

Hope this helps,


It does help Dan. Wink

So we can expect that when these improvements are unleashed there will be absolutely no problems for the customer because internally they would have been spotted.

In other words the customer is not going to become guinea pigs for BT?
Dan
Grafter
Posts: 121
Registered: 05-04-2007

Don’t Make Me Wait!

Quote
The portal has been running ever since I can remember, so it seems a bit odd to embark upon major project work like this now that Plusnet are within striking distance of being taken over.

Wouldn't it be better to wait and see what the new owners want, rather than run the risk of having the whole thing ditched or changed, which is normally what happens with most Plusnet projects and decisions? Tickets/questions, Comms Team/Voice of the Customer etc.


It's a good question but hopefull answered in part by today's numerous media coverage. BT have practically stated that they wish to keep PlusNet unchanged - for now at least. A significant and hugely positive announcement was that Lee Strafford will remain CEO, and I know that Lee would not want us to change our plans, especially those that aim to improve our customer experience.

Furthermore, I believe that PlusNet must continue with a "business as usual" attitude as the sale is not yet complete. We can't just put things on hold, takeovers can take months to complete.

I hope this answers your question, but please feel free to shoot one back.

Cheers,

Dan
Dan
Grafter
Posts: 121
Registered: 05-04-2007

Don’t Make Me Wait!

Quote

So we can expect that when these improvements are unleashed there will be absolutely no problems for the customer because internally they would have been spotted.


It would be very foolish of me to guarentee that these major upgrades will go problem free, but I can promise and guarentee that we are doing extensive testing and using ourselves as guinea pigs.

Cheers,

Dan
N/A

Don’t Make Me Wait!

Hmm.. well, i believe Mr Dan Kirkland has posted his 'plan' here (wrong forum?) by honest mistake, because most visitors to this forum probably dont understand half of the technical gobbledegook he included in the post..

In any case, the post itself is an embarassing addmission by the PN chief of software design that during many years of operation of his team it has been incapable to even produce and run a decent web portal Smiley

I'm a PHP webdeveloper (obviously not working for PN) and i can tell you that if ALL of his 'planned' measures were never implemented before, then the website (portal) simply cannot be called a 'pro job'.. rather it was (is) being made and run by total idiots who have no clue what they were doing and have absolutely no respect whatsoever for webstandards and best php/db coding practices..

You dont need to believe me, just look at the source of webpages and/or run them through W3C validator - I've done a couple of pages for you Mr Kirkland, here you go Wink
1. http://validator.w3.org/check?uri=http://www.plus.net/
2. http://validator.w3.org/check?uri=http://portal.plus.net/central/forums/viewforum.php?f=2

The webpages are totally non-standard, therefore cannot be even expected to work in a cross-browser fashion.

In case Mr Kirkland was looking for a 'community support' suggestions from the PN community, rather than from professional web developers (not sure what he was look at/for, though), I'd also advise him to include in his plans a deployment of php bytecaching (accelerator) which would be a sensible, if not necessary, measure for a medium/large size php driven website.

Good luck with your 'innovations' (mild sarcasm intended), Mr Kirkland!

Andy
(p.s. corrected a couple of my typos - too early in the morning :shock: )
Dan
Grafter
Posts: 121
Registered: 05-04-2007

Don’t Make Me Wait!

Quote
In any case, the post itself is an embarassing addmission by the PN chief of software design that during many years of operation of his team it has been incapable to even produce and run a decent web portal Smiley


There is no admission here. In fact, I've been very proud of my guys over the last 3 years.

I can't deny there have been issues, some considerable, but I do believe that on the whole we have made massive improvements to the design and implementation.

Quote
I'm a PHP webdeveloper (obviously not working for PN)


We're looking for talented PHP developers, maybe you'd like to share some of your expertise and help us make things even better. And I am not being facetious.

Quote
rather it was (is) being made and run by total idiots who have no clue what they were doing and have absolutely no respect whatsoever for webstandards and best php/db coding practices..


You're right about non-conformance to some of the web standards, but this is largely a legacy issue. We certainly do have coding standards and our new code and web pages do conform to these.

To also pick up on your point about caching, we do use tools to do this and can assure you that we have done for a long time. However, we are always looking at ways we can improve things and this is one of the items on my earlier posting.

Quote
not sure what he was look at/for,


I'm not looking for anything. Just letting people know what we're doing.

I believe that the work we have planned is hugely positive for our Portal users. Hopefully you'll notice the difference and give us some positive feedback. But if you don't see any positive improvements then I deserve any future slating you give me :-)