Website access denied [.htaccess and hence index.shtml ignored]

[#1]

This is one for which I have just raised a ticket, but others might have the same problem. 

From today 2nd March, access to my numerous sites (on the 'home pages' servers) has been unpredictable and generally unavailable.

All sites have home page index files of index.shtml (because they use server-side includes) but act as if the index files were index.html. 

The .htaccess files themselves (which define the use of .shtml) appear OK, but are apparently being ignored by Apache! 

[#2]

The problem occurs on Forec9 homepages for me too. It looks like SSI commands are not being recognised and parsed.

It's not just index.shtml that has the problem, anything.shtml does too. However the Apache AddType and DirectoryIndex directives in .htaccess must be being processed.

[#3]

Thanks spraxyt.

Not sure about "...Apache AddType and DirectoryIndex directives in .htaccess must be being processed." however, at least for my sites.

My .htaccess files all read:

AddType text/html .shtml
AddHandler server-parsed .shtml
DirectoryIndex index.shtml

but if http://www.somewebsite.com on PN home pages servers has an index.shtml and has no index.html, access is being bounced with a Forbidden - you do not have permission....

If if the site does also have an index.html (as well as index.shtml) in the root directory, it loads that instead.

But I can access the site correctly longhand using http://www.somewebsite.com/index.shtml

Apache isn't getting as far as trying to parse the <!--#include virtual="someurl" -->  -  it isn't even loading the right page!

[#4]

My .htaccess has the first two of those lines, but the third is

DirectoryIndex index.shtml index.htm index.html

However my pages are working again now - though a page refresh (Ctrl-F5) is needed to reload it.

[#5]

CSC tell me that in response to my ticket they have "restored the sites' permissions" - which this (Fri) morning made no difference. The problem appears clearer by the minute and is always replicable: Apache is ignoring or misinterpreting my .htaccess directives (which have been unchanged for years).

It is not a permissions problem per se: when told to find the index page Apache goes looking for index.html, doesn't find it, and replies with a 403 ("smacky wrist -  stop trying to list the root directory") rather than a 404 (which it would have done if I had been trying to load a page other than an index page).   

[#6]

I am also seeing this issue.  SSIs not always working.  (Page top and tail menus not displayed)

Sometimes also get:

Forbidden You don't have permission to access / on this server. (no message number on page for 403 on tap heading)

Raiding a ticket. 

[#7]

This "now it works and now it doesn't" behaviour has already been seen by PN's CSC who are investigating.

Because in my case either my index.shtml page is loaded successfully or Apache tries to load index.html (which isn't there, hence a 403 response), one obvious possible cause is one of the Apache server-farm servers having SSI disabled; such a server would indeed read a .htaccess with SSI-based operands and simply ignore them. This particular server presumably doesn't have a Includes (or Includes NOExec) or the AllowOverride optional directives in its configuration.     

[#8]

Could you put in a simple redirect within a plain index.html file? Not particularly elegant but may get around the issue you're seeing.

[#9]

Tnx. However I also want SSI #includes to work in the .shml files, and as others have commented this is hit and miss at present.

[#10]

I set up test pages in a special folder to get  a better idea of what was going on. The folder contains both index.shtml and index.html files, and the printed output identifies which is loaded.

As mentioned as least one of the servers in the farm ignores my .htaccess DirectoryIndex entry and loads index.html even though index.shtml is listed first. However if index.shtml is specifically requested it displays but the SSI instructions in the file are ignored. Refreshing the page generally gives a working server which displays the SSI information.

Unfortunately I don't think there's a way to identify the hostname of the Homepages server delivering the page.

[#11]

Absolutely spot on!

Since the problem has been extant for at least four days, the quickest way to identify which Apache instance is badly configured is to put each instance offline, execute a few accesses (or ask one of us to do so - CSC's own access may be proxied or cached), and put the instance back up again.

Brute force and 'unprofessional' perhaps, but home-pages server pages are not, by definition, doing anything vital such as executing PHP or directly updating a database. 

[#12]

For those with a similar problem, it has (on 8th March) been placed 'on hold' by PN for a week... 

[#13]

I suspect topic https://community.plus.net/t5/Everything-else/Forbidden-on-website-folders-that-do-not-have-an-index... might reflect a different manifestation of this problem.

[#14]

I'd hoped @bobpullen would have seen this topic by now and had the issue investigated!

[#15]

I'm under the impression that some work was done a couple of nights ago in a bid to resolve this. Looks like there are still some issues though based on recent replies and the thread here.

We're looking into it...

Edit: ref 98399