Summary


In order to drop the JED index.html requirement, .htaccess and web.config files should be adjusted to prohibit directory traversal by default.
Test Instructions for Apache Servers:
Apache Server:
1. Go to Global Configurations
2. Use URL Rewriting -> Set to Yes
3. In the root of your File Manager, set htaccess.txt to .htaccess (as Brian mentioned below).
4. Delete an index.html file within a directory and try to make a direct access.
5. Apply Michael's patch and try to access the directory again.
Local Apache Server:
1. Open your httpd.conf file, uncomment this line:
LoadModule rewrite_module modules/mod_rewrite.so
2. Within the httpd.conf file, also find the <directory /></directory> area and change this line:
AllowOverride None to AllowOverride All
3. Restart Apache Server
4. Go to Global Configurations
5. Use URL Rewriting -> Set to Yes
6. In the root of your File Manager, set htaccess.txt to .htaccess (as Brian mentioned below).
7. Delete an index.html file within a directory and try to make a direct access.
8. Apply Michael's patch and try to access the directory again.
Results: Patch prohibits direct access to all directories.
Opened On:
2 Mar 2013, 22:39 by Tessa Mero
Closed On:
24 Aug 2014, 15:25
Status:
Closed

Responses

Posted on 3 Mar 2013, 15:10 by Michael Babker
Try the patch and instructions listed above to test this. .htaccess should be good to go, I don't have an IIS environment to verify the web.config change, so I'm depending on my Google-fu for this one.
Posted on 3 Mar 2013, 15:11 by Michael Babker
Also note that this should be applied to 2.5 as well as 3.1.
Posted on 3 Mar 2013, 15:28 by Brian Teeman
Please note this will have no effect unless the htaccess.txt file is renamed on the live site. This is unlikely to happen on upgraded sites so will require publicity of the change
Posted on 3 Mar 2013, 15:59 by Tessa Mero
@test

Tested the htaccess on apache server, successful.

Would like someone to test changes on Windows server as well.

Also updated test instructions in the Details.
Posted on 3 Mar 2013, 16:05 by Emerson Rocha Luiz
Tested with "Server: Apache/2.2.21 (Win32) PHP/5.3.14".

Returns "Index of /folder/tmp" with no content for http://joomlasite/folder/tmp instead of list files and folders.
Posted on 3 Mar 2013, 16:58 by Michael Babker
I went back and tweaked the htaccess a bit more so that instead of displaying the "Index of" message, you'll get a 403 "You don't have permission to access /CMS/my_folder/ on this server." message.
Posted on 3 Mar 2013, 17:26 by Tessa Mero
Even better, thanks Michael.
Posted on 4 Mar 2013, 3:21 by Jean-Marie Simonet
@test

Followed test instruction for local Apache.
I do get for example a "You don't have permission to access /trunkgitnew/templates/ on this server."
Therefore this works OK here

Shall we not keep the index.html requirements for people not using SEF?
Posted on 4 Mar 2013, 5:44 by Michael Babker
There's other ways that you can prevent directory listing aside from the .htaccess file. Also, we could look at the possibility of automatically moving the htaccess.txt or web.config.txt files to the appropriate name as Mark has done with the proposed robots.txt change at http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=29677.
Posted on 4 Mar 2013, 19:30 by Mark Dexter
I believe the default for Use URL rewriting is No. So I'm concerned that this approach will not be set up for new installs by default. Also, do we know that Windows IIS has the same capability? Should we default this to Yes? Should we force this to Yes and remove the option?

It seems that if we no longer have index.html files, then we need a way to ensure that all users will have this set up correctly. Does that make sense? Thanks.
Posted on 5 Mar 2013, 1:16 by Jean-Marie Simonet
I think we should definitely keep the index.html files as a requirement and nevertheless implement this solution to prevent eventual errors by 3pd, or users creating specific folders manually, knowing that, when URL rewriting is Off, it will not change the situation for these.
I would keep URL rewriting default to No as some users have issues on their hosts with .htaccess and keep index.php in their urls.
Basically, although this solution will not help everybody, I see it as an improvement for the majority.

@Michael: If there are other ways to prevent directory listing, are they complex to implement?
Posted on 6 Mar 2013, 12:08 by Radek Suski
I'm sorry Jean-Marie but I respectfully disagree.
The requirement to keep index.html inside every folder not only doesn't protect anything but causing also some problems.

Maybe instead of trying to disable indexes we should try to prevent access to particular type of files.
There is already a working solution in the docs.joomla.org AFAIR.

I'm using usually something like this:

RewriteCond %{REQUEST_FILENAME} -f
RewriteCond %{REQUEST_URI} \.php|\.ini|\.xml [NC]
RewriteCond %{REQUEST_URI} \/components\/ [OR]
RewriteCond %{REQUEST_URI} ^\/includes\/|^\/administrator\/includes\/ [OR]
RewriteCond %{REQUEST_URI} \/language\/ [OR]
RewriteCond %{REQUEST_URI} \/libraries\/ [OR]
RewriteCond %{REQUEST_URI} \/modules\/ [OR]
RewriteCond %{REQUEST_URI} \/templates\/
RewriteRule ^(.*)$ index.php [R=404,L]
Posted on 1 Apr 2013, 5:44 by Matias Griese
If you want to keep index.html files, there's easy solution to do that. Every time folder is created by using Joomla APIs, it should automatically add the file.

But that said, index.html files are real issue and should go away. If hosting companies don't care about their security, it's not Joomla's fault.

There could be a warning message if .htaccess or web.config files are missing. BTW, there's yet another issue where you can prevent server from looking up .htaccess files, but those servers are probably configured already for safety.
Posted on 1 Apr 2013, 5:49 by Michael Babker
Personally, I think the changes to .htaccess and web.config should be accepted, even if the CMS has no intention of dropping the index.html files anytime soon. With or without the index.html discussion, this change simply hardens the security features provided by enabling our default .htaccess and web.config files, and we should look at the patch as just that. FWIW, the JED did follow through and drop its index.html requirement.
Posted on 2 Apr 2013, 4:25 by Matias Griese
I see two options in here:

1) Drop index.html files and implement .htaccess changes
2) Keep index.html files, implement .htaccess changes and change folder creation to add empty index.html file.

I would prefer 1), but second option will keep the "security expert" users from complaining extension people from ignoring security (which is not true)...

Another thing: can we prevent access to the manifest files? They reveal too much information on which extension versions have been installed to the server.
Posted on 2 Apr 2013, 4:40 by Brian Teeman
Just remember that not all sites have the htaccess or webconfig enabled as we cannot set this to on by default
Posted on 3 Apr 2013, 4:25 by Matias Griese
How about calling this after mkdir calls in JFolder::create($path, $mode, $index = true):

class JFolder
{
public static function createIndexHtml($folder)
{
// Make sure we have an index.html file in the current folder
if (!is_file($folder.'/index.html'))
{
$contents = '<html><body></body></html>';
JFile::write($folder.'/index.html', $contents);
}
}
}
Posted on 3 Apr 2013, 4:43 by Brian Teeman
when would that be used Matias? when you create a folder with media manager a blank index.html is already created
Posted on 3 Apr 2013, 6:08 by Matias Griese
For example extension installer would automatically add index.html to all folders it creates (now that JED doesn't have the requirement anymore). Index would be created also whenever someone calls JFolder::create(), so all directories would automatically have the file. Often extensions forget to add the file when dynamically creating directories under /media or /images.

But that said... Should we really keep using index.html files as they are only useful when hiding user created files from public (media and images) and perhaps also cache, logs and parent folders for installable extensions?

Attackers are not using file listings to attack the targets. They either attack blindly or try to load files in known locations (for example manifest files). Both ways can give you exact information of which extensions have been installed and even precise version information which can be used to locate every installed file without ever seeing directory listing.

So index.html files serve no real purpose in most directories. They just provide false sense of security except when the files inside protected directories are custom only to the current installation.
Posted on 4 Apr 2013, 7:59 by Brad Gies
The JED requirement for index.html files needs to be kept. What percentage of websites use .htaccess files? It's definitely not 100%. I have about 30 Joomla websites and I don't use .htaccess on even one of them.
Posted on 4 Apr 2013, 8:02 by Brian Teeman
Brad that decisions is made after a period of notice and discussion.

I'm willing to bet that you dont have a single web site running on a web server that will display a list of the files inside a folder. (Thats what the rule existed to prevent)
Posted on 4 Apr 2013, 11:11 by Matias Griese
Hmm.. Actually that last comment changes my mind: It's not the task of site administrator to protect folders from displaying file listings. Every server should come by default with those protective settings turned on! So it's really responsibility of Linux distributions, control panel software software and VPS providers to have these basic protective settings turned on by default.

So instad of all of this I would do the same as Radek Suski in his comment -- to provide some extra security for those who do enable .htaccess.
Posted on 12 Aug 2013, 13:00 by George Wilson

BUMP. @test works like a charm. I think we still should be implementing this in the default .htaccess for those who choose to use it - even if we keep index.html files otherwise.
Posted on 6 Sep 2013, 8:31 by Thomas Hunziker

I would also say to add this to the default settings, regardless if we drop index.html or not. It's a good move anyway.

However I wonder why we even need the +IndexIgnore * and the All part in the Options. Imho all that is needed is to block indexing is using:

Options +FollowSymLinks -Indexes

That should do fine.

Or do I miss something here?
Posted on 6 Sep 2013, 9:03 by Michael Babker

I had to go through my comments here to get the answer for that. As posted back in March: "instead of displaying the "Index of" message, you'll get a 403 "You don't have permission to access /CMS/my_folder/ on this server." message."
Posted on 8 Sep 2013, 2:07 by Thomas Hunziker
Yep, that is what the "Options -Indexes" part does. It Shows the 403 error.
Imho the other line "+IndexIgnore *" is not needed, as it is meant to restrict files from being listed, not restricting the listing itself.
Also I'm not sure if the "All" part in the Options line is needed. Imho it is not.
Posted on 8 Sep 2013, 11:33 by Michael Babker

I'm actually not 100% certain myself.  I just copied both lines out of my Admin Tools generated .htaccess file in all honesty.
Posted on 8 Sep 2013, 12:21 by Thomas Hunziker

Maybe we have to ask Nicholas then why he did it this way :)
Posted on 8 Sep 2013, 13:06 by Thomas Hunziker

I think it just dawned on me why it makes sense to have both lines: Some hosting providers restrict the use from the Options directive in the .htaccess file. So including the "IndexIgnore *" would provide kind of a fallback solution in this case.

I'm still not sure about "Options +FollowSymLinks All -Indexes". "Options All" is the default setting which would include "FollowSymLinks". So either "Options All -Indexes" or "Options +FollowSymLinks -Indexes" would make sense here.

 
Posted on 11 Sep 2013, 3:19 by Elin Waring

It is very common on shared hosting to have to add the joomla .htaccess code to the host's .htaccess file. So I would not default copy .htaccess since that may not work or may be wiped out anyway.
Posted on 11 Sep 2013, 10:52 by Brian Teeman

That would indeed be a very bad idea Elin but I don't see that in this proposal anywhere or am I missing something
Posted on 11 Sep 2013, 11:49 by Nicholas Dionysopoulos

Thomas, the "Options" directive is often blocked (or at least some of the options are blocked). Most usually Options All is blocked so it's bad form to use it. I would use two lines:

Options +FollowSymlinks
Options -Indexes

and have the users try to disable them separately, as maybe one of the two is not allowed. Using the +/- in front of an option acts as a modifier.

Elin, regarding the merge of .htaccess files, it's usually one or two directives required on badly setup hosting to enable PHP 5.3 or later. These shouldn't be in a default .htaccess on an empty hosting space. I'd rather Joomla! to ship with a .htaccess file (and not htaccess.txt). I've seen scores of people who think that htaccess.txt is actually doing something. I usually find out about it the hard way, doing support: I ask them to add something to their .htaccess, they say they did but there is no change. A couple of frustrating posts later I figure out it's htaccess.txt they modified, not .htaccess. After all, we do ship a web.config file, not a web.config.txt one.
Posted on 23 Aug 2014, 8:14 by Brian Teeman

Michael are you going to create a Pull Request for this?
Posted on 24 Aug 2014, 15:25 by Brian Teeman

Closed please see http://issues.joomla.org/tracker/joomla-cms/4171