.htaccess

There are a lot of sites that describe what you can do with .htaccess files. But I've noticed that most of them only deal with one particular use of .htaccess files and make no mention of the other things you can do. With this in mind, since there is a lot of interest in using .htaccess files on ProHosting, I decided to make my own guide to .htaccess files.

This guide may not include everything that you can do with .htaccess files. Some things that the Apache Web Server manual says do not work on ProHosting's servers. I'll only include some of the more the interesting things. If you need more information on what you can do with them, see the Apache documentation (which is a general info page and not all things on it apply to .htaccess).

An .htaccess file (pronounced "H. T. Access") is a special file that provides specific control over the functioning of an Apache web server. The server already has it's basic settings, but using .htaccess files lets you control specific things in specific places, overriding the server's normal instructions. These things only affect people viewing your site with a web browser, not when you connect via FTP.

An .htaccess file is simply a plain text file. Each directory can have a .htaccess file in it, each defining different things. An .htaccess file in one directory affects all the subdirectories below it, unless limited with <FILE> tags or another .htaccess file in a subdirectory that specifically overrides the one above it.

.htaccess files must be spelled in lowercase letters. .HTAccess is not the same thing.

This is not specific to .htaccess files. Any file in Unix that starts with a "." is considered a hidden file. In a normal directory listing, you won't see those files listed, because normally, they are files you don't need to see all the time, but files that need to be there, so Unix hides them.

Some FTP applications have "Show Hidden Files" command to show all the hidden files, but I haven't seen it work right in some. Just because you can't see it, however, doesn't mean the file isn't there.

For WS_FTP: If you use WS_FTP, in the "Session Properties" dialog's "Startup" tab, type "-a" in the "Remote file mask" field.

For Command-line FTP: If you use a command-line FTP (Dos, Unix), type "ls -a" to show hidden files in the directory listing.

As you may have noticed, you can't normally see the .htaccess file in the directory listing in your FTP application (see previous question). If your FTP application does not have a "Show Hidden Files" command (see previous question), or it doesn't work (never seems to for me), you can delete the file doing one of two things.

If your FTP application has a quote command, you can type in commands to the server (same for those using command-line FTP programs). Select the quote command and type (whichever works):

The other way is to just upload a new .htaccess file with nothing in it to overwrite the old one, which works just as well.

Of course, keep in mind this works for any "." hidden files, so be careful what you delete. With Unix-based machines, there may be a number of important hidden files laying around. Just be careful what you delete.

Well, it can't slice bread, but it can sure do a lot of nifty little things to help make your web site a nicer place. The things described in this guide are:

• General Info - some basic info
• Access Control - block people by domain or IP address
• Password Protection - people can't enter without passwords
• Custom Error Pages - point to special pages on errors
• URL Redirection - redirecting from one page to another
• Directory Indexes - allowing listings of files in a directory

An .htaccess file's commands apply to the directory that it is in and all files and subdirectories below it. If you place an .htaccess file in a subdirectory below another .htaccess file, any commands in the subdirectory file will override the same commands from the file in the partent directory.

Within a directory, you can limit the control to affect specific files in a directory by including .htaccess commands between the following tags:

...where FILENAME is a specific file, for example:

Wildcards (*) can be used for filenames, for example:

...will apply to all ".txt" files. Any commands inside the tags applies only to the file(s) the tag defines.

One caveat is that you need a separate set of <FILES> tags for each file you want to limit. It's not supposed to be that way, according to the Apache documentation, but that's how it is at ProHosting.

There are supposed to be <LOCATION> and <DIRECTORY> tags that limit the scope of the commands to a specified directory, but they don't seem to work at ProHosting.

There is another set of tags, <LIMIT>, lets you limit things more generally by limiting by the method that one uses to access a file.

...where METHOD is one or more HTTP commands, separated by spaces, like GET, POST, or HEAD. Limit only affects the specified methods. All other methods will behave normally. <LIMIT> tags can be used inside the above mentioned tags. (If using GET, HEAD is restricted also.) This can sometimes be useful in conjunction with CGIs, but .

You can include comments in .htaccess files by starting the line with a "#"

Access control is controlling who can see what files in your account. You can block or allow access to certain files or whole directories of files in a number of ways, depending on what you need. If you want to limit access with the use of passwords, see the next section.

The main uses of access control is to block or allow only people that are browsing your pages from specific IP addresses, subnets, host names and entire domains.

One reason you might want to do this is if you have someone that won't stop leaving you offensive comments in your guestbook. You can find out the user's IP address or host name by adding the right code to the guestbook script and then use that to block the user from accessing the guestbook form, or the entire site. Another use would be to allow all people that are members of a specific domain, for example, to have an AOL-only site which only allows users of AOL to see it.

Simply define who can view the site or who can't, as in the following example:

The first line tells the server which order to evaluate the directives. For "order allow,deny", any "deny" directives override the "allow" directives. For "order deny,allow", any "allow" directives override the "deny" directives. NOTE: Do not put a space between the allow and deny directives in the order statement.

The second line tells the server that anyone can view the site.

The rest of the lines tell the server who can't view the site.

In this case, anyone can see the site, unless they are coming from the specified addresses or domains in the "deny" directives.

You can hide the entire directory with an .htaccess file that contains this:

Password protecting files and directories is probably the most interesting use for .htaccess files. Password protecting directories is a part of the overall concept of access control. But it offers more specific control over your files, basing access on who is viewing it instead of where they come from.

What password protection does is to make the file or directory (and any files in it), unreadable by someone through a web browser that doesn't have a valid username and password. When the user browses to a file or directory that is protected, the server requires the browser to give it authentication information, e.g. username and password. The browser will pop up a dialog to request this information from the user to send to the browser. If it's accepted, the user can get what he/she wants. If it's rejected, they are given page that tells them so. This is controlled solely by the web server and the browser.

However, since CGIs run on the server directly, they can access a protected directory. This is important because you can use a protected directory to store things like database files and password lists that might be used by CGIs, while hiding them from casual browsers.

The files and directories that are protected still need to have the same permissions that you would give any other file (644) or directory (755).

You should be aware that the browser remembers this authentication information for a site until the browser application is exited. So even if a user views other pages, he/she can always go right back to your site without logging in again. This may not be a problem, per se, but if they leave the browser open, and someone else uses it, then that other person, who perhaps shouldn't be there, can also see your pages. If you have sensitive information on your site, you should point this out to your users. There is no way to force the browser to forget authentication information.

You can protect the directory by granting access only to a specific user, any valid user, or any member of a group of users. The usernames and passwords for users are kept in one file, and the group lists are kept in another file. Using groups is optional, of course.

Here are the steps to password protecting a directory:

1. Create The Directories:

   Of course, you create the directory or directories that you want to protect. But you should create a separate directory to hold the password and groups files. Using FTP, create a new directory, which I'll refer to as "hidden". Create an .htaccess file for the "hidden" directory with the following lines:

   order deny,allow
   deny from all

   ...and upload this into the "hidden" directory. This blocks anyone from seeing the contents of this directory through the web browser. This directory can then be used to store other sensitive data that is used by CGIs. NOTE: Do not put a space between the allow and deny directives in the order statement.

   
   
2. Create The Password File:

   The password file has a list of usernames and associated passwords for the people that may access your directory. It is typically called ".htpasswd", but you can name it anything you like.

   The passwords in the file must be encrypted. The passwords should be encrypted on the system that they'll be used on. To encrypt them, you need a crypt script, which is provided below. Copy the script to a file, upload it to your "cgi-bin" directory, change the permissions to 755 and view the script in your web browser to use it. Similar programs or scripts can be run on Windows or other systems, but they may not produce the proper results.

   #!/usr/bin/perl # crypt.cgi by David Efflandt, last updated 9/04/99 # Crypts a password for use with web authentication use CGI qw/:standard :netscape/; srand( time() ^ ($$ + ($$ << 15))); print header,start_html('Crypt a Password'),"n","n"; if(param()) { $word = param('word'); @range = ('0'..'9','a'..'z','A'..'Z','.','/'); $salt = $range[rand(int($#range)+1)] . $range[rand(int($#range)+1)]; $pass = crypt($word, $salt); print h1("Result of crypt $word"),hr,'A crypted version of ', em($word),' is: ',em($pass),p, 'Example of a line in a password file for web authentication '. '(colon separated):',p,pre("username:$pass"),hr; } print b("Note: "),a({href=>'crypt.txt'},'This Script'), ' works best when run on the system you need passords for because some systems use a more secure ',em('crypt'),' that ends up with more than the usual 13 character crypted password.'; print start_form,center(table({border=>1,cellpadding=>10}, Tr([th([("Crypt a Plain Text Password").br.textfield('word').br.submit]) ]))),end_form,p,end_html;

   Once you encrypt the password, copy the encrypted word, which should be exactly 13 characters long regardless of the password's actual length (see Encrypted passwords explained. below), and paste it in the password file with the username before it on the same line and use a colon (:) to separate the words, like this:

   username1:lP3fUlTTCufWJ
   username2:EGWt1bWgxjFyc
   ...

   After you create this file, upload it to the "hidden" directory.

   
   
3. Create The Groups File: (Optional)

   The groups file contains group names and the usernames of those belonging to the groups. It is typically called ".htgroups", but you can name it anything you like. You can use a groups file, for example, to limit different sets of people from different directories. The file has the following format:

   group1: user1 user2 user3 user4 ...
   group2: user1 user5 user6 ...

   After you create this file, upload them to the "hidden" directory.

   
   
4. Create The .htaccess File:

   The next step is to create the .htaccess file for the directory to be protected. The .htaccess file needs to include the following:

   AuthName "My Site Title"
   AuthType Basic
   AuthUserFile /usr/home1/username/html/hidden/.htpasswd
   AuthGroupFile /usr/home1/username/html/hidden/.htgroups
   
   require user user1 user2 ...
        and/or
   require group group1 group2 ...
        or
   require valid-user
   

   The meaning of each line being:

   • AuthName is the name that appears in the login prompt that the user will see.
   • AuthType must be "Basic" (there are other types, but they are not supported).
   • AuthUserFile is the path to the password file.
   • AuthGroupFile is the path to the groups file, if using groups.
   • require user username allows access to only the specified user(s).
   • require group groupname allows access to any properly authenticated user that is a member of the specified group(s).
   • require valid-user allows access to any properly authenticated user in the password file. This options overrides use of user or group.

   You can use <LIMIT> tags around the "require" statements to control access through specific methods, but the other methods will still be open. I mention this again because a lot of sites say to use them, but they aren't necessary.

   After you create this file, upload it to the directory to be protected.

   
   
5. Done:

   At this point you are done, and assuming you've followed the steps correctly, your files and directories are now protected from unwanted viewers. If you want to let people sign up automatically, you can create a script to add users to the password file (and the groups file, if necessary). The password encryption script at this page contains a link to the script's source, which you can take the encrypting part from to use in your script.

Normally, if there's an error, like a page not found or a script that didn't work right, you receive a generic error page. These have been customized for ProHosting, but perhaps you'd rather have it display a different page that looks more like it fits with your site or has your contact information on it so people can tell you about the problem.

The first step is to create your error pages. These are nothing more then plain old html pages with whatever images and information you want them to have. I'd suggest making a directory called "errors" in your "html" directory and name the files with the error code numbers that they represent. Click here for a list of common error codes: HTTP error codes.

Then add lines to your .htaccess file for each error code that you have a custom page for. The format is like this:

For example, a 404 error means that the page was not found:

While you can use a full URL (http://www.domain.com/...), I wouldn't recommend doing so because of the way that full URLs are processed, especially if redirecting to a URL on the same server that ends up generating the same error.

You do not need to create error pages for all error codes, only the ones you need. Most of them are rarely, if ever, used. Any error codes that you do not specify will use the server's default error pages, if they occur.

You can redirect URLs using a CGI, or using a META Refresh tag in the header of an HTML page. But you can also do so from an .htaccess file. Why? It's faster for the server to do the redirect then to run a CGI or load a web page with a META Refresh tag in it, which some browsers may not support.

Why redirect? Suppose you have an old page that's been around for a while, and maybe it's in a few search engines. But you want to change the structure of your site and rename some pages. Instead of worrying about people coming to your site from search engines or old bookmarks, you can put a line in a .htaccess file for each page you want to redirect users from. The format is like this:

For example:

Now when the user goes to your site to the "mypage.html" page, the server automatically redirects them to the new page. Also, the old page, "mypage.html", does not actually have to exist, so you can remove old pages and save account space.

You can redirect entire directories by only specifying the old directory name and the new directory name in each URL. The filename that is entered is appended to the redirected URL, as needed. For example:

Then, someone browsing http://thunder.prohosting.com/~username will be redirected to http://thunder.prohosting.com/~username/otherdir.

It used to be that if you didn't specify the name of a file in a URL, the web server would simply respond with a list of the contents of that directory. One day someone decided this wasn't very secure, since anyone could see what was in the directory. Now most web servers come with an option to disable directory indexes, and most servers, including ProHosting's, have this option set.

As you may have noticed, if you don't specify a specific file in the URL, the server will return a file called "index.html" automatically, or if not found, an error page. But sometimes, you want to be able to let someone get a list of what's in the directory. You can create a CGI script to read the directory's contents and display it with SSI, but that can be a lot of work, and is relatively slow. This may be necessary, however, if you don't like the way the server generates the index. You can go to David's page and get the file called "dirlist.txt", which is a Perl script that will create an index "on the fly".

Otherwise, you can use an .htaccess file to tell the server to create a directory index instead.

If the directory contains an "index.html", "index.htm" or "index.shtml" file, the server will return that file instead of a directory index.

Here's how to do it:

1. Enable Indexes:

   The first thing is to enable the directory indexing. Do this by adding the following line to the .htaccess file:

   Options Indexes

   This is usually done in an .htaccess file in your "html" directory. You can override this in subdirectories with an .htaccess file in them with this line:

   Options -Indexes
   
   
2. Custom Header: (Optional)

   You can create a header file to be added to your indexes. This is an optional item, but using it allows you to make the index reflect the design of your site. The header file is a HTML file or plain text file which is included at the top of the index page. Add the following line to your .htaccess file after the "Options Indexes" line:

   HeaderName FILENAME

   ...where FILENAME is the name of the file without the extension. For example, if your header file is called INDEX_HEAD.html, the line should read:

   HeaderName INDEX_HEAD.html
   
   
3. Ignore List: (Optional)

   You may want to have the index not list certain files or types of files. For example, you might want to have the index not show the .htaccess file itself. You can add an IndexIgnore line, which tells the server which files not to include in the index. Simply list them all separated with spaces in this format:

   IndexIgnore FILE FILE

   ...where FILE is a whole file name or wild card expression (such as "*.gif" to hide all GIF files). For example, to not display the ".htaccess" file, the index header file or style sheet files:

   IndexIgnore .htaccess INDEX_HEAD.html *.css

   This doesn't prevent someone from accessing those files, it only keeps them from being listed in the index.

   
   
4. Index Options: (Optional)

   There are a number of options you can use to affect how the list of files in the index are displayed. In the .htaccess file, use the line:

   IndexOptions +Option1 +Option2 +Option3 ...

   Here are the options you can use, which I've listed in order of what I think are most to least useful:

   • FancyIndexing -- Enables fancy indexing, which many of these options affect.
   • SuppressHTMLPreamble -- Normally, when using a header file, it doesn't need the <HTML>, <HEAD>, etc. tags, the index creates them. This option makes it so the header file needs those tags specified. Use this for HTML header files.
   • FoldersFirst -- For fancy indexing, lists folders separately from files.
   • SuppressDescription -- For fancy indexing, disables printing of files' descriptions.
   • SuppressLastModified -- For fancy indexing, disables printing of files' last modified date.
   • SuppressSize -- For fancy indexing, disables printing of files' size.
   • IconsAreLinks -- Makes the icons part of the link for fancy indexing.
   • IconWidth=pixels -- Icon width in pixels.
   • IconHeight=pixels -- Icon height in pixels.
   • NameWidth=n -- The width of the filename column, or * for automatic sizing.
   • DescriptionWidth=n -- The width of the description column, or * for automatic sizing.
   • ScanHTMLTitles -- For fancy indexing, this uses the string in the TITLE tag of HTML pages for the file's description if a separate description is not set. This will make it slower.
   • SuppressColumnSorting -- For fancy indexing, disables the column headings from becoming "sort by" links. Normally, selecting a column heading will sort the index by that column's values.

   For example (you will probably always want to use the FancyIndexing option):

   IndexOptions +FancyIndexing +SuppressHTMLPreamble +SuppressSize

   I should point out that I haven't tried all the options, so some may not work. If you find one that doesn't, please let me know.

   
   
5. Customize Icons: (Optional)

   In FancyIndexing mode, the index uses icons next to the file names in the list. They help the viewer to quickly identify the type of file it is. You can supply your own icons to use, which can be any image file that can be viewed in web browsers.

   -------
   IMPORTANT NOTE:

   You should make all the images the same size, and use the IconWidth and IconHeight options for the IndexOptions. I recommend them to be the same size because you can only define one IconWidth and IconHeight which will apply to all the icons displayed.

   -------
   

   To change the icons that are displayed, use one of these options, which are lines in the .htaccess file:

   AddIcon (ALT,ICON) NAME

   ALT is alternate text if the icon isn't found (keep it short). ICON is the URL path to the image file to use. NAME is either a file extension or complete filename, or use ^^DIRECTORY^^ for directories. Or use:

   AddIconByType (ALT,ICON) MIME-TYPE

   ALT is alternate text if the icon isn't found (keep it short). ICON the URL path to the image file to use. Each MIME-TYPE is the type of a particular file, e.g. "image/*" for any image types or "audio/wav" for WAV files, etc.

   There is a third option, AddIconByEncoding, but I don't list it here because it's too complicated to bother with.

   Any unspecified types will use the servers default icons, unless you add the line:

   DefaultIcon ICON

   ...in which ICON is the path to the image to use for any unspecified file types.

   Examples:
   
   AddIcon (IMG,/~username/icons/images.gif) .gif
   AddIcon (DIR,/~username/icons/folder.gif) ^^DIRECTORY^^
   AddIconByType (AUD,/~username/icons/audio.gif) audio/*
   DefaultIcon /~username/icons/def.gif
   
   
   
6. Adding File Descriptions: (Optional)

   You should be able to add descriptions for files, which would have to be for a specific file, but I couldn't get it to work. If anyone has figured it out, let me know. The format should be:

   AddDescription STRING FILE

   ...where STRING is the description text (keep it short), and FILE is the file to associate the description with. For example:

   AddDescription "rose picture" /usr/home1/username/html/images/rose.gif ...

AddType assigns a mime-type to a file extension. If the server doesn't have a mime-type defined for a file extension, or has a different mime-type for the file you're using then what it needs to be, then you should be able to override that with an AddType line. The format is:

For example, if MP3 files and ".shtml" files weren't supported:

The default type is usually "application/octet-stream" (I think), which will make the browser not display the file properly. If you don't know the mime-type for a file, you'll have to find it out elsewhere.

For more information on mime-types and what they are, check out The MIME Information Page. The simple explanation is that the mime-type tells the browser what type of data a particular file contains, which the browser uses to determine if it can display the file or needs to load a plug-in or pass the file to a helper application.

AddHandler maps the filename's extension to the handler. The format is:

There are two specific uses for AddHandler at ProHosting.

1. Direct the server to use ".exe" files in the "cgi-bin" as a CGI. For example:
   
   AddHandler cgi-script .exe

   ...will make ".exe" files work like CGIs.

   
   
2. Direct the server to parse a different file extension for SSI directives (other than ".shtml", which should already be parsed). For example:
   
   AddHandler server-parsed .html

   ...will make regular ".html" files work like ".shtml" files.

I don't know what other "handlers" are available at ProHosting.

If you know of anything else for .htaccess that works at ProHosting, e-mail me.

If you've encrypted passwords for .htaccess, you may have noticed that they always result in a 13-character string, regardless of the length of the password. You may have also noticed that encrypting the same password more then once will produce a different 13-character string each time you run it. While this certainly seems weird, it is correct. It has to do with the way the Unix "crypt()" function works.

Unix, which ProHosting runs a version of, has a crypt command that can encrypt text with an algorithm to create what is called a "hash". A hash is a string created from another string, made by performing a series of mathematical calculations with the original string, such that the result is irreversible. This means that the encrypted password cannot be turned back into the original word.

For some gory details on how the passwords are encrypted, read this excerpt from the Unix Encryption Tutorial:

The password you first enter is used as a key to encrypt a block of 64 NULL characters. The first seven bits of each character are then extracted to form a 56 bit key (this means that only the first eight characters of a password are used on a standard unix system). The hash table is then modified using a twelve bit salt (making it harder for today's faster equipment to crack by making the search space 4096 times bigger for each character). The DES encryption algorithm is then run 25 times on it. The resulting 64 bit hash is then forced into a 64 character alphabet (0-9, A-Z, a-z, "/", "."). This involves translations in which several different values are represented by the same character, which is why Unix passwords cannot be decrypted. Other algorithms used to hash passwords are MD5, and bigcrypt.

This doesn't really explain why the encrypted password is different each time the same word is encrypted. It has to do with the "salt" that is used in the encryption process, which adds a bit of randomness to the passwords. The salt is included as part of the encrypted password. When a user logs in, the password entered is encrypted using the same salt as found in the stored password, and the result, if the correct password is entered, will be the same encoded password.

I've set up a few of the .htaccess possibilities defined above as a brief sample to demonstrate them. Password protection for a directory, directory indexes, URL redirection for a specific file and a custom error page are used. Try the links below to see them in action.

1. Password Protection & Directory Indexes: http://thunder.prohosting.com/~sampieri/freefaq/test
   

   This directory is set up to keep people out without knowing the password, but once you're in, you see a directory listing (which uses custom icons) since there's no "index.html" file.

   Username: test
   Password: openup
   
   
2. URL Redirection: http://thunder.prohosting.com/~sampieri/freefaq/direct.html

   I don't want someone to see "direct.html", so I'll send you to "redirect.html".

   
   
3. Custome Error Page: http://thunder.prohosting.com/~sampieri/freefaq/blah.html

   "blah.html" doesn't actually exist, so you get a custom error page instead of the default ProHosting one.

For your reference, here are the complete set of .htaccess and password files used in these samples.

Here is the complete .htaccess file used in the Password Protection and Directory Indexes sample above.

This is the .htaccess used in the "hidden" directory, which contains the ".htpasswd" file.

This is the contents of the .htpasswd used in the examples on my site.

This is the .htaccess used in the "freefaq" directory for the URL Redirection and Custom Error Page samples above.