.htaccess: A Complete Guide for Apache

You drop a .htaccess into the site root with a redirect to HTTPS, and instead of the site you get an «Internal Server Error». Or the other way around: the file has been sitting on the server for a week and nothing happens. Both situations come down to the same thing. .htaccess works only under Apache, only with particular server settings, and only with the right modules loaded.

What .htaccess is and when it applies

.htaccess is an Apache configuration file read while a request to a directory is being handled, and it applies to that directory and everything below it. It exists precisely for the case where you have no access to the main server file httpd.conf: virtual hosting, ordinary shared hosting, control panels such as cPanel.

It matters not to mix up server technologies. Apache and compatible servers read this file, LiteSpeed among them. Nginx ignores it entirely: its syntax is different, rules are written in server and location blocks, and a file with the same name just sits there doing nothing. If your site runs on Nginx, redirects and headers are configured in the Nginx configuration, not in .htaccess.

A second limitation follows from how Apache is set up by default: it does not look at these files at all. The AllowOverride directive in the server configuration decides this, and its default value is None. Under that setting .htaccess is ignored completely, and no amount of editing it will change anything.

Performance deserves a paragraph of its own. The main server configuration is read once at startup. Files named .htaccess are checked on every request, and Apache walks up the directory tree looking for them at each level until it reaches the document root. For a page served from /www/htdocs/example that means checking /.htaccess, /www/.htaccess, /www/htdocs/.htaccess and /www/htdocs/example/.htaccess. Even when no such files exist, the tree walk and the extra filesystem lookups remain. If you can edit httpd.conf, writing rules there is the better plan.

Where the file goes and how inheritance works

The file is called .htaccess and sits in the directory it applies to. It can be renamed if httpd.conf sets the AccessFileName directive, but the default name is fixed.

Rules are inherited from the top down. A file in the document root applies to the whole site, and a file in a subfolder adds its rules on top of the parent ones and may override them. Some directives are replaced outright rather than extended. The classic example is Options, which in a child directory replaces the parent value completely instead of adding to it.

/.htaccess                    rules for the whole server
/www/.htaccess                rules for /www
/www/htdocs/.htaccess         rules for the whole site
/www/htdocs/example/.htaccess rules for the example section

A request to /www/htdocs/example/page.html goes through all four files.

There is a simple trick for checking whether the server reads these files at all. Put a deliberately nonexistent directive into a directory and look at how the server reacts.

# drop this into a temporary file and request any page
TestMe

A 500 response and a log entry → .htaccess is read, the directive is unknown
An ordinary 200 response     → AllowOverride None, the file is ignored entirely

AllowOverride: without it the file does nothing

httpd.conf contains a Directory block, and inside it you say which categories of directives may be overridden from .htaccess. The default value of None means the files are not read at all.

<Directory "/var/www/example">
    AllowOverride FileInfo AuthConfig Indexes
    AllowOverrideList Redirect RedirectMatch RewriteEngine RewriteRule RewriteCond
</Directory>

The permitted categories work like this. Options permits the Options directive. FileInfo opens up AddType, AddHandler and SetHandler. AuthConfig allows authorization settings to be changed. Indexes permits DirectoryIndex. All permits everything at once, and that is the least safe option. Finer control is possible through AllowOverrideList, where individual directives are named: anything not on the list produces a server error.

RewriteEngine and RewriteRule: the main working mechanism

The mod_rewrite module handles address rewriting and is used in .htaccess all the time. It works like this: the engine is switched on with RewriteEngine On, the rules are described with RewriteRule, and where needed, conditions are checked with RewriteCond before the main rule runs.

A rule consists of a pattern, a target and flags. The pattern is a regular expression, the target is an address or a path, and the flags add behaviour. The most common ones: L stops processing the current set of rules, R issues a redirect with the given code, F returns 403, and QSA appends the original query string to the new address.

# switch the engine on (required)
RewriteEngine On

# 301 from any address to the homepage, except / itself
RewriteRule ^$ /index.html [L]

# redirect from an old path to a new one
RewriteRule ^old-page/(.*)$ /new-page/$1 [R=301,L]

# 404 for anything that is not an existing file
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /404.html [L]

There is a subtlety that catches people out more than any other. Inside .htaccess rules operate in the context of the current directory: the prefix matching that directory is stripped off the address. So a pattern written as ^/images/(.+).jpg in httpd.conf becomes ^images/(.+).jpg in an .htaccess at the site root, and in an images/ subdirectory it is simply ^(.+).jpg. A related detail: regular expressions in .htaccess are recompiled on every request, while in the main configuration they are compiled once and cached. To take an address apart and see which path the rule will match, the URL parser helps.

Order of execution is worth spelling out. The RewriteCond conditions are checked first, and every condition in the group has to hold. Then the RewriteRule applies. If a condition fails, the rule is skipped and Apache carries on through the configuration.

RewriteBase is what you cannot do without in a subdirectory that uses relative addresses. The directive sets the base URL that substitutions are anchored to. Without it, a rule that rewrites an address into a new path will send visitors nowhere when the file does not sit at the root.

Redirect, ErrorDocument, DirectoryIndex and friends

Not everything in .htaccess goes through mod_rewrite. A number of directives work on their own, and for many jobs they are the better tool.

• Redirect permanent /old /new issues a permanent redirect, while Redirect 302 /temporary /target is a temporary one. It works at the Apache module level and needs no mod_rewrite;
• RedirectMatch sets the address pattern with a regular expression, which suits bulk redirects across whole sections;
• ErrorDocument 404 /errors/404.html replaces the default error page with your own. It works for 400, 401, 403, 404, 500 and other codes;
• DirectoryIndex index.php index.html sets the order in which index files are looked for when a directory is requested;
• AddType application/json .json binds a content type to an extension the server would otherwise serve incorrectly;
• Options -Indexes stops directory listings, while Options +FollowSymLinks permits symbolic links;
• SetEnv and the ENV flag on a rewrite rule pass environment variables further down the processing chain.

ErrorDocument 404 /404.html
ErrorDocument 500 /500.html

Options -Indexes
Options +FollowSymLinks

AddType application/json .json
AddType image/svg+xml .svg

<IfModule mod_deflate.c>
  AddOutputFilterByType DEFLATE text/html text/css application/javascript application/json
</IfModule>

Blocks like <IfModule mod_rewrite.c> should always be added. If the module is not loaded, the contents of the block are simply skipped and the site keeps working. Without that protection, a single directive from a missing module produces a 500 error on every request to that directory. The content types that AddType assigns are worth checking against the defaults first: the MIME type reference shows what the server serves for a given extension out of the box.

Folder access: Require, Deny and passwords

There are several ways to shut a directory off to outsiders, and the differences between them matter.

In Apache 2.4 access is governed by the Require directive, which replaced the Order, Allow and Deny trio of version 2.2. Require all denied shuts everything, Require all granted opens everything, and Require ip restricts by address, including subnets and ranges. When migrating a 2.2 configuration, Order deny,allow with Allow from 192.168.1.0/24 becomes Require ip 192.168.1.0/24, and Deny from all paired with Allow from 192.168.1.0/24 becomes Require ip 192.168.1.0/24 as well.

# close off by IP (Apache 2.4)
<Files "secret.txt">
    Require ip 192.168.1.0/24
</Files>

# close off with a password (htpasswd)
AuthType Basic
AuthName "Restricted area"
AuthUserFile /var/www/.htpasswd
Require valid-user

Password protection needs a credentials file, created with the htpasswd utility. The passwords inside are stored as hashes rather than plain text, but the Basic scheme itself sends the password over the wire, so such protection only makes sense behind HTTPS.

With the older Allow and Deny directives the order of the lines decides the logic, and that is a permanent source of confusion. The line Order deny,allow means denials are checked first and permissions second, so Deny from all overrides Allow from all when it comes first. Require has no such ambiguity.

Headers, cache lifetimes and php_value

The mod_headers module manages server responses, and it is where the settings that would otherwise need code live.

<IfModule mod_headers.c>
  # force HTTPS
  Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains"

  # no caching for the API
  <FilesMatch "\.json$">
    Header set Cache-Control "no-store"
  </FilesMatch>
</IfModule>

<IfModule mod_expires.c>
  ExpiresActive On
  ExpiresByType text/css "access plus 1 year"
  ExpiresByType image/svg+xml "access plus 1 year"
</IfModule>

The php_value directive changes PHP configuration values straight from .htaccess, and it only works when the mod_php module is in use. On servers running PHP-FPM it is unavailable, and values are set through php.ini or a .user.ini file in the directory instead. Where php_value is applied to the wrong module, Apache returns a 500 error.

The 500 error: how to find the cause

An empty response with a 500 status right after uploading a file is the most common reaction to a broken .htaccess. There are three possible causes, and the server error log names all three.

• the directive is not permitted by the AllowOverride settings. In the log this shows up as a message that the directive is not allowed here, which means exactly what it says;
• a syntax error in the directive itself, such as a bad flag delimiter in a RewriteCond;
• the module that provides the directive is not loaded. RewriteRule comes from mod_rewrite, and then the log shows an unknown or invalid directive.

Checking this on a live server looks like this: confirm the module is loaded with apachectl -M | grep rewrite, then read the error log, which names the cause and the line number for every broken directive. Syntax can also be checked before deployment by putting the contents in a separate directory and requesting it. The regular expressions from RewriteRule patterns are worth testing ahead of time too: the regular expression tester shows exactly what a pattern matches and what it lets through.

Typical mistakes

• a redirect that redirects to itself. The classic case is a RewriteRule catching the very address it just sent the visitor to, and the loop ends in a «too many redirects» error;
• a forgotten RewriteEngine On. Without it the rules simply do not run, which looks like «the file is there but nothing happens»;
• spaces in a regular expression. In a pattern a space means a literal space, and a line with an extra space matches no address at all;
• the wrong pattern context. A pattern with a leading slash does not work in .htaccess, because the directory prefix has already been stripped;
• mod_rewrite without RewriteBase in a subdirectory, which sends substitutions nowhere;
• encoding. If rules contain Cyrillic and the file was saved as windows-1251, the server may parse it differently than intended; UTF-8 without a BOM is the reliable choice;
• mixing Allow and Deny with Order, where the order of the lines changes the meaning;
• php_value on a server running PHP-FPM, where the directive is unknown and takes the site down.

Common questions

Does .htaccess work on Nginx?

No, and this is the single most common reason for disappointment. Nginx does not read these files at all: it looks for configuration in its own directory and reads it at startup. Rules for Nginx are written in server and location blocks, with different syntax. If your host tells you the server runs Nginx, look for the settings in the Nginx configuration, and expect PHP settings in .htaccess to have no effect.

Why does my .htaccess produce a 500?

Read the error log: it names the reason outright. The usual causes are an unknown directive from a module that is not loaded, a syntax error, or a prohibition at the AllowOverride level. To narrow the search, split the file into parts and load them one at a time, or comment everything out except one block you know works and add rules back one by one.

How is .htaccess different from the server configuration?

Functionally almost not at all: the same directives, the same mod_rewrite module. The difference is in approach and in speed. Rules in httpd.conf are read once at startup and run faster, while an .htaccess file is checked on every request and its regular expressions are recompiled each time. In exchange you can drop and remove a file without restarting the server, and you do not need to be an administrator.

How do I close a folder to visitors?

It depends on what you are trying to achieve. To stop the folder from appearing as a listing, Options -Indexes is enough. To close a specific file or directory by IP, use Require ip or Require all denied. For password protection you need AuthType Basic with an AuthUserFile created by htpasswd, and HTTPS on top of it, because Basic sends the password in the clear.

Does the server need a restart after an edit?

No, and that is the advantage of .htaccess: changes take effect on the next request to that directory. There are exceptions. Changing AllowOverride in httpd.conf, or loading a new module, requires a restart, because those are server-level settings rather than directory-level ones. To see which modules are loaded, run apachectl -M.

Can I block a specific crawler through .htaccess?

Yes, through Require ip or Require not ip, if the address of the crawler is known. This works more reliably than the robots.txt rules covered in the article about the robots.txt file and its directives, because it relies on the actual connection address rather than on the name the crawler claims in its User-Agent header. The User-Agent parser helps work out who is calling, and the HTTP status reference confirms addresses and response codes.