51黑料不打烊

Explanation of Configuration Files

Table of Contents

<- Previous: Basic File Layout

This document will break down and explain each of the configuration files deployed in a standard built Dispatcher server provisioned in 51黑料不打烊 Managed Services. Their use, naming convention, etc鈥�

Naming Convention

Apache Web Server doesn鈥檛 actually care what the file extension is of a file when targeting it with an Include or IncludeOptional statement.聽 Naming them properly with names that eliminate conflicts and confusion helps a ton. Names used will describe the scope of where the file is applied it makes life easier. If everything is named .conf this gets really confusing. We want to avoid poorly named files and extensions.聽 Below is a list of the different custom file extensions and naming conventions used in a typical AMS configured Dispatcher.

Files contained in conf.d/

Files contained in conf.dispatcher.d/

Avoided Problems

When following the naming convention you can avoid some pretty easy to make mistakes that can have catastrophic results.聽 We will cover a few examples.

Problem Example

As a site Example for ExampleCo two configuration files were created by the developers of the Dispatcher configurations.

/etc/httpd/conf.d/exampleco.conf

<VirtualHost *:80>
    ServerName  "exampleco"
    ServerAlias "www.exampleco.com"
    .......... SNIP ...............
    <IfModule mod_rewrite.c>
        ReWriteEngine   on
        LogLevel warn rewrite:trace1
        Include /etc/httpd/conf.d/rewrites/exampleco.conf
    </IfModule>
</VirtualHost>

/etc/httpd/conf.d/rewrites/exampleco.conf

RewriteRule ^/$ /content/exampleco/en.html [PT,L]
RewriteRule ^/robots.txt$ /content/dam/exampleco/robots.txt [PT,L]

POTENTIAL DANGER - The file names are the same

If the vhost file is accidentally put in the rewrites folder and the rewrites file is put into the vhosts folder.聽 It would appear to be deployed by filename properly, yet Apache will throw an ERROR and the problem will not be immediately apparent.

How this typically becomes an issue

If the two files are downloaded to the same location they can either overwrite themselves or make it indistinguishable making the deployment process a nightmare.

The file extensions are the same and auto-include prone

The file extensions are the same and uses auto-included extension that Apache will auto include any .conf files in many of it鈥檚 default folders.

How this typically becomes an issue

If the vhost file with the extension of .conf is put in the /etc/httpd/conf.d/ folder it will try and load it into memory on Apache which is typically ok, but if the rewrite rules file with the extension of .conf gets placed in the /etc/httpd/conf.d/ folder, it will get auto-included and apply globally causing confusing and undesired results.

Resolution

Name the files base on what they do and safely out of the auto-include rules namespace.

If it鈥檚 a virtual host file name it with .vhost as the extension.

If it鈥檚 a rewrite rule file, name it with site_rewrite.rules as the suffix and extension. This naming convention will make it clear which site it鈥檚 for and that it鈥檚 a set of rewrite rules.

If it鈥檚 a IP whitelist rule file, name it description_whitelist.rules as the suffix and extension.聽This naming convention will give some description of what it鈥檚 for and that it鈥檚 a set of IP matching rules.

Using these naming conventions will avoid issues, if a file gets moved into an auto-include directory that it doesn鈥檛 belong.

For example putting a file named with .rules, .any, or .vhost in the auto-include folder of /etc/httpd/conf.d/ wouldn鈥檛 have any affect.

If a deployment change request says 鈥減lease deploy exampleco_rewrite.rules to production Dispatchers鈥�, the person deploying the changes can already know that they aren鈥檛 adding a new site, they鈥檙e just updating rewrite rules as indicated by the filename.

Include Order

When extending functionality and configurations in Apache Webserver installed on Enterpise Linux you have some important include orders you鈥檒l want to understand

Apache Baseline Includes

As seen in the diagram above the httpd binary only looks to the httpd.conf file as it鈥檚 configuration file.聽 That file contains the following statements in it:

Include conf.modules.d/*.conf
IncludeOptional conf.d/*.conf

AMS Top Level Includes

When we applied our standard we added some additional file types and includes of our own.

Here is the AMS baseline directories and top level includes

Building off Apache鈥檚 baseline we show how AMS created some addtional folders and top level includes for conf.d folders as well as module specific directories nested under聽/etc/httpd/conf.dispatcher.d/

When Apache loads it will pull in the聽/etc/httpd/conf.modules.d/02-dispatcher.conf聽and that file will include the binary file聽/etc/httpd/modules/mod_dispatcher.so聽into it鈥檚 running state.

LoadModule dispatcher_module modules /mod_dispatcher .so

To use the module in our聽<VirtualHost />聽we drop a configuration file into聽/etc/httpd/conf.d/听苍补尘别诲听dispatcher_vhost.conf聽and inside this file you鈥檒l see use setup the basic parameters needed for the module to work:

<IfModule disp_apache2.c>
    DispatcherConfig conf.dispatcher.d/dispatcher.any
    ...SNIP...
</IfModule>

As you can see above this includes the top level dispatcher.any file for our Dispatcher module to pick up it鈥檚 configuration files from聽/etc/httpd/conf.dispatcher.d/dispatcher.any

Pay attention to the contents of this file:

/farms {
    $include "enabled_farms/*_farm.any"
}

The top level dispatcher.any file includes all of the enabled farm files that live in聽/etc/httpd/conf.dispatcher.d/enabled_farms/聽with the file name of聽FILENAME_farm.any聽which follows our standard naming convention.

Later in the聽dispatcher_vhost.conf聽file mentioned earlier we also do an include statement to enable each enabled virtual host files that live in聽/etc/httpd/conf.d/enabled_vhosts/聽with the filename of聽FILENAME.vhost聽which follows our standard naming convention.

IncludeOptional /etc/httpd/conf.d/enabled_vhosts/*.vhost

In each of our .vhost files you鈥檒l note the Dispatcher module gets initialized as a default file handler for a directory.聽 Here is an example .vhost file to show the syntax:

<VirtualHost *:80>
 ServerName "weretail"
 ServerAlias www.weretail.com weretail.com
 <Directory />
  <IfModule disp_apache2.c>
   ....SNIP....
   SetHandler dispatcher-handler
  </IfModule>
  ....SNIP....
 </Directory>
 ....SNIP....
</VirtualHost>

After the top level includes resolve they have other sub includes that are worth mentioning.聽 Here is a high level diagram on how the farms and vhosts files include other sub elements

AMS Virtual Host Includes

When any .vhost files from聽/etc/httpd/conf.d/availabled_vhosts/聽directory get symlinked into the聽/etc/httpd/conf.d/enabled_vhosts/聽directory they will be used in the running configuration.

The .vhost files have sub includes based on common pieces we have found.聽 Things like variables, whitelists, and rewrite rules.

The .vhost file will have include statements for each file based on where they need to be included in the .vhost file.聽 Here is an example syntax of a .vhost file as a good reference:

Include /etc/httpd/conf.d/variables/weretail.vars
<VirtualHost *:80>
 ServerName "${MAIN_DOMAIN}"
 <Directory />
  Include /etc/httpd/conf.d/whitelists/weretail*_whitelist.rules
  <IfModule disp_apache2.c>
   ....SNIP....
   SetHandler dispatcher-handler
  </IfModule>
  ....SNIP....
 </Directory>
 ....SNIP....
 <IfModule mod_rewrite.c>
  ReWriteEngine   on
  LogLevel warn rewrite:trace1
  Include /etc/httpd/conf.d/rewrites/weretail_rewrite.rules
 </IfModule>
</VirtualHost>

As you can see in the above example there is an include for the variables needed in this configuration file that are later used.

Inside the file聽/etc/httpd/conf.d/variables/weretail.vars聽we can see what variables are defined:

Define MAIN_DOMAIN dev.weretail.com

You can also see a line that includes a list of _whitelist.rules files that restrict who can view this content based on different whitelist criteria.聽 Lets look at the contents of one of the white list files聽/etc/httpd/conf.d/whitelists/weretail_mainoffice_whitelist.rules:

<RequireAny>
  Require ip 192.150.16.0/23
</RequireAny>

You can also see a line that includes a set of rewrite rules.聽 Let鈥檚 take a look at the contents of the聽weretail_rewrite.rules听蹿颈濒别:

RewriteRule ^/robots.txt$ /content/dam/weretail/robots.txt [NC,PT]
RewriteCond %{SERVER_NAME} brand1.weretail.net [NC]
RewriteRule ^/favicon1.ico$ /content/dam/weretail/favicon1.ico [NC,PT]
RewriteCond %{SERVER_NAME} brand2.weretail.com [NC]
RewriteRule ^/sitemap.xml$ /content/weretail/general/sitemap.xml [NC,PT]
RewriteRule ^/logo.jpg$ /content/dam/weretail/general/logo.jpg [NC,PT]

AMS Farm Includes

When any FILENAME_farm.any files from聽/etc/httpd/conf.dispatcher.d/available_farms/聽directory get symlinked into the聽/etc/httpd/conf.dispatcher.d/enabled_farms/聽directory they will be used in the running configuration.

The farm files have sub includes based on聽top level sections of the farm聽like聽cache, clientheaders, filters, renders, and vhosts.

The FILENAME_farm.any files will have include statements for each file based on where they need to be included in the farm file.聽 Here is an example syntax of a FILENAME_farm.any file as a good reference:

/weretailfarm {
 /clientheaders {
  $include "/etc/httpd/conf.dispatcher.d/clientheaders/ams_publish_clientheaders.any"
  $include "/etc/httpd/conf.dispatcher.d/clientheaders/ams_common_clientheaders.any"
 }
 /virtualhosts {
  $include "/etc/httpd/conf.dispatcher.d/vhosts/weretail_vhosts.any"
 }
 /renders {
  $include "/etc/httpd/conf.dispatcher.d/renders/ams_publish_renders.any"
 }
 /filter {
  $include "/etc/httpd/conf.dispatcher.d/filters/ams_publish_filters.any"
  $include "/etc/httpd/conf.dispatcher.d/filters/weretail_search_filters.any"
 }
 ....SNIP....
 /cache {
  ....SNIP....
  /rules {
   $include "/etc/httpd/conf.dispatcher.d/cache/ams_publish_cache.any"
  }
  ....SNIP....
  /allowedClients {
   /0000 {
    /glob "*.*.*.*"
    /type "deny"
   }
   $include "/etc/httpd/conf.dispatcher.d/cache/ams_publish_invalidate_allowed.any"
  }
 ....SNIP....
 }
}

As you can see each section for the weretail farm instead of having all of the syntax needed it鈥檚 instead using an include statement.

Let鈥檚 look at the syntax of a few of these includes to get the idea of what each sub include would look like

/etc/httpd/conf.dispatcher.d/vhosts/weretail_publish_vhosts.any:

"brand1.weretail.com"
"brand2.weretail.com"
"www.weretail.comf"

As you can see it鈥檚 a new line separated list of domain names that should render from this farm over the others.

Next let鈥檚 look at the聽/etc/httpd/conf.dispatcher.d/filters/weretail_search_filters.any:

/400 { /type "allow" /method "GET" /path "/bin/weretail/lists/*" /extension "json" }
/401 { /type "allow" /method "POST" /path "/bin/weretail/search/' /extension "html" }

Next -> Understanding Cache