How to enable the Analog Internal Search Query Report for WordPress

Did you realise that Analog CE can report in searches made through internal website search engines as well as external ones? This article discusses how to connect the WordPress search function to the Analog CE Internal Search Reports.

 

Keeping an eye on your users

Knowing what your users are searching for should be a core part of your SEO strategy. The Analog CE Internal Search Query and Internal Search Word reports can be a useful tool in understanding user interest and habits.

Analog CE’s two Internal Search reports are disabled by default. To enable them, you must edit your site config file. If you are using a Global/Site config file structure, you should use the local site config file. Not the Global config file. The exception should be if your Analog CE install is being used for WordPress web-farming.

 

Edit your Analog CE configuration file

Before you can edit the config file you must know the path to your WordPress install. The path should be relative to the root domain of your website.

Tip: Locate the xmlrpc.php, wp-config.php and wp-cron.php files for your install on your web server. This is the root folder of your WordPress installation.

For example. Assuming that your root domain is www.mysite.com:

  • If your wp-config.php is located at www.mysite.com/wp-config.php, your Internal Search Engine Path is “/”.
  • Alternately, if your wp-config.php is found at www.mysite.com/blog/wp-config.php, your Internal Search Engine Path is “/blog/”.

 

Edit your Analog CE configuration file by adding the following lines:

INTSEARCHQUERY ON
INTSEARCHENGINE <Internal Search Engine Path> s
INTSEARCHQUERYFLOOR 10r
INTSEARCHWORDFLOOR 10r

For example:

INTSEARCHQUERY ON
INTSEARCHENGINE / s
INTSEARCHQUERYFLOOR 10r
INTSEARCHWORDFLOOR 10r

This config sample will:

  1. Enable the report
  2. Tells Analog CE how to find the search term (the user provided value of ‘s’)
  3. Sets the Search Query report to only show searches that occur 10 or more times
  4. Configures the Search Word report to only show keywords that occur 10 or more times

 

The next time Analog CE runs, you will have “Internal Search Query Report” and “Internal Search Word Report” sections. If you are running Report Magic, they will be at the bottom of the side-navigation.

Creating a Link Anonymiser Service for Analog CE’s ANONYMIZERURL setting

This article discusses how to create an link anonymiser service redirector to make use of the Analog CE 6.0.16+ ANONYMIZERURL setting.

 

Why use an anonymiser?

If a user clicks a link on an Analog CE “Requesting Site” or “Requesting URL” report. The users web browser will send a HTTP Referrer header with the request to download the web page; this request will include the full URL or your Analog CE report. The receiving server will likely log the request, allowing its owner to see where the request originated.

This may expose the Internet or Intranet URL or your stats page to the target website owner. They may in-turn inadvertantly publicise it via their own statistics page and/or link-back tracker service. This makes it possible for other agents, including competitors, search engines and malicious users to discover information about your website. Worse your web server may become the target of SEO spammers.

 

What is SEO spam?

SEO spam is the practice of attempting to improve a website/page position on a search engine by creating ‘false’ links into that website. If your referring site/URL report is public it is possible for a malicious actor to artificially position one or more URLs on the report. This is achieved through a manipulated HTTP GET request containing a HTTP Referrer header with the URL/site that they want to inject onto your report. After making several hundred requests in this fashion the spammer will wait for the report to be updated. After confirming that their site has appeared in the report, they submit your statistics page(s) to search engines.

Once compromised, it is likely that your exploitability will be recorded in one or more botnets and will see wider exploitation.

 

Why create your own anonymiser?

You can use public anonymiser services such as anonymizer.info or anon.to with Analog CE using one of the code samples below.

ANONYMIZERURL https://anon.to/?

ANONYMIZERURL https://anonymizer.info/?

This may not be acceptable to you, or your organisational security policy. Firstly because while the owner of the resultant web server will not discover the true origin of the request, the public anonymiser service will. Secondly, there is no contract assuring service availability or the privacy of its log files. Finally, it is inevitable that the service is going to profit from your transaction. Advertising placement is likely, creating a delay in the redirect.

 

Code your own Anonymous Link Redirector

The following code snippets can be used to program your own basic redirector using service side scripting technology.

 

ASP 3 / Classic ASP

<%@LANGUAGE="VBSCRIPT" CODEPAGE="65001" EnableSessionState="False"%>
<% Option Explicit %>
<%
  Response.Status = "302 Found"
  call Response.AddHeader("Location", Request.QueryString)
  Response.End()
%>

Save the file as redirector.asp and add the following to your Analog CE global configuration file:

ANONYMIZERURL http://my-server.domain.com/redirector.asp?

 

ASP.net

<%@ Page Language="C#" %>
<script runat="server">
  private void Page_Load(object sender, EventArgs e)
  {
    Response.Redirect(HttpContext.Current.Request.ServerVariables["QUERY_STRING"], true);
  }
</script>

Save the file as redirector.aspx and add the following to your Analog CE global configuration file:

ANONYMIZERURL http://my-server.domain.com/redirector.aspx?

 

PHP

<?php
  header('Location: ' . $_SERVER['QUERY_STRING'], true, 302);
  exit;
?>

Save the file as redirector.php and add the following to your Analog CE global configuration file:

ANONYMIZERURL http://my-server.domain.com/redirector.php?

 

Conclusion

The above code samples illustrate how to create a redirector in several different languages. The redirector URL will be sent to the destination server however the originating statistics page will now be protected. This protects your Analog CE stats pages from prying eyes while reducing the risk of SEO spamming.

Analog’s Search Query Report is empty or sparsely populated

If you are an Analog CE user, you may have noticed that the Search Query and Search Word reports suggest that your site has been receiving little to no search engine traffic. The Search reports may be empty or sparsely populated with results. This article discusses the cause of the problem.

 

What are the External Search Reports?

The premise of the external (rather than internal) search report is to show you what a visitor typed into a search engine before landing on your site. The Search Query report shows the entire search phrase used, while the Search Word report creates a word count report for anything de-marked by a space.

They are intended to help you identify what your users are searching for as part of your SEO activities. Unfortunately, over the last couple of years the data available on the report has been declining. On some sites, the report may have disappeared altogether as no data is available to Analog CE.

 

Fixing the empty or sparsely populated report

There are three reasons for the decline in report quality. They are outlined below in order of lest to most severe.

 

Missing Search Engines

The default config file in the Analog CE source release is a clone of the original Analog 6 config file. It (and your own) config files have not been updated with modern search engines. Consequently traffic from Bing and DuckDuck Go (and others) will be wholly absent.

I will be including an updated config file sample in Analog CE starting with Analog CE 6.0.16. Analog CE users will need to replace or merge this into your running configuration files. To perform this manually, you must add relevant SEARCHENGINE entries to your global/site config files.

 

HTTP > HTTPS

As a result of security issues and the gross mismanagement of private data transiting public networks by the CIA. The world made a rapid switch from HTTP to HTTPS in 2015. The default Analog 6 config file only included Search Engine detection definitions for unencrypted HTTP connections. Consequently Analog CE has stopped reporting traffic from any HTTPS search engine source.

Starting with Analog CE 6.0.16, I will fix this in the bundled sample file. You will need to merge the SEARCHENGINE changes into you respective config files.

 

Privacy & Commercialisation of Analytics

The crux of the problem stems from Search Engines no-longer send the search query in their referrer header. Some see this as being for legitimate privacy reasons, others for commercial benefit. DuckDuck Go would certainly argue the former. For Google, the reason is less clear. With up to 98% market share, Google is effectively forcing webmasters to use Google Search Console and/or Google Analytics to understand their user needs. In turn, this feeds Google with more accurate data on user activity for ad-profiling. It also ensures that you as a webmaster are creating the web that Google wants, above any other concern. Any webmaster threatened with de-listing due to mobile device incompatibility will understand this.

The following sample log line is from the 5th April 2015

2015-04-05 15:38:29 W3SVC5 web02 xxx.xxx.xxx.xxx GET /downloads/msie/ie60sp1/ - 80 - yyy.yyy.yyy.yyy HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+98) - http://www.google.nl/search?q=internet+explorer+download+windows+98&ie=ISO-8859-1&hl=nl&source=hp&gbv=1 www.hpcfactor.com 200 0 0 10747 331 406

This is a similar one from 4th June 2019

2019-06-04 00:17:31 W3SVC5 web01 xxx.xxx.xxx.xxx GET /hardware/devices/specification.asp d=142 443 - yyy.yyy.yyy.yyyy HTTP/2 Mozilla/5.0+(Linux;+Android+6.0;+HM-G552-FL+Build/MRA58K)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/56.0.2924.87+Mobile+Safari/537.36 - https://www.google.com/ www.hpcfactor.com 200 0 0 12534 491 495

If you scroll to the right, you will notice that in the earlier file, the user searched for “Internet Explorer Download Windows 98”. In the latter file, Google only informs the server that the request came from google.com. The context of the search is no longer offered.

As a consequence, the data required to create the Search Query report is not available to Analog CE. This is the reason why the Analog Search Query Report is empty or sparsely populated. It is not a bug. Rightly or wrongly, it is a symptom of the modern web.

Using Analog CE to provide automatic statistics on a multi-site IIS Server

System Requirements:

  • Windows NT 4.0 Workstation
  • Windows 2000 Professional
  • Windows XP
  • Windows NT 4.0 Server
  • Windows 2000 Server
  • Windows Server 2003, R2
  • Windows Server 2008, R2
  • Windows Server 2012
  • Windows Server 2016
  • Windows Server 2019
  • Internet Information Services 4.0, 5.0, 5.1, 6.0, 7.0, 7.5, 8.0, 10
  • Analog CE

The Problem:

This article discusses how to automate the dissemination of statistics data to multiple web site clients on a typical, single instance production IIS web server. The article discusses the configuration of Analog and the file system considerations required to automate the delivery of statistics to all IIS web sites hosted on the server instance.

More Info

In the default Analog CE sample config file, Analog is configured to support a single web site through a single instance of the Analog executive.

In a larger IIS web host environment, a single IIS server may potentially host hundreds or even thousands of individual web sites. In this model, it is necessary to tweak the configuration of Analog to act as a single instance, multi-target model – instead of the default single instance, single target model.

How-to

The how-to is split into the following sections:

  1. What is not covered here
  2. Pre-requisites
  3. File System Considerations
  4. Prepare Analog
  5. Configure the Analog Server Settings
  6. Configure the Analog Site Settings
  7. Running Analog on a Schedule

What is not covered here

This guide is specifically written to demonstrate the configuration of Analog. To that end it does not cover the installation, setup or configuration of IIS in either a manual or an automatic capacity.

This guide will provide you with a scalable solution towards running Analog on a production IIS server, however most administrators will wish to integrate this process into the creation of web sites on IIS.

Pre-requisites

You will need to ensure that scripts run through an account with appropriate permissions to be able to write to the output folders on your web server. If you are using UAC on your server, ensure that you execute the scripts with elevation.

File System Considerations

By default, IIS provides a separation between the sites web root and the logs associated with that web site instance. By default IIS bases its web site root under c:\inetpub and the log files necessary for Analog to function are stored under c:\windows\system32\LogFiles\<Instance>.

Hosting providers should move these files to a different location, one that is preferably separate from the operating system volume. In most cases, these will be accessible by the tenant. How you wish to do this will depend primarily on whether you want your client users to gain access to the logs or not.

A suggestion for how to structure your web sites on disk in both scenarios is shown below. In both examples the client would FTP directly into d:\sites\<domain> to upload or download content.

User has access to Logs User does not have access to Logs
d:
- \sites
-   \www.domain.com
-      \web
-        \stats
-      \logs
-        \W3SVC1
-   \www.website.net
-      \web
-        \stats
-      \logs
-        \W3SVC2
d:
- \sites
-   \www.domain.com
-      \stats
-   \www.website.net
-      \stats
- \logs
-   \W3SVC1
-   \W3SVC2
In this example the client has access to their own log files (presumably over SFTP). This allows them to download their logs for offline analysis, however you will also generate an Analog report for them into the \stats folder under their web site root folder (\web). In this example, log files are available for internal users only, you plan to use these to generate a report for the client using Analog into the \stats folder.

Prepare Analog

To fit in with the file-system structure outlined above, this guide will assume that the following file system structure is in use:

d:
- \parser
-   \Analog
-   \ReportMagic
- \sites
-   \www.domain.com
-      \web
-        \stats
-      \logs
-        \W3SVC1
-   \www.website.net
-      \web
-        \stats
-      \logs
-        \W3SVC2

Copy the contents of the latest Analog CE binary zip file into d:\parser\Analog.

 

Configure the Analog Server Settings

Analog’s global analog.cfg file must be reconfigured to be site independent so that it becomes the global configuration file. This is achieved by removing all references to a particular web site to create a baseline configuration. In essence, you are creating the default values for all Analog executions that will be used unless overridden in the local config.

Open the analog.cfg file in notepad.

Using the Analog CE Configuration File Generator output as an example, you should remove the highlighted lines to form a baseline configuration.

# Analog CE Baseline Statistics Configuration

# Version 1.0.1

# See http://www.c-amie.co.uk/ for updates and more
LOGFILE c:\windows\system32\logfiles\w3svc1\*.logOUTPUT HTML

OUTFILE c:\inetpub\wwwroot\stats\index.html

HOSTNAME "Analog Test Site"

HOSTURL http://www.domain.com/

IMAGEDIR "images/"

STYLESHEET images/analog.css

# Reports Enabled/Disabled List
ALL ON
ALLCHART ON GENERAL ON #General Summary
YEARLY ON #Yearly Report
QUARTERLY ON #Quarterly Report
MONTHLY ON #Monthly Report
WEEKLY ON #Weekly Report
DAILYREP ON #Daily Report
DAILYSUM ON #Daily Summary
HOURLYREP ON #Hourly Report
HOURLYSUM ON #Hourly Summary
WEEKHOUR ON #Hour of the Week Summary
QUARTERREP ON #Quarter-Hour Report
QUARTERSUM ON #Quarter-Hour Summary
FIVEREP ON #Five-Minute Report
FIVESUM ON #Five-Minute Summary
HOST ON #Host Report
REDIRHOST ON #Host Redirection Report
FAILHOST ON #Host Failure Report
ORGANISATION ON #Organisation Report
DOMAIN ON #Domain Report
REQUEST ON #Request Report
DIRECTORY ON #Directory Report
FILETYPE ON #File Type Report
SIZE ON #File Size Report
PROCTIME ON #Processing Time Report
REDIR ON #Redirection Report
FAILURE ON #Failure Report
REFERRER ON #Referrer Report
REFSITE ON #Referring Site Report
SEARCHQUERY ON #Search Query Report
SEARCHWORD ON #Search Word Report
INTSEARCHQUERY ON #Internal Search Query Report
INTSEARCHWORD ON #Internal Search Word Report
REDIRREF ON #Redirected Referrer Report
FAILREF ON #Failed Referrer Report
BROWSERREP ON #Browser Report
BROWSERSUM ON #Browser Summary
OSREP ON #Operating System Report
VHOST ON #Virtual Host Report
REDIRVHOST ON #Virtual Host Redirection Report
FAILVHOST ON #Virtual Host Failure Report
USER ON #User Report
REDIRUSER ON #User Redirection Report
FAILUSER ON #User Failure Report
STATUS ON #Status Code Report

# Referring URL Report
REFLINKINCLUDE *
REFREPEXCLUDE http://domain.com/*
REFREPEXCLUDE http://www.domain.com/*
REFFLOOR 1r

# Referring Site
REFSITEEXCLUDE http://domain.com/
REFSITEEXCLUDE http://www.domain.com/

# Request Report
REQFLOOR 1r
REQEXCLUDE *.jpg
REQEXCLUDE *.gif
REQEXCLUDE *.png
REQEXCLUDE *.bmp
REQEXCLUDE *.class
REQEXCLUDE *.js

# Status Code Report
304ISSUCCESS ON # Includes 304 errors on the request report

# Redirected Referrers Report
REDIRREFLINKINCLUDE *

# Failed Referrers Report
FAILREFLINKINCLUDE *

# Browser Summary
SUBBROW */*

# Operating System Report
OSCHARTEXPAND Windows

# Default Documents
# PAGEINCLUDE *.shtml
PAGEINCLUDE *.asp
PAGEINCLUDE *.aspx
# PAGEINCLUDE *.jsp
# PAGEINCLUDE *.cfm
# PAGEINCLUDE *.pl
# PAGEINCLUDE *.php
# PAGEINCLUDE *.rb

# Custom Exclusions
#FILEEXCLUDE /admin/*

# Robots & Crawlers
ROBOTINCLUDE REGEXPI:robot
ROBOTINCLUDE REGEXPI:spider
ROBOTINCLUDE REGEXPI:crawler ROBOTINCLUDE Baiduspider/*
ROBOTINCLUDE bingbot/*
ROBOTINCLUDE Googlebot*
ROBOTINCLUDE Infoseek*
ROBOTINCLUDE msnbot*
ROBOTINCLUDE Scooter*
ROBOTINCLUDE *Slurp*
ROBOTINCLUDE Ultraseek*
ROBOTINCLUDE *Validator*
ROBOTINCLUDE YandexBot/*
ROBOTINCLUDE YodaoBot/* # Search Engines
SEARCHENGINE http://*altavista.*/* q
SEARCHENGINE http://*yahoo.*/* p
SEARCHENGINE http://*google.*/* q,as_q,as_epq,as_oq
SEARCHENGINE http://*bing.*/* q
SEARCHENGINE http://*lycos.*/* query
SEARCHENGINE http://*aol.*/* query
SEARCHENGINE http://*excite.*/* search
SEARCHENGINE http://*go2net.*/* general
SEARCHENGINE http://*metacrawler.*/* general
SEARCHENGINE http://*msn.*/* MT
SEARCHENGINE http://*hotbot.com/* MT
SEARCHENGINE http://*netscape.*/* search
SEARCHENGINE http://*looksmart.*/* key
SEARCHENGINE http://*infoseek.*/* qt
SEARCHENGINE http://*webcrawler.*/* search,searchText
SEARCHENGINE http://*goto.*/* Keywords
SEARCHENGINE http://*snap.*/* keyword
SEARCHENGINE http://*dogpile.*/* q
SEARCHENGINE http://*bbc.*/* q
SEARCHENGINE http://*askjeeves.*/* ask
SEARCHENGINE http://*ask.*/* ask
SEARCHENGINE http://*aj.*/* ask
SEARCHENGINE http://*directhit.*/* qry
SEARCHENGINE http://*alltheweb.*/* query
SEARCHENGINE http://*naver.*/* query
SEARCHENGINE http://*northernlight.*/* qr
SEARCHENGINE http://*nlsearch.*/* qr
SEARCHENGINE http://*dmoz.*/* search
SEARCHENGINE http://*newhoo.*/* search
SEARCHENGINE http://*netfind.*/* query,search,s
SEARCHENGINE http://*/netfind* query
SEARCHENGINE http://*/pursuit query
SEARCHENGINE http://*/mamma.*/* query
SEARCHENGINE http://*ixquick.*/* metasearch.pl
SEARCHENGINE http://*vivisimo.*/* search
SEARCHENGINE http://*mysearch.*/* searchfor

# Static Internet Documents
TYPEOUTPUTALIAS .html ".html [Hypertext Markup Language]"
TYPEOUTPUTALIAS .htm ".htm [Hypertext Markup Language]"
TYPEOUTPUTALIAS .shtml ".shtml [Server-parsed HTML]"
TYPEOUTPUTALIAS .ps ".ps [PostScript]"
TYPEOUTPUTALIAS .gz ".gz [Gzip compressed files]"
TYPEOUTPUTALIAS .tar.gz ".tar.gz [Compressed archives]"
TYPEOUTPUTALIAS .txt ".txt [Plain text Documents]"
TYPEOUTPUTALIAS .cdf ".cdf [Channel Definition File]"

# Scripting & Dynamic Internet Content Files
TYPEOUTPUTALIAS .asp ".asp [Active Server Pages]"
TYPEOUTPUTALIAS .aspx ".aspx [Active Server Pages .net]"
TYPEOUTPUTALIAS .cgi ".cgi [CGI scripts]"
TYPEOUTPUTALIAS .pl ".pl [Perl scripts]"
TYPEOUTPUTALIAS .css ".css [Cascading Style Sheets]"
TYPEOUTPUTALIAS .class ".class [Java class files]"
TYPEOUTPUTALIAS .hqx ".hqx [Macintosh archives]"
TYPEOUTPUTALIAS .jsp ".jsp [Java Server Pages]"
TYPEOUTPUTALIAS .cfm ".cfm [Cold Fusion]"
TYPEOUTPUTALIAS .php ".php [PHP Hypertext Processor]"
TYPEOUTPUTALIAS .js ".js [JavaScript code]"
TYPEOUTPUTALIAS .dll ".dll [Dynamic Link Library]"
TYPEOUTPUTALIAS .asa ".asa [Web Server Scripting Configuration]"
TYPEOUTPUTALIAS .url ".url [Windows Internet Shortcut]"
TYPEOUTPUTALIAS .lnk ".lnk [Windows Explorer Shortcut]"
TYPEOUTPUTALIAS .ini ".ini [Configuration Settings File]"
TYPEOUTPUTALIAS .log ".log [Log Files]"
TYPEOUTPUTALIAS .diz ".diz [DIZ Text File]"
TYPEOUTPUTALIAS .inc ".inc [SSI Inclusion File]"
TYPEOUTPUTALIAS .xml ".xml [eXtensible Markup Language File]"
TYPEOUTPUTALIAS .rdf ".rdf [Resource Description Framework File]"
TYPEOUTPUTALIAS .rb ".rb [Ruby script file]"

# Image Files
TYPEOUTPUTALIAS .jpg ".jpg [JPEG graphics]"
TYPEOUTPUTALIAS .jpeg ".jpeg [JPEG graphics]"
TYPEOUTPUTALIAS .jpe ".jpe [JPEG graphics]"
TYPEOUTPUTALIAS .gif ".gif [GIF graphics]"
TYPEOUTPUTALIAS .gfa ".gfa [GIF graphics]"
TYPEOUTPUTALIAS .png ".png [Portable Network Graphics]"
TYPEOUTPUTALIAS .bmp ".bmp [BitMap]"
TYPEOUTPUTALIAS .bmz ".bmz [BitMap]"
TYPEOUTPUTALIAS .dib ".dib [BitMap]"
TYPEOUTPUTALIAS .rle ".rle [BitMap]"
TYPEOUTPUTALIAS .2bp ".2bp [Windows CE 4 Tone BitMap]"
TYPEOUTPUTALIAS .ico ".ico [Icon File]"
TYPEOUTPUTALIAS .tif ".tif [Tag Image File Format]"
TYPEOUTPUTALIAS .tiff ".tiff [Tag Image File Format]"
TYPEOUTPUTALIAS .wmf ".wmf [Windows Metafile (ClipArt)]"
TYPEOUTPUTALIAS .pct ".pct [Macintosh PICT]"
TYPEOUTPUTALIAS .pict ".pict [Macintosh PICT]"
TYPEOUTPUTALIAS .pcz ".pcz [Macintosh PICT Compressed]"
TYPEOUTPUTALIAS .pcd ".pcd [Kodak Photo CD]"
TYPEOUTPUTALIAS .pcx ".pcx [PC Paintbrush]"
TYPEOUTPUTALIAS .cdr ".cdr [Corel Draw]"
TYPEOUTPUTALIAS .cgm ".cgm [COmputer Graphics Metafile]"
TYPEOUTPUTALIAS .eps ".eps [Encapsulated PostScript]"
TYPEOUTPUTALIAS .fpx ".fpx [FPX Format]"
TYPEOUTPUTALIAS .wpg ".wpg [WordPerfect Graphics]"
TYPEOUTPUTALIAS .mix ".mix [Picture IT! Format]"
TYPEOUTPUTALIAS .psd ".psd [Adobe Photoshop Document]"

# Multimedia Audio, Video & Misc
TYPEOUTPUTALIAS .wav ".wav [WAV sound files]"
TYPEOUTPUTALIAS .avi ".avi [AVI movies]"
TYPEOUTPUTALIAS .arc ".arc [Compressed archives]"
TYPEOUTPUTALIAS .mid ".mid [MIDI sound files]"
TYPEOUTPUTALIAS .midi ".midi [MIDI sound files]"
TYPEOUTPUTALIAS .rmi ".rmi [MIDI sound files]"
TYPEOUTPUTALIAS .ivf ".ivf [Indeo Video Format movie]"
TYPEOUTPUTALIAS .aif ".aif [AIFF sound files]"
TYPEOUTPUTALIAS .aifc ".aifc [AIFF sound files]"
TYPEOUTPUTALIAS .aiff ".aiff [AIFF sound files]"
TYPEOUTPUTALIAS .au ".au [AU sound files]"
TYPEOUTPUTALIAS .snd ".snd [AU sound files]"
TYPEOUTPUTALIAS .mp3 ".mp3 [MP3 sound files]"
TYPEOUTPUTALIAS .m4a ".m4a [MPEG 4 Audio]"
TYPEOUTPUTALIAS .m4v ".m4v [MPEG 4 Video]"
TYPEOUTPUTALIAS .mov ".mov [Quick Time movie]"
TYPEOUTPUTALIAS .mpg ".mpg [MPEG movie]"
TYPEOUTPUTALIAS .mpeg ".mpeg [MPEG movie]"
TYPEOUTPUTALIAS .m1v ".m1v [MPEG 1 Video]"
TYPEOUTPUTALIAS .mp2v ".mp2v [MPEG 2 Video]"
TYPEOUTPUTALIAS .mpe ".mpe [MPEG movie]"
TYPEOUTPUTALIAS .wax ".wax [Windows Media Audio Extension]"
TYPEOUTPUTALIAS .wvx ".wvx [Windows Media Video Extension]"
TYPEOUTPUTALIAS .m3u ".m3u [MPEG 3 Audio]"
TYPEOUTPUTALIAS .wma ".wma [Windows Media Audio]"
TYPEOUTPUTALIAS .wmv ".wmv [Windows Media Video]"
TYPEOUTPUTALIAS .ra ".ra [Real Audio File]"
TYPEOUTPUTALIAS .ram ".ram [Real Audio Media]"
TYPEOUTPUTALIAS .asf ".asf [Microsoft Advanced Streaming Format]"
TYPEOUTPUTALIAS .asx ".asx [Microsoft Advanced Streaming Extensions]"
TYPEOUTPUTALIAS .pdf ".pdf [Adobe Portable Document Format]"
TYPEOUTPUTALIAS .swf ".swf [Adobe Flash Object]"

# Microsoft Office & Pocket Office
TYPEOUTPUTALIAS .mdb ".mdb [Microsoft Access Database]"
TYPEOUTPUTALIAS .accdb ".accdb [Microsoft Access 2007 Database]"
TYPEOUTPUTALIAS .ppv ".ppv [PowerPoint Viewer File]"
TYPEOUTPUTALIAS .ppt ".ppt [PowerPoint File]"
TYPEOUTPUTALIAS .pptx ".ppxt [PowerPoint File]"
TYPEOUTPUTALIAS .xls ".xls [Excel SpreadSheet]"
TYPEOUTPUTALIAS .xlsx ".xlsx [Excel 2007 SpreadSheet]"
TYPEOUTPUTALIAS .doc ".doc [Microsoft Word Document]"
TYPEOUTPUTALIAS .docx ".docx [Microsoft Word 2007 Document]"
TYPEOUTPUTALIAS .pwd ".pwd [Microsoft Pocket Word Document]"
TYPEOUTPUTALIAS .pwt ".pwt [Microsoft Pocket Word Template]"
TYPEOUTPUTALIAS .pxl ".pxl [Microsoft Pocket Excel Document]"
TYPEOUTPUTALIAS .pxt ".pxt [Microsoft Pocket Excel Template]"
TYPEOUTPUTALIAS .rtf ".rtf [Rich Text Format]"
TYPEOUTPUTALIAS .pub ".pub [Publisher File]"
TYPEOUTPUTALIAS .mps ".mps [Microsoft Pocket Streets Map]"
TYPEOUTPUTALIAS .psm ".psm [Microsoft Pocket Automap Streets]"
TYPEOUTPUTALIAS .ics ".ics [vCalendar / iCalendar File]"

# Databases
TYPEOUTPUTALIAS .db ".db [DataBase File]"
TYPEOUTPUTALIAS .csv ".csv [CSV File]"
TYPEOUTPUTALIAS .dbf ".dbf [Database File]"

# Executables, Installations, Execution & Applications
TYPEOUTPUTALIAS .msi ".msi [Microsoft Installer Package]"
TYPEOUTPUTALIAS .cab ".cab [Cabnet Archive]"
TYPEOUTPUTALIAS .bat ".bat [Batch Files]"
TYPEOUTPUTALIAS .com ".com [Compiled Executable]"
TYPEOUTPUTALIAS .exe ".exe [Executables]"
TYPEOUTPUTALIAS .zip ".zip [Zip archives]"
TYPEOUTPUTALIAS .zip ".rar [Rar archives]"
TYPEOUTPUTALIAS .hlp ".hlp [Windows Help Files]"
TYPEOUTPUTALIAS .chm ".chm [Compiled HTML Help]"
TYPEOUTPUTALIAS .dat ".dat [Internet Explorer Installer Data File]"
TYPEOUTPUTALIAS .dll ".dll [Dynamic Link Library]"

# Misc
TYPEOUTPUTALIAS .bin ".bin [Binary File]"
TYPEOUTPUTALIAS .iso ".iso [CD-ROM/DVD-ROM Image file]"
TYPEOUTPUTALIAS .nrg ".nrg [Nero Burning ROM CD-ROM/DVD-ROM Image file]"
TYPEOUTPUTALIAS .ida ".ida [IIS default.ida - Code Red II Attack]"
TYPEOUTPUTALIAS .reg ".reg [Windows Registry META Data]"

SUBTYPE *.gz,*.Z

To save time, disk space and bandwidth it is recommended that you edit the style sheet location inside the baseline configuration to point to a common location

STYLESHEET images/analog.css

For example, you could create an administrative resource on a separate web to host these resources e.g.

STYLESHEET http://static.mycompany.com/analog.css

Note: This shared resource site will be equally useful if deploying Report Magic stats along with Analog (not covered in this guide).

 

Configure the Analog Site Settings

Now that you have created a common baseline for Analog’s global config, the next step is to configure Analog for each of your sites.

Navigating to d:\sites\www.domain.com create a file called analog.cfg

Open this configuration file and this time enter only the information that you removed from the baseline configuration, ensuring that you tailor the settings to this particular web site For example:

# Default Local Web Statistics Configuration

# Version 1.0.1
LOGFILE d:\sites\www.domain.com\logs\W3SVC1\*.log
OUTFILE d:\sites\www.domain.com\stats\index.html
HOSTNAME "Statistics For Domain.com"

#HOSTURL http://www.domain.com/

IMAGEDIR "images/"

# Referring URL
REFLINKINCLUDE *

# Refering Site

# Refering Site Alias
#REFALIAS http://domain.com/* http://www.domain.com/

#Request Report
REQFLOOR 10r
REQEXCLUDE *.jpg
REQEXCLUDE *.gif
REQEXCLUDE *.png
REQEXCLUDE *.bmp
REQEXCLUDE *.class
REQEXCLUDE *.js

# Custom Exclusions
FILEEXCLUDE /stats/*

You should automate the creation of this configuration file as part of your site creation activities. The fastest way to achieve this is to write the configuration file to the d:\sites\<web site>\analog.cfg as part of your website creation script.

In the above example, a default Analog CE HTML report will be generated in d:\sites\www.domain.com\stats\ (http://www.domain.com/stats/ to the user).

Ensure that you repeat the creation of this file for all web sites on your server.

Running Analog on a Schedule

Now that Analog has been configured, the last step is to schedule it to run. The simplest way to do this is to use a batch script and the Windows task scheduler.

With some creative scripting (exampled below) you can configure the scheduled job to crawl through the d:\sites folder looking for site-level analog.cfg files to parse through to the Analog CE executable.

The example below should be saved via Windows Notepad as d:\parser\GenerateStats.cmd

cls

@echo off

SET ALGROOT=d:\parser\Analog\

SET WEBROOT=d:\sitesFOR /f "tokens=*" %%A IN ('dir /b d:\sites') DO (echo Looking for: analog.cfg in %%A

IF EXIST "%WEBROOT%\%%A\analog.cfg" (
echo Found analog.cfg for %%A
md "%WEBROOT%\%%A\web\stats"
echo.
"%ALGROOT%analog.exe" +g%WEBROOT%\%%A\analog.cfg > %WEBROOT%\%%A\analog.log
echo.
) ELSE (
echo ANALOG CONFIG FILE NOT FOUND IN "%WEBROOT%\%%A\"
echo.
)
)

This script searches all sub-folders of d:\sites for the presence of an analog.cfg file e.g. d:\sites\www.mydomain.com\analog.cfg. If it finds one it ensures that there is an appropriate output directory (\web\stats) and then runs the Analog CE executable using the baseline configuration (this is implicit) as well as the local site-level analog.cfg configuration file (explicit) to produce the report.

The report will be output as specified in the site-level analog.cfg.

Any Analog parser errors will be written into the d:\sites\www.mydomain.com folder as analog.log, allowing for analysis by the client.

In Windows Task Scheduler, create a basic task that will run d:\parser\GenerateStats.cmd ever day, 12 hours, 6 hours (or at whatever interval you require).

Once complete, as you add new web sites (and their associated site-level analog.cfg files) or remove old ones they will automatically be picked-up and parsed through the statistics engine without any administrative intervention.

See Also

View: Analog CE Configuration File Generator