How to enable the Analog Internal Search Query Report for WordPress

Did you realise that Analog CE can report in searches made through internal website search engines as well as external ones? This article discusses how to connect the WordPress search function to the Analog CE Internal Search Reports.

 

Keeping an eye on your users

Knowing what your users are searching for should be a core part of your SEO strategy. The Analog CE Internal Search Query and Internal Search Word reports can be a useful tool in understanding user interest and habits.

Analog CE’s two Internal Search reports are disabled by default. To enable them, you must edit your site config file. If you are using a Global/Site config file structure, you should use the local site config file. Not the Global config file. The exception should be if your Analog CE install is being used for WordPress web-farming.

 

Edit your Analog CE configuration file

Before you can edit the config file you must know the path to your WordPress install. The path should be relative to the root domain of your website.

Tip: Locate the xmlrpc.php, wp-config.php and wp-cron.php files for your install on your web server. This is the root folder of your WordPress installation.

For example. Assuming that your root domain is www.mysite.com:

  • If your wp-config.php is located at www.mysite.com/wp-config.php, your Internal Search Engine Path is “/”.
  • Alternately, if your wp-config.php is found at www.mysite.com/blog/wp-config.php, your Internal Search Engine Path is “/blog/”.

 

Edit your Analog CE configuration file by adding the following lines:

INTSEARCHQUERY ON
INTSEARCHENGINE <Internal Search Engine Path> s
INTSEARCHQUERYFLOOR 10r
INTSEARCHWORDFLOOR 10r

For example:

INTSEARCHQUERY ON
INTSEARCHENGINE / s
INTSEARCHQUERYFLOOR 10r
INTSEARCHWORDFLOOR 10r

This config sample will:

  1. Enable the report
  2. Tells Analog CE how to find the search term (the user provided value of ‘s’)
  3. Sets the Search Query report to only show searches that occur 10 or more times
  4. Configures the Search Word report to only show keywords that occur 10 or more times

 

The next time Analog CE runs, you will have “Internal Search Query Report” and “Internal Search Word Report” sections. If you are running Report Magic, they will be at the bottom of the side-navigation.

Creating a Link Anonymiser Service for Analog CE’s ANONYMIZERURL setting

This article discusses how to create an link anonymiser service redirector to make use of the Analog CE 6.0.16+ ANONYMIZERURL setting.

 

Why use an anonymiser?

If a user clicks a link on an Analog CE “Requesting Site” or “Requesting URL” report. The users web browser will send a HTTP Referrer header with the request to download the web page; this request will include the full URL or your Analog CE report. The receiving server will likely log the request, allowing its owner to see where the request originated.

This may expose the Internet or Intranet URL or your stats page to the target website owner. They may in-turn inadvertantly publicise it via their own statistics page and/or link-back tracker service. This makes it possible for other agents, including competitors, search engines and malicious users to discover information about your website. Worse your web server may become the target of SEO spammers.

 

What is SEO spam?

SEO spam is the practice of attempting to improve a website/page position on a search engine by creating ‘false’ links into that website. If your referring site/URL report is public it is possible for a malicious actor to artificially position one or more URLs on the report. This is achieved through a manipulated HTTP GET request containing a HTTP Referrer header with the URL/site that they want to inject onto your report. After making several hundred requests in this fashion the spammer will wait for the report to be updated. After confirming that their site has appeared in the report, they submit your statistics page(s) to search engines.

Once compromised, it is likely that your exploitability will be recorded in one or more botnets and will see wider exploitation.

 

Why create your own anonymiser?

You can use public anonymiser services such as anonymizer.info or anon.to with Analog CE using one of the code samples below.

ANONYMIZERURL https://anon.to/?

ANONYMIZERURL https://anonymizer.info/?

This may not be acceptable to you, or your organisational security policy. Firstly because while the owner of the resultant web server will not discover the true origin of the request, the public anonymiser service will. Secondly, there is no contract assuring service availability or the privacy of its log files. Finally, it is inevitable that the service is going to profit from your transaction. Advertising placement is likely, creating a delay in the redirect.

 

Code your own Anonymous Link Redirector

The following code snippets can be used to program your own basic redirector using service side scripting technology.

 

ASP 3 / Classic ASP

<%@LANGUAGE="VBSCRIPT" CODEPAGE="65001" EnableSessionState="False"%>
<% Option Explicit %>
<%
  Response.Status = "302 Found"
  call Response.AddHeader("Location", Request.QueryString)
  Response.End()
%>

Save the file as redirector.asp and add the following to your Analog CE global configuration file:

ANONYMIZERURL http://my-server.domain.com/redirector.asp?

 

ASP.net

<%@ Page Language="C#" %>
<script runat="server">
  private void Page_Load(object sender, EventArgs e)
  {
    Response.Redirect(HttpContext.Current.Request.ServerVariables["QUERY_STRING"], true);
  }
</script>

Save the file as redirector.aspx and add the following to your Analog CE global configuration file:

ANONYMIZERURL http://my-server.domain.com/redirector.aspx?

 

PHP

<?php
  header('Location: ' . $_SERVER['QUERY_STRING'], true, 302);
  exit;
?>

Save the file as redirector.php and add the following to your Analog CE global configuration file:

ANONYMIZERURL http://my-server.domain.com/redirector.php?

 

Conclusion

The above code samples illustrate how to create a redirector in several different languages. The redirector URL will be sent to the destination server however the originating statistics page will now be protected. This protects your Analog CE stats pages from prying eyes while reducing the risk of SEO spamming.

Analog’s Search Query Report is empty or sparsely populated

If you are an Analog CE user, you may have noticed that the Search Query and Search Word reports suggest that your site has been receiving little to no search engine traffic. The Search reports may be empty or sparsely populated with results. This article discusses the cause of the problem.

 

What are the External Search Reports?

The premise of the external (rather than internal) search report is to show you what a visitor typed into a search engine before landing on your site. The Search Query report shows the entire search phrase used, while the Search Word report creates a word count report for anything de-marked by a space.

They are intended to help you identify what your users are searching for as part of your SEO activities. Unfortunately, over the last couple of years the data available on the report has been declining. On some sites, the report may have disappeared altogether as no data is available to Analog CE.

 

Fixing the empty or sparsely populated report

There are three reasons for the decline in report quality. They are outlined below in order of lest to most severe.

 

Missing Search Engines

The default config file in the Analog CE source release is a clone of the original Analog 6 config file. It (and your own) config files have not been updated with modern search engines. Consequently traffic from Bing and DuckDuck Go (and others) will be wholly absent.

I will be including an updated config file sample in Analog CE starting with Analog CE 6.0.16. Analog CE users will need to replace or merge this into your running configuration files. To perform this manually, you must add relevant SEARCHENGINE entries to your global/site config files.

 

HTTP > HTTPS

As a result of security issues and the gross mismanagement of private data transiting public networks by the CIA. The world made a rapid switch from HTTP to HTTPS in 2015. The default Analog 6 config file only included Search Engine detection definitions for unencrypted HTTP connections. Consequently Analog CE has stopped reporting traffic from any HTTPS search engine source.

Starting with Analog CE 6.0.16, I will fix this in the bundled sample file. You will need to merge the SEARCHENGINE changes into you respective config files.

 

Privacy & Commercialisation of Analytics

The crux of the problem stems from Search Engines no-longer send the search query in their referrer header. Some see this as being for legitimate privacy reasons, others for commercial benefit. DuckDuck Go would certainly argue the former. For Google, the reason is less clear. With up to 98% market share, Google is effectively forcing webmasters to use Google Search Console and/or Google Analytics to understand their user needs. In turn, this feeds Google with more accurate data on user activity for ad-profiling. It also ensures that you as a webmaster are creating the web that Google wants, above any other concern. Any webmaster threatened with de-listing due to mobile device incompatibility will understand this.

The following sample log line is from the 5th April 2015

2015-04-05 15:38:29 W3SVC5 web02 xxx.xxx.xxx.xxx GET /downloads/msie/ie60sp1/ - 80 - yyy.yyy.yyy.yyy HTTP/1.1 Mozilla/4.0+(compatible;+MSIE+4.01;+Windows+98) - http://www.google.nl/search?q=internet+explorer+download+windows+98&ie=ISO-8859-1&hl=nl&source=hp&gbv=1 www.hpcfactor.com 200 0 0 10747 331 406

This is a similar one from 4th June 2019

2019-06-04 00:17:31 W3SVC5 web01 xxx.xxx.xxx.xxx GET /hardware/devices/specification.asp d=142 443 - yyy.yyy.yyy.yyyy HTTP/2 Mozilla/5.0+(Linux;+Android+6.0;+HM-G552-FL+Build/MRA58K)+AppleWebKit/537.36+(KHTML,+like+Gecko)+Chrome/56.0.2924.87+Mobile+Safari/537.36 - https://www.google.com/ www.hpcfactor.com 200 0 0 12534 491 495

If you scroll to the right, you will notice that in the earlier file, the user searched for “Internet Explorer Download Windows 98”. In the latter file, Google only informs the server that the request came from google.com. The context of the search is no longer offered.

As a consequence, the data required to create the Search Query report is not available to Analog CE. This is the reason why the Analog Search Query Report is empty or sparsely populated. It is not a bug. Rightly or wrongly, it is a symptom of the modern web.

Using Analog C:Amie Edition to generate live statistics for inclusion on web pages

System Requirements:

  • Analog C:Amie Edition

The Problem:

While it isn’t very common these days, and is generally considered to be somewhat crass, there are occasions where you might want to display statistical information in terms of page views or hit counts directly on a web page – for example you may have an administrative view for your website, be running a competition and need to track hits on specific files or just want to tell the world how popular you are.

This article outlines how to use Analog C:Amie Edition’s XML mode to do this.

More Info

By default most users will either generate the HTML output for display or Computer output for piping into ReportMagic when configuring Analog. There are however other options for the structure of the data export as originally implemented by Stephen Turner. One of which allows the export of statistics into XML which you can use to parse for live statistical data and ultimately include on a web page.

How-to

The how-to is split into the following sections:

  1. What is not covered here
  2. Pre-requisites
  3. File System Considerations
  4. Prepare Analog
  5. Configure the Analog Server Settings
  6. Running Analog on a Schedule
  7. Displaying the Page Statistics
  8. Final Considerations

What is not covered here

This guide is specifically written to demonstrate the configuration of Analog and associated web scripts running under IIS on Windows when used in conjunction with ASP 3. The process is equally possible in PHP, JSP or Ruby as well as .net however only ASP 3 VBScript examples are given.

Pre-requisites

Before you begin, you will need to be running Analog C:Amie Edition 6.04 or higher as there is a bug in the XML DTD of lower versions of both Analog and Analog C:Amie edition.

File System Considerations

You need to consider where you place the XML output from Analog. While placing it in the public web root is entirely possible, this will make it potentially available to anyone who happens to become aware of its existence. Visibility of its existence is highly possible when debugging or in the event that there is an error on the part of the parser, file system or file system permissions.

It is therefore suggested that you keep the output XML file outside of the publicly accessible web root. A good candidate for this would be in the same location as your analog.cfg file.

For the rest of this example the following structures will be assumed

Public Web Root D:\sites\www.domain.com\web\
Log Files D:\sites\www.domain.com\logs\W3SVC1\
Configuration (.cfg) and Output Destination (.xml) D:\sites\www.domain.com\
Analog Executable D:\parser\Analog\Analog.exe

Configure the Analog Server Settings

In most cases, you will probably want to maintain your existing statistics output in HTML format for casual browsing. Therefore it is necessary to create a second Analog configuration file which will generate the required output.

The configuration file for Analog’s XML process can be less complex compared to that of the main one, although some customisation to improve parser time and minimise the size of the XML output should be considered essential.

Assuming that an existing Analog .cfg file exists at D:\sites\www.domain.com\logs\analog.cfg we will create a new .cfg file at D:\sites\www.domain.com\logs\analog-xml.cfg.

The example .cfg file shown below outlines an example XML output configuration for our example with paths highlighted for completeness.

# Analog C:Amie Edition XML Statistics Configuration
# Version 1.0.2
# See http://www.c-amie.co.uk/ for updates and moreLOGFILE d:\sites\www.domain.com\logs\w3svc1\*.logOUTPUT XML
OUTFILE d:\sites\www.domain.com\analog.xml
HOSTNAME "Analog Test Site"
HOSTURL http://www.domain.com/
IMAGEDIR "images/"

# Reports Enabled/Disabled List
GENERAL OFF #General Summary
YEARLY OFF #Yearly Report
QUARTERLY OFF #Quarterly Report
MONTHLY OFF #Monthly Report
WEEKLY OFF #Weekly Report
DAILYREP OFF #Daily Report
DAILYSUM OFF #Daily Summary
HOURLYREP OFF #Hourly Report
HOURLYSUM OFF #Hourly Summary
WEEKHOUR OFF #Hour of the Week Summary
QUARTERREP OFF #Quarter-Hour Report
QUARTERSUM OFF #Quarter-Hour Summary
FIVEREP OFF #Five-Minute Report
FIVESUM OFF #Five-Minute Summary
HOST OFF #Host Report
REDIRHOST OFF #Host Redirection Report
FAILHOST OFF #Host Failure Report
ORGANISATION OFF #Organisation Report
DOMAIN OFF #Domain Report
REQUEST ON #Request Report
DIRECTORY OFF #Directory Report
FILETYPE OFF #File Type Report
SIZE OFF #File Size Report
PROCTIME OFF #Processing Time Report
REDIR OFF #Redirection Report
FAILURE OFF #Failure Report
REFERRER OFF #Referrer Report
REFSITE OFF #Referring Site Report
SEARCHQUERY OFF #Search Query Report
SEARCHWORD OFF #Search Word Report
INTSEARCHQUERY OFF #Internal Search Query Report
INTSEARCHWORD OFF #Internal Search Word Report
REDIRREF OFF #Redirected Referrer Report
FAILREF OFF #Failed Referrer Report
BROWSERREP OFF #Browser Report
BROWSERSUM OFF #Browser Summary
OSREP OFF #Operating System Report
VHOST OFF #Virtual Host Report
REDIRVHOST OFF #Virtual Host Redirection Report
FAILVHOST OFF #Virtual Host Failure Report
USER OFF #User Report
REDIRUSER OFF #User Redirection Report
FAILUSER OFF #User Failure Report
STATUS OFF #Status Code Report

# Referring URL Report
REFLINKINCLUDE *
REFREPEXCLUDE http://domain.com/*
REFREPEXCLUDE http://www.domain.com/*
REFFLOOR 10r

# Referring Site
REFSITEEXCLUDE http://domain.com/

# Request Report
REQFLOOR 1r
REQEXCLUDE *.jpg
REQEXCLUDE *.gif
REQEXCLUDE *.png
REQEXCLUDE *.bmp
REQEXCLUDE *.class
REQEXCLUDE *.css
REQEXCLUDE *.js

REQEXCLUDE *.ico

# Status Code Report
304ISSUCCESS ON # Includes 304 errors on the request report

# Custom Exclusions
FILEEXCLUDE /stats/*

Optimisation

It is important to disable any and all unwanted reports, this will speed-up processing of the log files, reduce the size of the output file and reduce the expense of processing and reading data into your website later on. If you are going to follow this example and are only interested in accessing the hit count then the only report that you need to turn on is the REQUEST report.

Additionally, you should configure the REQFLOOR and REQEXCLUDE options to optimise the scope of the output. In most cases REQFLOOR will be set to 1r (1 request) while exclusions should include files that you have no intention or ability to measure. For example, if you are only going to display the hit count of the currently visible .asp file then everything apart from .asp can be excluded from the report.

Running Analog on a Schedule

Now that Analog has been configured, you need to run it. You can run the process manually (including via Task Scheduler) via the command

"d:\parser\analog\Analog.exe" +gd:\sites\www.domain.com\analog-xml.cfg

This will merge any master configuration file with the custom XML .cfg file in generating the output as instructed in the analog-xml.cfg

If you wish to schedule it as part of a larger stats run process, then following the example from my “Using Analog C:Amie Edition to provide automatic statistics on a multi-site production IIS 4.0, 5.0, 5.1, 6.0, 7.0, 7.5 or 8.0 web server” guide, the following script can be used to automate both the parsing of the HTML and XML output

cls
@echo off
SET ALGROOT=d:\parser\Analog\
SET WEBROOT=d:\sitesFOR /f "tokens=*" %%A IN ('dir /b d:\sites') DO (echo Looking for: analog.cfg in %%A

IF EXIST "%WEBROOT%\%%A\analog.cfg" (
echo Found analog.cfg for %%A
md "%WEBROOT%\%%A\web\stats"
echo.
"%ALGROOT%analog.exe" +g%WEBROOT%\%%A\analog.cfg > %WEBROOT%\%%A\analog.log
echo.

IF EXIST "%WBROOT%\%%A\analog-xml.cfg" (
"%ALROOT%analog.exe" +g%WBROOT%\%%A\analog-xml.cfg > %WBROOT%\%%A\analog.log
) ELSE (
echo ANALOG XML NOT FOUND
)

) ELSE (
echo ANALOG CONFIG FILE NOT FOUND IN "%WEBROOT%\%%A\"
echo.
)
)

This script searches all sub-folders of d:\sites for the presence of an analog.cfg file e.g. d:\sites\www.mydomain.com\analog.cfg. If it finds one it ensures that there is an appropriate output directory (\web\stats) and then runs the Analog C:Amie Edition executable using the baseline configuration (this is implicit) and the local site-level analog.cfg configuration file (explicit) to produce the report.

It will then repeat the process, looking for analog-xml.cfg and if present will generate the associated XML output.

Displaying the Page Statistics

Moving forward with our example, we will now have a XML file ready to re-parse located at d:\sites\www.domain.com\analog.xml. The next step is to create a piece of code that will match up the currently displayed web page with its entry in the XML output. The steps to do this are:

  1. Normalise the current page URL
  2. Filter the current page URL
  3. Load the XML file
  4. Query the XML file
  5. Display the output

Normalise the current page URL

Your web servers log file will log anything that it is sent by the browser, often the structure of this is entirely at the mercy of the web user or web service sending the request. You should therefore take precautions to protect the parser and increase the likelihood of finding a match in the XML file.

The following obtains the current URL from ASP’s ServerVariables object, removes unnecessary spaces and enforces that we will only search the XML file using Lower Case URL’s.

Dim strLocalPath
strLocalPath = Request.ServerVariables("URL")
strLocalPath = Trim(strLocalPath)
strLocalPath = Replace(strLocalPath," ", "+")
strLocalPath = LCase(strLocalPath)

Filter the current page URL

Assuming that you are using ASP 3, by default your index page will be identified as “default.asp” (for example www.domain.com/folder/default.asp). It is however possible for a client to access the same page using “/” (for example www.domain.com/folder/ or www.domain.com/folder). Which of these is logged will depend entirely on which of the three (equally valid) options the client chose to use. In theory this should be normalised, however Analog may record impressions for www.domain.com/folder/default.asp, www.domain.com/folder/ and www.domain.com/folder separately, therefore what we need to do is ensure that when we query the XML file later on that we are querying for all valid combinations on your server.

The simplest way to achieve this is to start building a query that assumes we will need all permissible options in order to obtain a valid result.

Dim strSearchSelector
' Normalise the Path String, generate the XPath OR Statement
if ((strLocalPath = "/") OR (strLocalPath = "/default.asp")) then
strSearchSelector = "col=""/"" or col=""/default.asp"""
else
if (Right(strLocalPath, 12) = "/default.asp") then
strLocalPath = Left(strLocalPath, (Len(strLocalPath) - 12))
end ifif (Right(strLocalPath, 1) = "/") then
strLocalPath = Left(strLocalPath, (Len(strLocalPath) - 1))
end if
strSearchSelector = "col=""" & strLocalPath & """ or col=""" & strLocalPath & "/"" or col=""" & strLocalPath & "/default.asp"""
end if

In the above code we first check to see if the use is querying the absolute website home page (either www.domain.com/ or www.domain.com/default.asp) and create a select statement accordingly.

If we are not on the absolute home page, we remove the trailing “/default.asp” from the URL and we remove the trailing “/” from the URL and then construct a select statement that will query for all three possible combinations:

  1. www.domain.com/folder
  2. www.domain.com/folder/
  3. www.domian.com/folder/default.asp

Load the XML file

Now that we have a search query, it is time to load the XML file and configure Microsoft XML to perform the leg work required to find our up to three possible URL combinations for the current page.

Dim xmlDoc
set xmlDoc = Server.CreateObject("MSXML2.DOMDocument.6.0")
xmlDoc.async = False
xmlDoc.setProperty "ServerHTTPRequest", true
xmlDoc.setProperty "ProhibitDTD", false ' Required for ~MSXML 6 (default in 6 = true; <6 = false)
xmlDoc.setProperty "SelectionLanguage","XPath"xmlDoc.validateOnParse = false' Load the specified XML file (returns XML output)
xmlDoc.load("d:\sites\www.domain.com\analog.xml")

In the above example, a Microsoft XML 6.0 object is created with configuration for the use of XPath to execute our query and permission to use the Analog DTD. ServerHTTPRequest is required for processing of synchronous XML documents within ASP.

xmlDoc.load() copies the XML file from the file system, parses it into memory and permits us to progress to the data extraction phase.

Query the XML file

As mentioned above, the weapon of choice in this example will be to query the XML file using XPath syntax. XPath in a somewhat complicated way allows us to walk the DOM hierarchy of an XML file by applying selection filters are we transverse the file towards our desired result.

Without wishing to overcomplicate analysis on how this works or the processes involved, the XPath query required to extract a hit count from the Analog C:Amie Edition 6.04+ DTD is:

/analog-data/report[@name='rep_req']/row[@level='1'][col='/folder/default.asp']/col[@name='col_reqs']

Or in a more logical format:

  1. Select the analog-data parent
  2. Select the report sub tree
  3. Expand all elements named row where the attribute level = 1
  4. Then find sub elements named col where the data (value) equals /folder/default.asp
  5. If all of the above matches return the value of the element col under row under report under analog-data where the attribute name = col_reqs

In our code, we substitute the value of strSearchSelector into the query so that all three valid path combinations are returned

Dim xmlResult
set xmlResult = xmlDoc.selectNodes("/analog-data/report[@name='rep_req']/row[@level='1'][" & strSearchSelector & "]/col[@name='col_reqs']")

Display the Output

The final task is to display the request count. Our xmlResult object does not contain a single value for this, but may contain 3 (or more) values, one for each of our valid paths – more if there is an anomaly in the data.

In order to obtain the actual page request count, we need to aggregate the three values together

Dim lngOut
Dim strOut
if (xmlResult.length > 0) then
for i = 0 to (xmlResult.length - 1)
if (IsNumeric(xmlResult(i).text)) then
lngOut = (lngOut + CLng(xmlResult(i).text))
end if
next
strOut = CStr(FormatNumber(lngOut,0))
else
strOut = "0"
end if

In this code segment, if there are no valid results in the XML file, we return “0”. Else, we read the value of each of the results into a variable lngOut, adding each time. We then convert this number into a formatted string (adding comma’s as the thousands separator) and can return the actual page request count as defined by analog.

The complete code sample

Bringing the above segments together, the final algorithm should look something like this:

' Analog XML Stats Printer Function
' © C:Amie. All Rights Reserved. 1996 - 2013
' http://www.c-amie.co.uk/
' Not for resale or use in commercial profit making activities. Use of this script sample is permitted as long as attributions is maintained.' Call the function and write the value
Response.Write("The Request Count Is: " & getPageHits(Request.ServerVariables("URL")) )Function getPageHits(ByVal strLocalPath)
Dim i
Dim xmlDoc
Dim xmlResult
Dim lngOut
Dim strSearchSelector
Dim strOut

strLocalPath = Trim(strLocalPath)
strLocalPath = Replace(strLocalPath," ", "+")
strLocalPath = LCase(strLocalPath)

' Normalise the Path String, generate the XPath OR Statement
if ((strLocalPath = "/") OR (strLocalPath = "/default.asp")) then

strSearchSelector = "col=""/"" or col=""/default.asp"""

else

if (Right(strLocalPath, 12) = "/default.asp") then
strLocalPath = Left(strLocalPath, (Len(strLocalPath) - 12))
end if

if (Right(strLocalPath, 1) = "/") then
strLocalPath = Left(strLocalPath, (Len(strLocalPath) - 1))
end if
strSearchSelector = "col=""" & strLocalPath & """ or col=""" & strLocalPath & "/"" or col=""" & strLocalPath & "/default.asp"""
end if

set xmlDoc = Server.CreateObject("MSXML2.DOMDocument.6.0")
xmlDoc.async = False
xmlDoc.setProperty "ServerHTTPRequest", true
xmlDoc.setProperty "ProhibitDTD", false ' Required for ~MSXML 6 (default in 6 = true; <6 = false)
xmlDoc.setProperty "SelectionLanguage","XPath"
xmlDoc.validateOnParse = false
' Load the specified XML file (returns XML output)
xmlDoc.load("d:\sites\www.domain.com\analog.xml")
if (xmlDoc.parseError.errorCode <> 0) then
strOut = "Error in parsing XML file"
else
set xmlResult = xmlDoc.selectNodes("/analog-data/report[@name='rep_req']/row[@level='1'][" & strSearchSelector & "]/col[@name='col_reqs']")
if (xmlResult.length > 0) then
for i = 0 to (xmlResult.length - 1)
if (IsNumeric(xmlResult(i).text)) then
lngOut = (lngOut + CLng(xmlResult(i).text))
end if
next
strOut = CStr(FormatNumber(lngOut,0))
else
strOut = "0"
end if
set xmlResult = nothing
end if
set xmlDoc = nothing
getPageHits = strOut
End Function

Final Considerations

Now that you have the ability to extract statistics from analog, you can get creative. You can pass a URL into the function getPageHits(“/downloads/myfile.exe”) to extract the request count for other content that is not directly attached to the current URL of the page.

You should also consider how frequently you want to generate updated statistics. You could run Analog continually in a loop if you wanted to generate near real-time statistical data, every 2 hours or daily (or anything in between). Windows Task Scheduler or cron under unix is the easiest way to schedule updates in most cases. Don’t forget to balance the processing load requirements of the server against the size of the log file store that you are scanning. You may also want to consider limiting the amount of logs in the statistics to the “last year” or “last month”. The logs for C:Amie (not) Com go back to January 2002, HPC:Factor’s repository is even older than that and so it would be utterly impractical for a production web server to undertake real-time scanning activities.

With a dedicated log parser server however…?

See Also

View: Analog C:Amie Edition Configuration File Generator

View: Using Analog C:Amie Edition to provide automatic statistics on a multi-site production IIS 4.0, 5.0, 5.1, 6.0, 7.0, 7.5 or 8.0 web server