How to read web server log files

A wealth of information about the activities of your visitors is available from your web server log files. The best method to obtain key marketing information from log files is to use a log file analyzer which orders the data into meaningful statistics.

A typical log file analyzer gives insights into visitor behavior to allow you to improve overall site performance.

It extracts from your log files, information about web site visitor patterns, referring sites, search terms, demographics, visitor paths and more. However, it is important to understand how to read the raw log files so let us examine a typical entry:

Typical log file entry

Web server log file entries typically look similar to this:

212.209.212.66 - [29/Jul/2001:00:35:33 -0500] "GET /data-mining.htm HTTP/1.1" 200 11631 "http://internetmarketingengine.com/" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"

Note that some of these entries may be in a different order in your log files.

Log file entry where the IP address has been resolved

Some web servers are set to automatically resolve IP addresses by conducting a Whois lookup. Here is how the above entry would look if that were done:

Automat Annonsbyra - [29/Jul/2001:00:35:33 -0500] "GET /data-mining.htm HTTP/1.1" 200 11631 "http://internetmarketingengine.com/" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"

What does this log file entry mean?

212.209.212.66 = IP Address (or XX if the IP address has been resolved)

29/Jul/2001:00:35:33 = Date and Time of the entry

-0500 = Time difference to Greenwich Mean time (Universal Time). This log file entry was created when the web server was on US Central Summer time

GET = Action

data-mining.htm HTTP/1.1 = Object - i.e. retrieve the page data-mining.htm

200 = result (Result 200 means the task has been completed)

11631 = size of object, in bytes

http://internetmarketingengine.com/ = Referring URL (i.e. this particular page was accessed from the home page of the Internet Marketing Engine)

Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) = Browser / version and platform - i.e. this person was using Microsoft Internet Explorer 5.5 and the Windows 2000 operating system.

How do I know when the visitor has found the page through a search engine?

This is how the same log file entry would look had the visitor found the page by conducting a search in Google using the search term "data mining":

12.91.124.214 - [29/Jul/2001:00:35:33 -0500] "GET /data-mining.htm HTTP/1.1" 200 11631 "http://www.google.com/search?q=%22data+mining%22&hl=en&safe=off&start=10&sa=N" "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0)"