I recently moved some sites from ispconfig2 to ispconfig3 and only realised after a week that I had forgotten to first copy over the awstats data files and now have some web logs archived and new stats pages created.

So an idea came to mind to script something together to reprocess all logs for all sites or just a single site using a -d {www.website.tld} parameter and it worked like a charm.

First thing that needed to be done is clear all the awstats*.www.website.tld.txt files from the current/new site so we can copy over from the old server to thenew server.

On current/new ispconfig3 server :

1. rm /var/lib/awstats/awstats*.www.website.tld.txt # change the path to your awstats data dir.

Now copy old awstats files to ispconfig3 server.

2. scp ispc2server:/var/lib/awstats/awstats*.www.website.tld.txt /var/lib/awstats/

Now we have the old data and now just need to reprocess the new server logs. Using my script, here are the 2 methods.

3a. php awstats_log_reprocess.php # Reprocess all websites, this shouldn't damange current logs as it'll ignore already processed entries.

3b. php awstats_log_reprocess.php -d www.website.tld # This will process a single site only, good for a test run.

This script worked well for me, only catch which I'd like to warn you now is that since processing is done on a daily basis via cron, you may want to run a /usr/bin/perl /var/lib/awstats/awstats_updateall.pl -awstatsprog=/usr/lib/cgi-bin/awstats.pl now as some entries in your logs may not be processed when you moved to the new ispconfig3 server. This caught me out but only lost half a days info.

Download 1.0

1

NOTE: Please be aware this was done on Debian Lenny. CentOS and others may require path changes, etc.

NOTE: This HAS been confirmed to work up until version 3.0.3 of ISPConfig.

There are some settings you need to set in the file as other things installed before this will work.

1. Create the awstats apache2 config. Be aware the "AuthUserFile" location should match the file location in the awstats_plugin.inc.php file ($globalpasswordfile).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

# vi /etc/apache2/conf.d/awstats.conf
 
 
 
# cgi enabled
<Directory /usr/lib/cgi-bin/>
Order allow,deny
Allow from all
 
AuthType Basic
AuthName "Site Client Access Only"
AuthUserFile /usr/lib/cgi-bin/.htpasswd_stats
<limit GET PUT POST>
require valid-user
</limit>
AddHandler cgi-script .cgi
AddHandler cgi-script .pl
 
</Directory>
 
<Directory /usr/share/awstats/icon/>
Order allow,deny
Allow from all
</Directory>
 
Alias /awstats-icon/ /usr/share/awstats/icon/
ScriptAlias /ispcstats/ /usr/lib/cgi-bin/

2. copy the plugin file awstats_plugin.inc.php into the /usr/local/ispconfig/server/plugins-available/ directory.

3. # ln -s /usr/local/ispconfig/server/plugins-available/awstats_plugin.inc.php /usr/local/ispconfig/server/plugins-enabled/

4. Add the awstats service anywhere inside the config file. (just add it near the end)

1

$conf['services']['awstats'] = true;

Some parameters need changing within the plugin. The main one is at least the "update_all_domains", first time you may want this set to true to populate all domains which aren't configurd yet saving you clicking each one but there's no problem leaving this one as "true".

I used my own logging and could have used the built in ispc3 ones but long story short, I could get logging working until a little slap in the head later, I realised why but it was already in place.... so be it.

$globalpasswordfile should match the path using in your /etc/awstats/conf.d/awstats.conf, so you can move it to a secure location and making changes to both will make it still work.

$admin_pass will not be used if the setting is empty. When it's set with a password this will activate the global ispc3admin user account to all domains (you need "$update_all_domains=true" to update domains if enabled later).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

####### START OF PER SETUP CONFIGURABLE PARAMETERS ########
var $admin_name = "ispc3admin";
var $admin_pass = ""; # Empty means there will be no superadmin account.
 
var $globalpasswordfile = '/usr/lib/cgi-bin/.htpasswd_stats';
 
var $update_all_domains = false;
// true = all regardless of which was added/updated
// false = only the single domain which was added/updated
 
var $update_all_clients = false;
// true = all regardless of which was added/updated
// false = only the single user which was added/updated
 
var $logging = true;
 
var $log2syslog = true;
// true = linux syslogs (need i say more?)
// false = use normal logfiles specified settings below
 
var $logfile = "/var/log/ispc3-awstats.log";
var $logtype = "increment";
// increment = incrementally add updates (this can be very verbose if 'all/all' set above
// overwrite = new updates will overwrite the log file (good for 'all/all' setting above to see overall configs)
####### END OF PER SETUP CONFIGURABLE PARAMETERS ########

Edit the /usr/local/ispconfig/server/cron_daily.php and find the line below (since 3.0.2.1 it's around line 147) and add the awstats command after it.

PLEASE NOTE: ispconfig upgrades will overwrite this file. You could move /usr/bin/webalizer out of the way and replace it with a script that does either or both updates. Refer to "safe upgrades" below.

1
2
3
4
5

if(!@is_dir($statsdir)) mkdir($statsdir);
echo "$domain -> $logfile\n";
exec("$webalizer -c $webalizer_conf -n $domain -s $domain -r $domain -q -T -p -o $statsdir $logfile");
exec("/usr/lib/cgi-bin/awstats.pl -update -config=$domain -LogFile=$logfile >/var/log/cron_daily.awstats.log 2>&1"); // ADD THIS LINE
}

1

AllowAccessFromWebToAuthenticatedUsersOnly=1

Safe Upgrades

Modifying the crontab_daily.php in step 5 will always be required after ispc3 update but it's the best method to process the log before cron daily archives the log making it harder to do later.

You could make a script called /usr/bin/webalizer and rename the webalizer bin file to webalizer-real and in the script add this below. This will update both webalizer (if you want to keep that) and awstats but also allow ispconfig updates as the php cron script isn't modified. not tested but technically should work.

1
2
3

#!/bin/bash
/usr/bin/webalizer-real $@
/usr/lib/cgi-bin/awstats.pl -update -config=$4 -LogFile=$14 >/var/log/cron_daily.awstats.log 2>&1

Download 1.1

If you feel this was helpful to you and would like to donate $5 or any figure you feel, please help my research/work by donating below.

Keep my research alive.