Satrting April 29, 2014, I saw the following error from webalizer cron job:
Code: Select all
Error: Skipping oversized log record
So I took some time and checked what's going on.
It was reported by webalizer when it finds the referral field in log file that is too long.
I then tried to increase the limit but I was unable to compile it due to lack of libgd (-lgd).
I tried to recompile GD but it failed too due to missing freetype2 header ftheader.h.
I then downloaded freetype2-2.5.3 and tried to create Makefile by "sh autogen.sh" but it complained my libtoolize is too old (1.5.2) so I downloaded 2.4.2 and configured and compiled. It worked.
In order for freetype2 to be compiled, I need to have a Makefile. It has a different way of creating the Makefile. No need to use 'configure'. It needs to use the following command to create the Makefile:
That means I have to get the cmake package. I tried to download the source and compile but it didn't work so I grabbed the package from Oracle. I had to use -v to install the package or it will reject it.
After cmake (2.8.6) was installed, I was able to run the following commands to compile freetype2.5.3.
Code: Select all
cmake CMakeLists.txt
make
make install
Then, I tried to compile GD 2.0.35.
I had to specify the new freetype2 header path before it would configure right.
Code: Select all
configure --with-freetype=/usr/local/include/freetype2
make
make install
When I tried to compile webalizer, it still complained about missing libgd. I figure it may be time to get the latest webalize and try so I went to download the 2.23.08 version. Still no luck.
I then took a look at libgd library files in /usr/lib.
I saw:
Code: Select all
lrwxrwxrwx 1 root root 29 Apr 2 2012 libgd.so.2 -> /usr/local/lib/libgd.so.2.0.0
I didn't see libgd.so's symbolic link. So I checked png's lib files:
Code: Select all
lrwxrwxrwx 1 root root 11 Mar 25 2012 libpng.so -> libpng14.so
lrwxrwxrwx 1 root root 16 Mar 25 2012 libpng.so.2 -> libpng.so.2.56.0
From my experience, libgd.so should be there or compilation would complain (cannot find the library). So I created a symbolic link for it.
Code: Select all
lrwxrwxrwx 1 root root 29 May 1 12:50 libgd.so -> /usr/local/lib/libgd.so.2.0.0
lrwxrwxrwx 1 root root 29 Apr 2 2012 libgd.so.2 -> /usr/local/lib/libgd.so.2.0.0
Then, webalizer found -lgd.
It is because the symbolic was missing. I didn't have to do all the work above!!
So configure was able to create makefile finally. However, it was unable to compile.
Code: Select all
cahtoh02:/work/applications/webalizer-2.23-08%make
gcc -Wall -O2 -DETCDIR=\"/usr/local/etc\" -DGEODB_LOC=\"/usr/share/GeoDB\" -DPACKAGE_NAME=\"webalizer\" -DPACKAGE_TARNAME=\"webalizer\" -DPACKAGE_VERSION=\"V2.23\" -DPACKAGE_STRING=\"webalizer\ V2.23\" -DPACKAGE_BUGREPORT=\"\" -D_FILE_OFFSET_BITS=64 -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -Du_int64_t=unsigned\ long\ long -DHAVE_GETOPT_H=1 -DHAVE_MATH_H=1 -DHAVE_SYS_SOCKET_H=1 -DUSE_DNS -DHAVE_ALTZONE -c webalizer.c
In file included from webalizer.c:71:0:
/usr/include/db.h:74:23: error: duplicate 'unsigned'
/usr/include/db.h:74:23: error: 'long long long' is too long for GCC
*** Error code 1
make: Fatal error: Command failed for target `webalizer.o'
For some reason, it doesn't like this definition:
So, I made a copy of the original db.h and created a new db.h and commented out this line:
Code: Select all
/*typedef unsigned long u_int64_t;*/
Then, it worked.
Still, the error is showing.
I looked into the source code (webalizer.c) again and here are the codes related to the error:
Code: Select all
if (verbose)
{
fprintf(stderr,"%s",msg_big_rec);
if (debug_mode) fprintf(stderr,":\n%s",buffer);
else fprintf(stderr,"\n");
}
I then checked the definition of verbose.
Code: Select all
int verbose = 2; /* 2=verbose,1=err, 0=none */
int debug_mode = 0; /* debug mode flag */
By default, it is verbose mode and debug is turned off.
I then saw:
Code: Select all
case 'q': verbose=1; break; /* Quiet (verbose=1) */
case 'Q': verbose=0; break; /* Really Quiet */
That means parameters can be passed on to the binary. I then checked the help from webalizer.
Code: Select all
%webalizer -h
Usage: webalizer [options] [log file]
-h = print this help message
-V = print version information
-v = be verbose
-d = print additional debug info
-F type = Log type. type= (clf | ftp | squid | w3c)
-f = Fold sequence errors
-i = ignore history file
-p = preserve state (incremental)
-b = ignore state (incremental)
-q = supress informational messages
-Q = supress _ALL_ messages
-Y = supress country graph
-G = supress hourly graph
-H = supress hourly stats
-L = supress color coded graph legends
-l num = use num background lines on graph
-m num = Visit timout value (seconds)
-T = print timing information
-c file = use configuration file 'file'
-n name = hostname to use
-o dir = output directory to use
-t name = report title 'name'
-a name = hide user agent 'name'
-r name = hide referrer 'name'
-s name = hide site 'name'
-u name = hide URL 'name'
-x name = Use filename extension 'name'
-O name = Omit page 'name'
-P name = Page type extension 'name'
-I name = Index alias 'name'
-K num = num months in summary table
-k num = num months in summary graph
-A num = Display num top agents
-C num = Display num top countries
-R num = Display num top referrers
-S num = Display num top sites
-U num = Display num top URLs
-e num = Display num top Entry Pages
-E num = Display num top Exit Pages
-g num = Group Domains to 'num' levels
-X = Hide individual sites
-z dir = Use country flags in 'dir'
-D name = Use DNS Cache file 'name'
-N num = Number of DNS processes (0=disable)
-j = Enable native GeoDB lookups
-J name = Use GeoDB database 'name'
I then tried to run webalizer from command line and found openwebmail is the one that shows the error message.
I tried a few commands and here is the finding:
Code: Select all
%/usr/local/bin/webalizer -c /export/home/cah/webalizer/openwebmail.conf
Webalizer V2.23-08 (SunOS 5.11 i86pc) English
Using logfile /export/home/cah/logs/access_log_openwebmail (clf)
Creating output in /export/home/www/html/openwebmail/stats
Hostname for reports is 'openwebmail.hsiao.net'
Reading history file... /export/home/cah/webalizer/webalizer_openwebmail.hist
Reading previous run data.. webalizer.current
Error: Skipping oversized log record
Saving current run data... [05/01/2014 14:12:17]
Generating report for May 2014
Error: Unable to open file usage_201405.html!
Saving history information...
Generating summary report
78288 records (78116 ignored, 1 bad) in 1 seconds, 78288/sec
Code: Select all
%/usr/local/bin/webalizer -q -c /export/home/cah/webalizer/openwebmail.co>
Error: Skipping oversized log record
Code: Select all
/usr/local/bin/webalizer -Q -c /export/home/cah/webalizer/openwebmail.co>
%
From the source code, I thought the debug function may help so I tried the following command:
Code: Select all
%/usr/local/bin/webalizer -q -d -c /export/home/cah/webalizer/openwebmail>
Error: Skipping oversized log record:
nd223008.global.medtronic.com - - [29/Apr/2014:11:35:17 -0400] "GET /cgi-bin/openwebmail/openwebmail-abook.pl?action=addrlistview&sessionid=cah*openwebmail.hsiao.net-session-0.21543475154704&abookkeyword=&abooksearchtype=&listviewmode=composeselect&to=&cc=&bcc=%22Bill%20Holland%22%20%3CHarrisonwhh@aol.com%3E%2C%20%22Bin%20%u658C%20Wu%20%u5433%22%20%3Cwuhan05@gmail.com%3E%2C%20%22Brent%20Morrow%22%20%3Cwaymor@gmail.com%3E%2C%20%22Charles%20Smith%22%20%3Ccharles.smith4@fedex.com%3E%2C%20%22Creighton%20Van%20Horn%22%20%3Ccreighton.vanhorn@hvhtek.com%3E%2C%20%22Daniel%20Kleinman%22%20%3Cdank@att.com%3E%2C%20%22Dilip%20Vazirani%22%20%3Cdvazirani@yahoo.com%3E%2C%20%22DJ%20Burdette%22%20%3Cdj_burdette@hotmail.com%3E%2C%20%22Erik%20Fuller%22%20%3Ceefuller@yahoo.com%3E%2C%20%22Eugene%20Tseng%20%u66FE%22%20%3CETseng@aol.com%3E%2C%20%22Family%20Members%22%20%3Cfamily@hsiao.net%3E%2C%20%22Feng%20Lin%20%u6797%22%20%3Cfuanfeng@yahoo.com%3E%2C%20%22Frank%20%u5F37%20Fu%20%u5085%22%20%3CFrank.Fu@dmv.ca.gov%3E%2C%20%22George%20Patterson%22%20%3Cgeorgewpatterson@yahoo.com%3E%2C%20%22James%20Ray%22%20%3Cjamesray@charter.net%3E%2C%20%22Jeff%20Imes%22%20%3Cjeff.imes@gmail.com%3E%2C%20%22Jeffri%20Johari%22%20%3Cg_eurasian@yahoo.com%3E%2C%20%22Jenny%20Dennin%22%20%3Ct.dennin@sbcglobal.net%3E%2C%20%22Jim%20Gaston%22%20%3Cmachus1@gmail.com%3E%2C%20%22Josh%20Thorin%20Messer%22%20%3Cme@thorinmesser.com%3E%2C%20%22Khanh-Linh%20Huynh%20%u9EC3%22%20%3Cbljkqueen@hotmail.com%3E%2C%20%22Krizelle%20Dizon%22%20%3Ckrizelledzn@yahoo.com%3E%2C%20%22Liang%20Wei%20%u9B4F%22%20%3Clwei2k1@yahoo.com%3E%2C%20%22Linda%20Hong%20%u6D2A%22%20%3Cpeyyen_hong@yahoo.com%3E%2C%20%22LiPin%20Tan%20%u8B5A%22%20%3Clipintan@hotmail.com%3E%2C%20%22Mike%20Criswell%22%20%3Cmlc@columbus.rr.com%3E%2C%20%22Nick%20Chen%20%u9673%22%20%3Cnickchen98@gmail.com%3E%2C%20%22Paul%20Wang%20%u738B%22%20%3Cpaulpwang@aol.com%3E%2C%20%22Pengcheng%20Jia%20%u8CC8%22%20%3Cjiapc@yahoo.com%3E%2C%20%22Peter%20Khin%22%20%3Cpeter.khin@gmail.com%3E%2C%20%22Ravi%20Kalidindi%22%20%3Ckalidindi_ravi@yahoo.com%3E%2C%20%22Ray%20Chiou%20%u90B1%22%20%3Crccama@gmail.com%3E%2C%20%22Richard%20Beals%22%20%3Cbealsrg@gmail.com%3E%2C%20%22Rudy%20Santacruz%22%20%3Celmostro88@hotmail.com%3E%2C%20%22Saravanan%20Pollachi%22%20%3Cindyhotels@gmail.com%3E%2C%20%22Scott%20%u9D3B%u658C%20Chin%20%u91D1%22%20%3Chpscott.chin@gmail.com%3E%2C%20%22Shaik%20Dawood%22%20%3Csdawood@yahoo.com%3E%2C%20%22Shay%20Logan%22%20%3Cshay@shaylogan.com%3E%2C%20%22Stan%20Khouw%22%20%3Cstank516@gmail.com%3E%2C%20%22Tolu%20Makinde%22%20%3Ctolu_makinde@yahoo.com%3E%2C%20%22Tom%20Siebert%22%20%3Ctomsiebert@aol.com%3E%2C%20%22Tony%20Salony%22%20%3Casalony@comcast.net%3E%2C%20%22Uzy%20Christensen%22%20%3Cuzyoc@yahoo.com%3E%2C%20%22Victor%20Santiago%22%20%3Ckinkos101@yahoo.com%3E%2C%20%22Vijay%20Atmavilas%22%20%3Cvijay.atmavilas@gmail.com%3E%2C%20%22Yandong%20Zheng%20%u912D%22%20%3Cyan91326@gmail.com%3E%2C%20%22%u738B%u73B2%22%20%3Camandawang@hotmail.com%3E%2C%20%22%u5433%u6E05%u548C%22%20%3Cchwu999@gmail.com%3E%2C%20%22%u4F59%u65E5%u5149%22%20%3Ca0918935330@yahoo.com.tw%3E%2C%20%22%u4F59%u627F%u7FF0%22%20%3Cyu.junee@gmail.com%3E%2C%20%22%u674E%u6606%u80B2%22%20%3CKy_Li@pegatroncorp.com%3E%2C%20%22%u674E%u5EFA%u7965%22%20%3Cjclchina@hotmail.com%3E%2C%20%22%u674E%u8ECD%22%20%3Cjunewier@verizon.net%3E%2C%20%22%u6797%u78A9%22%20%3Cshuo@vmstat.org%3E%2C%20%22%u5F90%u6587%u96C4%22%20%3Cmhsucpa@yahoo.com%3E%2C%20%22%u5F90%u69AE%u570B%22%20%3Ckib30220@gmail.com%3E%2C%20%22%u5F35%u552F%u5A1F%22%20%3Cwcwu888_1@msn.com%3E%2C%20%22%u5F35%u6DF5%u667A%22%20%3Cvchang@yahoo.com%3E%2C%20%22%u9673%u70B3%u5091%22%20%3Cbenchen69@hotmail.com%3E%2C%20%22%u9673%u52E4%u5B78%22%20%3Cqinxuec@gmail.com%3E%2C%20%22%u9673%u5922%u541B%22%20%3Cechen67@hotmail.com%3E%2C%20%22%u6E38%u5B50%u9F8D%22%20%3Cjammy@vollmacht.com%3E%2C%20%22%u7A0B%u90C1%u6587%22%20%3Cpooh101573@yahoo.com%3E%2C%20%22%u96F7%u5168%u5FA9%22%20%3Cquanful@hotmail.com%3E%2C%20%22%u8D99%u6C38%u5149%22%20%3Cjeffrey_chao@yahoo.com%3E%2C%20%22%u8521%u7D39%u5B97%22%20%3Cscchoy@gmail.com%3E%2C%20%22%u912D%u5B50%u78A9%22%20%3Cpedodocjc@hotmail.com%3E%2C%20%22%u8607%u82F1%u4EC1%22%20%3Crobsu007@gmail.com%3E HTTP/1.1" 200 18159 "http://openwebmail.hsiao.net/cgi-bin/openwebmail/openwebmail-send.pl?action=composemessage&sessionid=cah*openwebmail.hsiao.net-session-0.21543475154704&sort=date_rev&msgdatetype=sentdate&keyword=&searchtype=subject&folder=INBOX&page=1&compose_caller=main" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0"
This is the root cause of the error message. I was trying to send an email to multiple recipients and that became the referral for some reason.
I have 2 solutions to this error:
1. Ignore the error since it is no harm.
2. Delete the entry from the log file.
Since it doesn't hurt, I will just ignore it.