$Id: BUGS,v 1.263 2008/03/09 20:22:16 vanbaal Exp $ Lire BUGS list. ( severities are as used by the debian bugtracking system: critical makes unrelated software on the system (or the whole system) break, or causes serious data loss, or introduces a security hole on systems where you install Lire. grave makes Lire unuseable or mostly so, or causes data loss, or introduces a security hole allowing access to the accounts of users who use the package. serious makes the package unsuitable for release. important a bug which has a major effect on the usability of Lire, without rendering it completely unusable to everyone. normal the default value, applicable to most bugs. minor a problem which doesn't affect Lire's usefulness, and is presumably trivial to fix. wishlist for any feature request, and also for any bugs that are very difficult to fix due to major design considerations. ) - serious: announce pkgsrc port on announcement list, once stable release is available. - lire >= 2:2.0.2.99.2-1 is ready for upload to Debian unstable - serious: Lire requires GNU Make. Specifically all/po/Makefile.am does. This is tedious for Solaris users (and BSD users, as well). See Date: Sat, 30 Oct 2004 23:13:33 +0200 From: Joost van Baal To: "Francis J. Lacoste", Wolfgang Sourdeau Cc: LogReport Questions List Subject: Lire requires GNU Make (was: Re: the workaround ...) Message-ID: <20041030211333.GO9648@nagy.mdcc.cx> It was reported too in Date: Thu, 3 Aug 2006 10:35:34 +0200 From: Joost van Baal To: lhc Cc: LogReport Development List Subject: Re: Unable to compile lire-2.0.2 under SunOS 5.8 Message-ID: <20060803083534.GJ3428@nagy.mdcc.cx> Either update http://download.logreport.org/pub/current/doc/user-manual/ch02s02.html#sect:standalone-install-req or fix the Makefile.am. - important: Lire fails to build with a non-GNU Make. See http://logreport.org/contact/lists/questions/msg00559.php . Either fix this or document GNU Make in INSTALL as a requirement. This especially hurts the Lire on Solaris community. JvB, 30 October 2004 - important: It seems DBD::SQLite2 v 0.33 (released 09-Aug-2004), which uses SQLite 2.8.12 is unusable on Solaris: bus error. See http://logreport.org/contact/lists/questions/msg00561.php . This seems to make Lire unusable on Solaris. Should we drop Solaris support? JvB, 30 October 2004 - normal: get rid of Debian-centric .3pm extension for manpages: make it configurable. Thanks to Peter Bex for reporting this. JvB, 9 March 2008 - normal: Add support for "Message forwarded from hostname:" tag in Lire::Syslog. Reported by Edward Quick. FLJ, 20 December 2002 - normal: dubious bug in lr_anonymize : Malformed domainnames do not get anonymized. (See also one of the wishlist items below). It can be argued that this really is not a bug, because the domain name is not conform the RFC. However, lr_anonymize already deals with other non-conformant domain names. (where to draw the line...) - normal: sendmail2dlf chokes on some wacky sendmail logs. see the BUGS section in the sendmail2dlf manpage. Reported by Edward Eldred in JvB, 26 July 2002 - normal: Table shouldn't size themselves according to the longest possible entry in the table. It should try to fit *most* entries, long one should be cropped in PDF. FLJ, Jul 8 2003 - normal: all DLF converters should get migrated to the Perl module style setup. Migrated ones are the ones in superservice/lib/fooDlfConverter.pm, they have a supporting /etc/lire/plugins/*_init file. Unmigrated ones live as a script in lib/lire/convertors/*2dlf. JvB, 21 March 2004 - normal: On OpenBSD, running lr_log2report -o html ipfilter < /etc/motd (Lire 2.0) fails with: Formatting report as html in -... tar: Failed open to write on /dev/rst0: Permission denied This is likely due to differences in BSD tar and GNU tar; it might fail with Solaris tar too. JvB, 4 Sept 2004 - normal: Fix bug in lr_log2report's html commandline, see bugs section in manpage. JvB, 4 Oct 2006 - minor: contents of TODO should get merged in this file. JvB, 22 June 2002 - minor: when specifying `0' as parameter in a top-report, the title shows up as `Foo by Bar, Top 0', while it actually no longer is a top-report, but shows all values. JvB/FLJ, 22 June 2002 - minor: documentation needs to get reviewed and completed: - The list of supported applications/log files in the user manual is obsolete/incomplete (though it does mention how to generate an authoritive list) JvB, 7 may 2003 - minor: Handle warning: line in qmail log files nicely instead of rejecting them: Oct 7 10:52:37 hibou lire: email qmail lr_tag-20021007105232-19195 qmail2dlf warning skipping line '1033731266.302474 warning: trouble opening remote/15/1079428; will try again later': first field should be one of 'new', 'info', 'starting', 'delivery', 'end', 'bounce' or 'status:', not 'warning:' FLJ, 10 Oct 2002 - minor: subst-configvars should be replaced by calls to AC_DEFINE_DIR from the Autoconf Macro Archive ( http://www.gnu.org/software/ac-archive/ ): the problem we're trying to solve has been solved by others before! JvB, 27 Apr 2003 - wishlist: drop lr_scale_ configuration variable and make the unit to use fixed so that numbers are comparable in the table. As suggested by Edwin Groothuis make that unit overridable in the report specifications (a unit attribute). FLJ, Nov 12 2002 - wishlist: the fw1_lea2dlf DLF converter requires the Date::Manip module which isn't a standard component. It also requires the use of a helper program available from http://www.fellhauer-web.de/projects/fw1-loggrabber.html FLJ, May 17 2003 - wishlist: the spamassassin2dlf convertor should document exact expected log file format. Bug found during discussion with Tom Northeast. JvB, Aug 24 2005 - wishlist: get reports generated via merging display from which data they where generated. E.g. "Weekly top-m report, generated from daily top-n reports". This in order to give the reader a clue on accuracy. JvB, 23 June 2002 - wishlist: make it possible to specify default output format in configure: ./configure --with-default-output=XXX Make configure fails if the needed requirements can't be found. Asked by Rob Dinoff. - wishlist: make it possible to build Lire on a system which lacks XML::Parser. This is useful for package builders. (The Debian package would no longer have libxml-parser-perl in its Build-Depends-Indep.) configure.in would have to check for some environment variable before failing. JvB, 21 Jul 2002 - normal: Get the Firewall-1 firewall DLF converter properly integrated and tested. Solve the LEA issues. Requested by Oscar Castaneda. JvB, 10 Apr 2003 - wishlist: Adapt WELF firewall convertor to fully support Netasq ( www.netasq.com ). See Date: Sat, 6 Apr 2002 13:01:29 -0500 From: "Francis J. Lacoste" To: Van-Hau TRAN Subject: Re: hibou Online Responder Report for 04/05/02 Message-ID: <20020406180129.GD613@Contre.COM> JvB, May 10 2002 - wishlist: Implement a Watchguard Soho ( http://www.watchguard.com/products/fireboxsoho.asp ) firewall DLF convertor. See From: Ben el Cherb To: logreport@logreport.org Cc: flacoste@logreport.org Subject: Re: hibou Online Responder Report for 04/05/02 Date: Sun, 07 Apr 2002 04:01:06 Message-ID: JvB, May 10 2002 - wishlist: add a convertor for Netscreen ScreenOS 3.0.1's native syslog logs. (We now only support the somewhat broken WELF-style logs from this box.) suggested in a discussion between Francis and Mark D. Nagel, see <03fe01c22ed4$e2b98a30$cc6fa8c0@BLEU> (not publicly archived.) JvB, July 30 2002 - wishlist: Add a "sort" attribute to the records group operation. - wishlist: Improve email reports : - Better error reports (Longer error messages, maybe with failed email split errors by to-relay and from-relay). See also Subject: Re: email reports From: "Francis J. Lacoste" To: development Message-Id: <1028124202.32487.29.camel@Arendt> - Add a summary (with received, sent, forwarded subcategories). - SMTPD reports (connections by hour). - Add total (to_domain and from_domain) domain statistics (requested by Matthias Jung) - Add domain-by-period reports (requested by Matthias Jung). - wishlist: handle postfix/pipe log messages. This is encountered, for example, on hosts that have postfix with the Amavis virus scanner installed, as well as hosts using the cyrus message transport. See also Message-ID: <3B2BE13C.7040101@tbcinc.net> Date: Sat, 16 Jun 2001 17:44:12 -0500 From: Justin Mecham To: LogReport Subject: Re: your log file to log@postfix.logreport.org (Inform Justin once we've fixed this.) - wishlist: handle 'reject:' log line in the postfix converter: Dec 30 07:12:21 smtp postfix/smtpd[680]: reject: RCPT from \ unknown[212.96.116.2]: 554 Service unavailable; [212.96.116.2] blocked \ using relays.ordb.org, reason: This mail was handled by an open relay \ - please visit ; \ from= to= Requested by Andy Thompson on Dec 29 FLJ, 30 Dec 2002 - wishlist: the email dlf format could better have a dlf record for _all_ message-queued events. We now only record the final action taken on the message. Only when the logfile doesn't contain the last action taken on the message at hand, a stat=queued events makes it into the dlf now. Queueing events migth very well be interesting information, and might be needed for some useful reports. Thanks Mark Huizer for reminding me about this. JvB, 31 May 2002 Even better: we should probably redesign the email DLF format to be more in line with what actual email logs contain, that is records for receiving, queued, sending, forwarding, bouncing, etc. events. This could be modelled after the exim logging model that is very clean (compared to all the others we support now). This would simplify the email DLF converters a lot, since the flow analysis can be moved to generic modules instead of having to be reimplemented in each of DLF converter. This would make all sorts of reports possible (bounce, forwarding, anomalies). It is possible to implement this in the current "single schema model" by using something like we did in the firewall superservice where some field are only relevant to IDS type of event and others only to packet accounting type of events. FLJ, 31 May 2002 - wishlist: we should map http requests with escaped url's to their unescaped equivalent in some reports, eg map '%7E' to '~'. See From: "E.L. Willighagen" Date: Tue, 30 Oct 2001 17:20:43 +0100 (MET) Message-Id: <200110301620.f9UGKhh16160@studs3.sci.kun.nl> To: bugs@logreport.org, edwin@mavetju.org Subject: re: hex-encoded pathnames Define a new derived DLF format to support this. The destinction needs to be made in the default dlf format, since apache will give a 404 when given a request for a cgi script with an escaped '&' in the url. See also Debian Bug #291063. Related to this: when writing an XML report, we are not guarded against a buggy report schema writing non-utf8 stuff to the XML. Lire::Report::write_report() assumes all subreports are in UTF-8 format, but does not check for this. It should. If a schema turns out to be buggy in this way, Lire dies with a message like: "not well-formed (invalid token) at line 1004, column 46, byte 53763 at /usr/lib/perl5/XML/Parser.pm line 187". updated by JvB, 16 jul 2006 - wishlist: Get rid of commands in the Subject: field of mails to a responder (currently we support 'anon'). Use subemailaddresses for this, e.g.: log-anon-html@.logreport.org . - wishlist: Get lr_log2mail support multipart/alternative emails, so that a responder could send both ascii and html reports in one response. JvB, 27 may 2002 - wishlist: define a DlfConverter that could parse log files based on field specifications. - wishlist: Make it possible to auto-detect log file types. This could be done by trying all the DlfConverters on a small sample of lines and compare errors/success statistics. - wishlist: implement a mechanism to translate IP address to FQDN. Two possibilities: 1) A lr_getaddrinfo filter which works on the DLF 2) A lr_xml2resolver filter which works on the XML report. Useful for DNS, firewall, www superservice (at least). See Date: Wed, 20 Oct 2004 12:53:19 +0200 From: Joost van Baal To: Jeffeny Hoogervorst Cc: LogReport Questions List Subject: Re: Question about logreport (resolving hostnames) Message-ID: <20041020105319.GM8544@nagy.mdcc.cx> archived at http://logreport.org/contact/lists/questions/msg00545.php . - wishlist: improve ftp reports: 1: Sort by file. It would display each time a particular file was accessed by a particular person and it would list the time. 2: Sort by user and file. It would display each time a particular file was accessed by a particular person. 3: Sort by user. It would show all files and times accessed by a particular user. reported by Brett Simpson on Thu, 15 Nov 2001 09:46:59 -0500, in Message-Id: , Subject: FTP improvement questions Furthermore: 1: Show the number of failed logins. 2: Show who failed to login and at what time. 3: Show the number of successful logins. 4: Show who was successful to login and at what time. 5: Differentiate between downloads and uploads. reported by Brett Simpson on Wed, 21 Nov 2001 09:59:08 -0500 in Message-Id: , Subject: Another idea for FTP reporting. This might better be handled in a new 'login' super service, however. JvB. - wishlist: improve www reports. See a comparison with our competitors, mentioned in Date: Tue, 11 Mar 2003 10:26:05 +0100 From: Wytze van der Raay To: people Cc: Stichting LogReport Subject: FYI: comparison of web access log analyzers Message-ID: <20030311102605.A24779@navarro.bebop.nlnet.nl> . JvB, 2003-11-20 - wishlist: document this: - why do you advise to create a dedicated user account? why not 'nobody' or 'daemon'? - wishlist: Define a cross-superservice report for virtual hosts. Requested by Jens Ott, Lars Magerkohl. FLJ, 24 Mar 2002 4 Jun 2002 - wishlist: Make Lire conforms to the NIST standard and uses GiB, MiB and KiB for size units and G, M and B for non-size fields. See http://kerneltrap.com/article.php?sid=434&target=new. Egonw Dec 27 2001 - wishlist: New email service: Exchange 2000. Requested by Andy Van den Heede. FLJ, Apr 06 2002 - wishlist: Support for IPv6 addresses in Lire. (E.g. the bind9 service and the lr_anonymize script.) Maybe we should put in a Lire::LogRegex module common regular expressions that could be used across service to parse IP addresses and other common regular expressions. Kimura Fuyuki's patch for Regexp::Common adds IPv6 support to this CPAN module. See <86elfmjy7r.wl@sz.homedns.org> . Requested by Kimura Fuyuki. FLJ, Jun 04 2002 - wishlist: Better layout in Excel output format (uses border around surreports, contrasting row colours, etc.) - wishlist: Define a login and daemon schema that could probably be used by lot of services. See message <1028130934.32593.80.camel@Arendt> to the development mailing list. FLJ, July 31 2002 - wishlist: Define a new user feedback interface that could be used to inform the users about the progress of the importation, analysis, report generation etc. Backend for syslog, stderr and Lire::UI could be made. FLJ, Aug 31 2004 - wishlist: Add menu items to import a log file, run an analysers and generate a report in Lire::UI. FLJ, Aug 31 2004 - wishlist: Add a store argument to the lr_log2mail, lr_log2report instead of always using a temporary one. FLJ, Aug 31 2004 - wishlist: Add a lr_import_log command that could import a log file into an existing DlfStore. FLJ, Aug 31 2004 - wishlist: rename the `dns' superservice to `dnsquery': more descriptive name. JvB, 16 Aug 2002 - wishlist: make bind8_query, bind9_query and qmail DLF converters support more log formats. These log files may be encapsulated in syslog log files. qmail can use other timestamp formats. FLJ, 16 Aug 2002 - wishlist: get rid of the term `address': it is used at some places, to refer to an online responders' address. That'd better be called `service', since these belong to exactly the same namespace. The file address.cf could better be named service_map.cf. JvB, 18 Aug 2002 - wishlist: qmail2dlf should support more timestamp formats and syslog encoded log files. FLJ, Sep 01 2002 - wishlist: provide configuration hooks to normalize usernames (for case-insensitive systems for example). Suggested by Brett Hales. FLJ, Nov 12 2002 - wishlist: support for Microsoft Internet Connection Firewall log files. Suggested by Diego Quintela. FLJ, Nov 23 2002 - wishlist: support for Real Server logfile format. Spec is available from http://service.real.com/help/library/guides/g270/htmfiles/report.htm#44459 Requested by Frank Elsner on questions mailing list. FLJ, Nov 29 2002 - wishlist: support for bind query log files logged through syslog. FLJ, Dec 03 2002 - wishlist: update argomail parser to support latest log files (version 1.8.1.9 log file) break because of different greeting message format and use of EHLO. FLJ, Jan 16 2003 - wishlist: add possibility to generate a cover sheets from a bunch of reports. Suggested by Stewart James (see http://news.its.vu.edu.au/reports/ for a cover sheet sample). FLJ, Feb 21 2003 - wishlist: Support the 'show-timezone' PIX option which adds a timezone string to the log. An example can be found in the email sent by Amit Sood to our bugs alias. FLJ, Feb 28 2003 - wishlist: Add support for OpenBSD pf firewall logs. Requested by Paul Weissmann. FLJ, Feb 28 2003. - wishlist: Add an option to display the timestamps in another timezone than the one of the log file. Requested by Kevin Krumwiede. This could be an additional propertiy on ImportJob and ReportJob. FLJ, Apr 11 2003. - wishlist: in Lire::UI, display the store manipulation utility through a tabbed interface so as to reduce the amount of widgets displayed. This would enable an easier handling of the widgets as well as a reduction of contraints related to the screen estate. WAS, Aug 31 2004. - wishlist: if a integer range contains only one integer, it should be listed as "n", not as ["n"-"n+1">. E.g. in User Sessions by Their Recurrence Visit # Sessions % Total ------------------------------------------------------- -------- ------- [1-2> 190 68.3 [2-3> 35 12.6 [3-4> 15 5.4 . Reported in From: "E.L. Willighagen" To: LogReport Support Subject: Re: logreport.com on hibou 20041024 Date: Mon, 25 Oct 2004 08:47:20 +0200 Cc: people Message-Id: <200410250847.21559.e.willighagen@science.ru.nl> EW, Oct 25 2004 - wishlist: make Lire easier extensible. E.g., it'd be nice if DLF files could be exported in their ascii representation from a DLF store. And it'd be nice if we had a CSV report format: that's machine parsable for people who don't know XML and/or Perl. Some people _are_ using old-style Lire ways to do these kind of things, e.g. in situations where just a part of Lire is used. JvB, Oct 13, 2004 - wishlist: some suggestion items on what could be part of a 2.1 release. (In no particular order) a. Convert remaining pre-1.3 DLF converters to the new API. b. Add reports for the lire_import_log and lire_import_stats superservice. c. Write new responder using new APIs. d. Interactive performance improvement, Lire takes much time to start because of all the XML files it has to parse. It would be easy to add a persistent cache to speed that part up. e. Add more information on where the data came from to the XML report (DLF sources used, reports merged) f. Report generation performance improvements. FJL, Oct 12, 2004