#=====================================================#
# LogScribe - Web Access Statistics CGI               #
# Frequently Asked Questions                          #
# FAQ date: 11/09/1996                                #
# Current LogScribe version: 1.0                      #
#=====================================================#
# by Usama Wazeer (usamaw@cs.utexas.edu)              #
#    http://www.cs.utexas.edu/users/usamaw            #
# and Daniel Cedras (cedras@texas.net)                #
#    http://www.texas.net/~cedras                     #
#=====================================================#


+-----------------------------------------------------+
| This FAQ attempts to answer most questions about    |
| LogScribe and installation problems one could face. |
| It was generated after numerous suggestions and     |
| countless similar questions repeatedly asked :-)    |
| This FAQ will grow as more questions are prompted.  |
| It probably does not answer ALL the questions that  |
| are out there, but feel free to provide additional  |
| information and correction to the FAQ, since its    |
| very existence is the result of collective          |
| contribution.  Installing LogScribe requires a      |
| minimal knowledge of the Unix operating system      |
+-----------------------------------------------------+


- Where can LogScribe and/or this FAQ be found ?
- What is LogScribe ?
- Why would I want to use LogScribe ?
- What do I need to run LogScribe ?
- What platforms are supported ?
- Who can run LogScribe ?
- How does LogScribe work ?
- How much disk space does one need to use LogScribe ?
- Will LogScribe tell me where users come from before my page ?
- Where are my logfiles ? I don't see them in the distribution
- How can someone view their logs ?
- What's the best way to troubleshoot LogScribe's installation ?
- I get a listing of the actual program when I provide the URL,
  and not the output I am supposed to get.  What did I do wrong ?
- I get an error when I give the URL to showlog.pl... why ?
- I get an output from showlog.pl but no data - where did it go ?
- My showlog.pl works fine but when I click on something in
  the second or third column I get an error
- I get core's in my directory and my logs are corrupted...why ?
- I get HTML_ERROR:Missing_Pagename/URL - why ?
- Everything works fine, but I cannot edit nor delete my
  logfile(s) manually - what's going on ?!
- When I try to run the scripts I get some EOF errors - how come ?
- My showlog page looks real funny on my browser
- I get a broken image icon - why is that ?
- Erroneous configuration entries can be a nightmare
- I tried EVERYTHING but NOTHING works !
- What is logscan.pl ?



- Where can LogScribe and/or this FAQ be found ?

  There are three distribution sites:
   http://www.texas.net/~cedras/logscribe
   http://www.cs.utexas.edu/users/usamaw/logscribe
   http://acm.vt.edu/~nisha/logscribe


- What is LogScribe ?

  LogScribe is a common gateway interface (CGI) programmed in 
  PERL (Practical Extraction and Report Language) and is designed
  to provide a graphical representation of accesses to a web site.
  The original idea came from Matt Kruse (mkruse@netexpress.net)
  but Usama Wazeer and Daniel Cedras rewrote it to make it more
  accessible to users who did not meet the requirements the 
  original program suggested. Also added are several features to
  enhance its usefulness, flexibility, and ease of use.


- Why would I want to use LogScribe ?

  LogScribe was written with normal users in mind.  Users who
  have access to their own cgi-bin but who do not have read 
  access to the server's main log file (access_log). To those 
  who do have access to it, the file can be enormous, since it 
  contains all accesses to EVERY user's homepage - therefore 
  containing more information than a user might be interested 
  in keeping track of.  Sometimes the access_log file can reach
  the 100 megabyte mark - imagine having to run a script that
  would have to dig only the accesses to your pages out of
  that file ! It would take forever ! In addition to that, most
  web statistic packages out there *must* be installed by the
  system administrator - a mere user cannot install them because
  they require certain priviledges that only the system adminis-
  trator has.  LogScribe does not require system administrator
  level nor access to the access_log file.  It can be installed
  by any user who has permission to run his/her own cgi's, and
  LogScribe creates its own access logs.  LogScribe is a PERL
  script, therefore should run fine on any system running PERL.
  LogScribe was not designed to be used as a server-side include 
  (SSI) because very few users have access to such priviledge :p
  

- What do I need to run LogScribe ?

  - A PERL compiler/interpreter on your system - LogScribe runs
    fine on PERL 4 (perl -v  at Unix/DOS prompt gives version)
  - The web pages whose accesses you want logged (duh !)
  - The ability to run your own cgi's (ask your system
    administrator if you are allowed to run your own cgi's and
    what directory they go in; and how scripts/programs should
    be called via URL)
  - A web browser capable of displaying tables
    (Netscape 2.0+ or Internet Explorer 2.0+ work fine)


- What platforms are supported ?

  So far LogScribe has been tested under the Unix operating
  system running PERL 4 or above. LogScribe was tested and
  proven to run successfully without modification on Windows
  95/NT, providing that a cgi-capable server was installed
  along with the 95/NT port of PERL. Recommended are the
  32 bit port of PERL (www.perl.hip.com) and O'Reilly's 
  Website (website.ora.com).  Please do NOT call your CGI's
  using the URL: http://.../perl.exe?makelog.pl - it's an
  open invitation to malicious hackers.  Instead associate
  the *.pl extension with your perl.exe so that you can
  directly call up your scripts: http://.../makelog.pl 
  just like on a Unix system.  Since LogScribe was mainly
  developed under the Unix operating system, no support
  can be provided for other operating systems. Try/Use it
  at your own risk.  LogScribe does not make any calls to 
  external system-specific libraries for maximum 
  cross-platform compatibility.  LogScribe has not been
  tested under MacOS.


- Who can run LogScribe ?

  Any user who has permission to run his/her own CGI's


- How does LogScribe work ?

  LogScribe is basically made up of 3 scripts: 
  makelog.pl, showlog.pl, and pagelog.pl

  - makelog.pl  does the actual logging - it creates a log
                file and records accesses to it.
  - showlog.pl  reads the log file and displays the accesses
                in an easy-to-read graph-bar manner.  It is
                also the menu interface to easily view your
                logs.
  - pagelog.pl  displays in details all the connections to 
                your pages in a listing manner, comprising:
                the date/time, host connecting, page accessed,
                browser used.

  makelog.pl    can only be called via an IMG tag on the page
                to be logged.  When called, makelog writes to
                the log file and returns a transparent gif/pixel
                as promised, to the IMG tag.  Makelog will also
                create a backup log in case things go bad (IE:
                the system hangs/dies while makelog is writing 
                to the log file) and will resume its activities
                by using the backup when all system functions
                are restored.  Makelog also makes use of a 
                "lockfile" to prevent simultaneous writes to the
                log file, therefore allowing only one access to
                be logged at a time.  This measure is to ensure
                that the log file isn't overwritten/deleted by
                mistake.  Makelog is not meant to be called 
                directly by inserting the URL to it in the URL
                box of your browser, but rather by the page to
                be logged.


- How much disk space does one need to use LogScribe ?

  As of the date this FAQ was last revised, LogScribe requires
  less than 150 kilobytes of disk space to be installed.  Keep
  in mind that your logfile will grow in size depending on how
  often your pages are accessed and on the number of days you 
  want logged (as defined in the configuration file).  A log
  file containing 15,000 hits will require about 3 megabytes
  of disk space (1.5 megs for the logfile and 1.5 for the
  backup).  If you do not have that amount of space to spare,
  consider reducing the number of days you want logged in 
  your configuration file.


- Will LogScribe tell me where users come from before my page ?

  This is not easy to implement for two reasons:
  1) The HTTP_REFERER variable contains the URL to the page where 
     the guest came from before accessing your page.  As soon as
     your page is accessed, it calls up the makelog.pl and the
     HTTP_REFERER value is now replaced by the URL to your page.
     LogScribe takes advantage of that to implement its
     'anti-"counter"-terrorism' feature by disallowing people 
     from adding access to your logs from their (outside) page.  
     Because of the URL substitution it is not possible to log 
     the origin of the access besides the page to be logged itself.
  2) Adding the origin's URL to the log file would make it grow
     faster and would significantly slow down its processing.

 
- Where are my logfiles ? I don't see them in the distribution

  access.log, access.bak, access.lok, and access.old (if you decide 
  to keep those names) are automatically created by makelog.pl itself 
  if they don't already exist.  They are created as soon as the pages
  to be logged are accessed.


- How can someone view their logs ?

  By typing in the URL to their showlog.pl in their browser's URL box.


- What's the best way to troubleshoot LogScribe's installation ?

  After installing the program according to the readme.txt, 
  insert the URL to your showlog.pl in your browser's URL box.
  If you get a menu with or without stats, you are on the right
  track.


- I get a listing of the actual program when I provide the URL,
  and not the output I am supposed to get.  What did I do wrong ?
    
  - Did you set the execute permission bit on your *.pl files ?
    Go into the directory where your *.pl files are and try
    one of these 3 lines - and keep trying showlog.pl:
    a) chmod 700 *.pl  then  chmod 700 .  (that's a dot)
    b) chmod 750 *.pl  then  chmod 770 .  (that's a dot)
    c) chmod 705 *.pl  then  chmod 707 .  (that's a dot)

  - Are your *.pl files in a subdirectory of your CGI directory ?
    Some servers want all your CGI's to be directly inside your CGI
    directory, and not in a subdirectory of it.  Try moving your
    scripts in your main CGI directory then try again.  If you get
    an error or the same problem, try one of the 3 lines below:
    a) chmod 700 *.pl  then  chmod 700 .  (that's a dot)
    b) chmod 750 *.pl  then  chmod 770 .  (that's a dot)
    c) chmod 705 *.pl  then  chmod 707 .  (that's a dot)

  - Some servers do not know the *.pl extension - Try changing
    the ending of all your scripts from *.pl to *.cgi by renaming
    them.  Don't forget to change inside all your *.pl scripts
    the line that says: require "${curpath}config.pl";  
                    to: require "${curpath}config.cgi"; 


- I get an error when I give the URL to showlog.pl... why ?

  - Your permission bits are set wrong - scripts must be executable
    to run.  Also make sure that the whole path down to your script
    is accessible to the server by setting the right permission on 
    each directory along that path.  Go inside the directory that
    contains your LogScribe files and try one of those 3 lines:
    a) chmod 700 *.pl  then  chmod 700 .  (that's a dot)
    b) chmod 750 *.pl  then  chmod 770 .  (that's a dot)
    c) chmod 705 *.pl  then  chmod 707 .  (that's a dot)

  - Somehow showlog.pl isnt able to access the configuration
    file (config.pl) - edit each *.pl script and change the line
    at the very top of each script that says: 
      require "${curpath}config.pl";
    to the full path to your configuration file...
    example: 
      require "/home/users/cedras/public_html/.cgi-bin/logscribe/config.pl";
    (hint: go into the directory where your files are and type: pwd [ENTER])

  - Your PERL interpreter/compiler is in a different directory than the
    preset.  Find out which directory by typing: which perl   or
    whereis perl   at your Unix prompt, then replace the '/usr/local/bin/perl'
    in the very first line of all your *.pl files by the appropriate path.

  - Are your *.pl files in a subdirectory of your CGI directory ?
    Some servers want all your CGI's to be directly inside your CGI
    directory, and not in a subdirectory of it.  Try moving your
    scripts in your main CGI directory and try again.  If you get
    an error or the same problem, go into the directory where your
    *.pl files are and try one of these 3 lines:
    a) chmod 700 *.pl  then  chmod 700 .  (that's a dot)
    b) chmod 750 *.pl  then  chmod 770 .  (that's a dot)
    c) chmod 705 *.pl  then  chmod 707 .  (that's a dot)

  - Some servers do not know the *.pl extension - Try changing
    the ending of all your scripts from *.pl to *.cgi by renaming
    them.  Don't forget to change inside all your *.pl scripts
    the line that says: require "${curpath}config.pl";  
                    to: require "${curpath}config.cgi"; 


- I get an output from showlog.pl but no data - where did it go ?

  - Did you remember to put the IMG tag at the bottom of the page
    to be logged ? If not, read the readme.txt that comes with 
    LogScribe for the proper syntax.

  - Did you try to connect to the page to be logged at all after
    putting in the IMG tag ? This is the only way to trigger
    makelog to create the logfile

  - ARE YOU SURE you set your "@referers" variable in the
    configuration file to equal your site's address ?!?!
    Please do so because this is the number one error most
    people make ! They forget to set it to their site's 
    address and leave the default site that comes in the
    configuration file !

  - Are you SURE you did not add the domain where you connect
    from usually in the "@ignoresites" variable in your config.pl ? 
    Because if you did it will never log any connections coming from 
    you !!!

  - Some PERL compilers are really stupid and do not recognize the
    empty array () -> for example, in the config.pl - some PERL 
    compilers recognize @ignoresites = (); while some others
    only recognize @ignoresites = (''); and even some others don't
    recognize either form.  To be on the safe side, we recommend
    that you fill out the "@referers" variable with your site
    address instead of leaving it blank, and fill out the 
    "@ignoresites" variable with something that will not be part
    of a valid hostname...
    example:
    @referers = ('www.texas.net/~cedras');
    @ignoresites = ('ignorenosites');

  - You provided the wrong URL to the makelog.pl script from your html 
    file.  Make sure to check with your system administrator for correct 
    URL to it.

  - Your directory is not writable by the server, go into the 
    directory where your *.pl files are and try one of these:
    a) chmod 700 *.pl  then  chmod 700 .  (that's a dot)
    b) chmod 750 *.pl  then  chmod 770 .  (that's a dot)
    c) chmod 705 *.pl  then  chmod 707 .  (that's a dot)


- My showlog.pl works fine but when I click on something in
  the second or third column I get an error

  The URL to your pagelog.pl is wrong in your configuration
  file.  Make sure you provide the URL to it and not the PATH
  to it.  Also double check the other settings in there to
  make sure everything is OK.


- I get core files in my directory and my logs are corrupted...why ?

  Well if your system administrator puts limits on CPU/memory usage, 
  on each cgi call, when your log file gets too big, the program 
  croaks.  Nothing we can do about this. Try reducing the number of 
  days to log and delete the logfiles - start over.  If you are still
  getting errors, try asking the system administrator to increase your
  allowance on CPU/memory usage.


- I get HTML_ERROR:Missing_Pagename/URL - why ?

  - Follow the syntax to log a page as described in the readme.txt
    Make sure there are NO spaces and that you NEVER pressed the 
    ENTER key within the quotes in the SRC="" part of your IMG tag.

  - Someone is accessing the makelog.pl via the URL box of their 
    browser :) that's a no-no :p

  - Edit the logfile to delete those lines


- Everything works fine, but I cannot edit nor delete my
  logfile(s) manually - what's going on ?!

  Well guess what - if the HTTP server is configured wrong, then
  it will probably be the only one to have read/write access to
  your log files.  Only the server can give you read/write access
  to those files.  What must you do ? Make the server run a script
  that will change the permission bits on your logfiles in order 
  to allow you to have access to them.  Here is the listing to
  such script... Put this in your LogScribe directory as chperm.pl
  and configure it the same way you did showlog.pl
  (change extension to .cgi, or path to PERL interpreter, or path
   to config.pl as instructed in previously discussed problems if
   you are not able to run it just the way it is right now)
  Once configured, give it the same permission bits as showlog.pl
  using the chmod command you used on showlog.pl and run it by
  passing the URL to it via your browser's URL box:

----8<------cut here--------
#!/usr/local/bin/perl
$curpath = substr($0, 0, rindex($0, "/")+1);
require "${curpath}config.pl";

if (-f $logfile) {chmod 0777, $logfile;}
if (-f $backup) {chmod 0777, $backup;}
if ($oldlogfile) {chmod 0777, $oldlogfile;}
if (-f $lockfile) {chmod 0777, $lockfile;}

print "Content-type: text/html\n\n";
print "ok - you can now read-edit-delete your log files\n";
----8<------cut here--------


- When I try to run the scripts I get some EOF errors - how come ?

  You decompressed the scripts wrong - perhaps you downloaded LogScribe
  to your PC, edited the files there and uploaded them to your Unix 
  accounts.  Chances are those files will contain the ^M character that
  DOS uses as one of its END-OF-LINE symbol.  Those ^M characters are
  invisible to most editors and can usually be seen using Unix's
  "vi" editor.

  - Try downloading the LogScribe distribution directly to your Unix
    account using the text-only lynx browser and providing the path
    to the distribution file right on the command line.  Example:
    % lynx http://www.texas.net/cgi-bin/user/~cedras/lssend.pl?logscribe.tar.gz

  - If you downloaded the distribution file to your PC, then upload
    it to your Unix account using a good FTP program such as WS_FTP
    using the BINARY mode.  WS_FTP removes the ^M's as it uploads.

  - If you decompressed the distribution file on your PC in order to
    edit the various files, use a good FTP program such as WS_FTP to 
    upload those files to your Unix account - send all the *.gif files
    as BINARY files and all the other ones as ASCII (Text).  WS_FTP 
    removes the ^M's as it uploads.

  - If you are still getting errors, try downloading the distribution
    file again - The one you already have may be corrupted.


- My showlog page looks real funny on my browser

  - If things look REALLY WEIRD maybe you aren't using a browser capable
    of displaying tables correctly (if at all). Recommended are Netscape's
    Navigator 2.0 or above or Microsoft's Internet Explorer 2.0 or above.
    They are freely available at http://www.netscape.com/cgi-bin/123.cgi
    and at http://www.microsoft.com/ie

  - If the tables show up correctly, you probably don't have any logs yet.
    Try one of the suggestions as previously discussed in troubleshooting
    a showlog page with no data.


- I get a broken image icon - why is that ?

  - If you get that broken image on the page to be logged, chances are
    that your makelog.pl is having difficulties being accessed or logging
    accesses.

    - You provided the wrong URL to the makelog.pl script from your html 
      file.  Make sure to check with your system administrator for correct 
      URL to it.

    - You have the wrong permission bits set.  Make sure your makelog.pl
      is executable by the server.

    - Try troubleshooting it using the guidelines found in those
      sections of this FAQ:

      "- I get a listing of the actual program when I provide the URL,
         and not the output I am supposed to get.  What did I do wrong ?"
      "- I get an error when I give the URL to showlog.pl... why ?"
      "- I get an output from showlog.pl but no data - where did it go ?"
      "- My showlog.pl works fine but when I click on something in
         the second or third column I get an error"
      "- My showlog.pl works fine but when I click on something in
         the second or third column I get an error"
      "- When I try to run the scripts I get some EOF errors - how come ?"

  - If you get broken images on a page where you are supposed to have
    bar graphs, check your config.pl and make sure your $gifdirurl contains
    the correct URL to the directory where your *.gif files are (try to
    put them with your regular *.html files instead of with the CGI's
    themselves).  If that isnt' the problem, make sure that the gifs are
    readable by the server. Try to go into the directory where the *.gif
    files are and type: chmod a+r dot?.gif [ENTER]


- Erroneous configuration entries can be a nightmare

  One easy mistake is to substitute PATH with URL and vice versa. 
  In the configuration file (config.pl) , if the variable name
  contains 'url' in it, it means you MUST put the URL to what is
  asked for.  If it does NOT contain 'url' then chances are that
  it's requiring a path.  URL is the address to a file that starts
  with "http:// ..." and PATH is the actual file system PATH to
  the file itself.
 
  $logfile =  is expecting a PATH
  $showloglurl =  is expecting a URL

  examples:
  $logfile = "/home/users/cedras/public_html/.cgi-bin/access.log";
  $showlogurl = "http://www.texas.net/cgi-bin/user/~cedras/showlog.pl";


- I tried EVERYTHING but NOTHING works !

  Are you sure you are allowed to run *YOUR* cgi's ? and not just the 
  standard ones that come with the server?! - ask your system 
  administrator.


- What is logscan.pl ?

  logscan.pl is a utility included to scan your logfile and fix
  erroneous entries.  It is VERY basic but yet very helpul in 
  locating those errors that make your log report pages look 
  funny. It is extremely easy to use and when run it will try
  to find the problem lines and display them along with the
  line number so that they are easier to locate.
  There are two ways to run this script (let's assume in the
  example below that access.log is your logfile):
  1) logscan.pl access.log | more
     This will scan your access log and pause after every
     error page so that you have time to view the errors
     (if any)
  2) logscan.pl access.log access.fix
     This will scan your logfile for errors and will output
     the corrected logfile to access.fix - It is not guaranteed
     that the access.fix will be 100% fixed but most bad entries
     will be removed. Note: using this method doesn't display 
     errors found to your terminal - corrected output is saved
     in the access.fix file - remember then to delete your
     access.log (rm access.log in Unix) and rename your access.fix
     back to access.log (mv access.fix access.log  in Unix)           

  This script can also be used to delete lines containing these 
  errors:
         HTML_ERROR:Missing_Pagename
         HTML_ERROR:Missing_URL

-end-
