JUGTAIL

NAME

SYNOPSIS

jugtail -B [ ?dmtv ] [ -l log ] dataFile

jugtail -S [ ?dtv ] [ -l log ] [ -M maxProcs ] [ -p port# ] [ -u username ] dataFile  

DESCRIPTION

jugtail is used to excavate through gopher menus, display the hierarchy of the encountered menus, acquire the name of the hosts accessed via a given server, build search tables, or run as a search engine waiting to do boolean searches on menu titles. The purpose of this program is to give a linear view of a gopher hierarchy, or allow users to search through menu titles of a given gopher server or group of gopher servers. The boolean operators AND, OR, and NOT are supported as well as partial word searches. Gopher was developed at the University of Minnesota.  

Parameters

-?

prints the usage information and terminates.

-A

prints a directory tree created from all directories served from host. This option overrides -a. You may also want to set the -n flag. See NOTES for directory tree format.

-a

prints the directory tree created from the total number of directories served from a particular host. See -A for format information. Both -a and -A print the tree after the menu has been traversed.

-B

builds an index table and a hash table for quick access to the data found in dataFile. As the tables are being built the number of items remaining to process is displayed. The display can be turned off with the -m flag.

-b

creates a data file called dataFile for use with the -B option.

-d

is for debugging mode which prints out diagnostic information. This flag disables use of the -l logging flag when used with the -b or -B flags.

-H

prints the hosts and ports accessed from host. This option overrides -h and is printed after the menus are traversed.

-h

prints the hosts accessed from host after the menus are traversed.

-I

prints host and port information to the right of all entries. This option overrides -i.

-i

prints host and port information to the right of directories only.

-l

says to log any errors encountered when building the dataFile or tables, or log search queries to the file log. It is recommended that the log file used with the -S option not be the same file used with either -b or -B. Use of this flag is disabled when used with the -d debugging flag. The log file format used with -S has the same format as the log file produced by gopherd.

-M

says to limit the number of jugtail processes to the maximum of maxProcs

-m

supresses output when used with the -B, flag, or prints the menus when used with the -b flag.

-n

prints line numbers preceeding each menu item. Useful with the -A flag or to determine quickly where an item was previously displayed.

-p

tells jugtail to use port# as the port number. The default port when not started as a search engine with the -S flag is the gopher port 70. The default port when started as a search engine is port 3000.

-S

runs jugtail as a server listening out a port for a string containing words to search on. This includes boolean operators AND, OR, and NOT. If no boolean operator is found between words, the operation to perform is an implied AND. All evaluations are processed left to right. Currently there is no way to search on the words AND, OR, or NOT, nor break the expression into sub expressions. The search engine also supports partial word searches, with the wildcard symbol "*". Currently the wildcard symbol "*" cannot be the the first character in a word to search for, and all characters following the wildcard "*" are ignored. When jugtail is running as a search engine, the SIGHUP signal is supported which rereads the dataFile tables into memory. See NOTES for more information.

-s

allows the use of sending a selector string selStr. By default this is the empty string.

-t

prints the time required to either load the index table into memory, build the index table, or process the menu of a given gopher server. The time is printed in the form "h:m:s".

-u

sets the process user id to username.

-V

builds a file dataFile.veronica which does not contain any "Disallow:" paths as found in the veronica.ctl file. The dataFile.veronica file is the file passed to veronica when jugtail received the "?datafile" command.

-v

prints the version of jugtail and terminates.

-X

informs jugtail to not search any menus which contain selStrNot2search as the selector string. This string can be wildcarded with the "*". You can have multiples of these just make sure each is preceeded by the -X flag.

-x

informs jugtail to not search the menus of hostNot2search out port port#. You can have multiples of these just make sure each is preceeded by the -x flag. hostNot2search can be wildcarded with "*", which would imply any host. Example: "-x * 7777" means don't search any host using port 7777.

-

flag means get the hosts2search from stdin. The hosts2search need to be separated by white space, and when you are finished entering the hosts2search enter EOF, ^D to quit. The hosts2Search can be a domain such as .edu or .gov. Note the preceeding period (.) when using a hosts2search as a domain.

hosts2search

is a list of hosts to have their menu structure printed out or searched. If no hosts2search is specifed host will be the only host searched. If you specify any host in hosts2search and want host to be searched also, you must specify host as one of the hosts2search.

host

is the Fully Qualified Domain Name of the machine with the gopher server to traverse the menus of.

dataFile

is either the name of the dataFile to create when used with -b, or the name of the dataFile used to build the index and hash table with -B, or the name of dataFile to use when acting as a search engine with -S.

NOTES

The

-hHiI flags make no sense with the -bBS flags as far as building the data and index files. They alter the behavior of the information printed to the screen when used with the -b flag. The order of the various flags is irrelevant with the exception host and dataFile must be the last argument.

When using the wildcard "*" as a parameter don't forget to escape it from the shell by surrounding it with double quotes.

When acquiring the data for use as a search engine, make sure the dataFile being built is stored in a temporary location. Otherwise any search queries given jugtail running concurently as a search engine will yield erroneous results. After building the tables move dataFile, dataFile.ih, dataFile.ix and dataFile.veronica, if dataFile was built with the "-V" option, into the proper location and immediatly send SIGHUP to jugtail. When jugtail receives the SIGHUP signal /usr/local/etc/jugtail.pid will be touched to assist a make script via cron to SIGHUP jugtail by allowing a target dependency on this file and the dataFile.

jugtail does NOT use aliased hostnames and requires Fully Qualified Domain Names or partial domain names. "gopher" is NOT the same as "gopher.some.domain".

When jugtail is started with the -S flag, a file is generated which contains the process-id and the command line used to start jugtail. By default this file is stored in /usr/local/etc/jugtail.pid. To send the SIGHUP signal to jugtail do the following:

kill -HUP `/usr/ucb/head -1 /usr/local/etc/jugtail.pid`

To start jugtail with the same arguments as a previous invocation do the following:

eval `/usr/ucb/tail -1 /usr/local/etc/jugtail.pid`

WARNING: If jugtail is currently using the specified port you will get a bind error. Only one server can run on a given port at a time.

jugtail supports a configuration file, by default /usr/local/etc/jugtail.conf, that contains variables used when running as a search engine. The format of this file is such that anything to the right of the pound sign "#" is a comment, blank lines are ignored, and assignment is as follows:

   variable_name [=] ["]variable_value["]

Where anything enclosed in square brackets is optional, and if variable_name contains whitespace it must be double quoted. variable_name may contain new-line '\n', carriage-return '\r', tab '\t', and double quote '\"'.

jugtail supports some special commands, when acting as a search engine, where each special command must be preceeded by a question mark '?', and are listed below with a brief description:

?all what

returns all the hits on what, otherwise items are only returned if there are less then 1024 items.

?datafile

returns the datafile used by jugtail. This command is used to assist veronica in harvesting its data.

Note: If the jugtail dataFile is built with the -V flag and the file dataFile.veronica exists in the same location as the data file dataFile, which jugtail is using as the data file when running as a search engine, dataFile.veronica will be given to veronica, otherwise dataFile will be returned.

?datasize

returns a gopher protocol item which gives the current size of the jugtail database to be returned via the ?datafile command <see ?datafile>. If this item is selected the appropriate jugtail database will be returned.

?help [what]

Gives you the ABOUT document and any optional hits on what.

?limit=n what

returns the first n items on what.

?version [what]

returns the version of jugtail and any optional hits on what.

?range=start-stop what

returns the start to stop range of hits on what.

Where what is a standard search string, anything enclosed in square brackets is optional, and all special commands must be preceeded with '?'.

Only 1 special command is supported per query, and if any syntax error is encountered it is reported as the title to the ABOUT document. To support this feature you need to edit JUGTAILHELP in the jugtail.conf.h or jugtailhelp in /usr/local/etc/jugtail.conf <by default> to specify where and what gopher server is serving the help document.

EXAMPLES

1.     To get a list of all items from the server "gopher.server"

enter:

jugtail gopher.server

2.      To get a list of all the items, as well as the hostname

and port in which the items reside from all machines in the
".server" domain, which are served from "gopher.server" enter:

jugtail -I .server gopher.server

3.      To get a list of items in the directory "directory"

with a selector string "1/directory" served from "gopher.server",
and have the results placed in the file "the.menu" enter:

jugtail "1/directory" gopher.server > the.menu

4.      To get a list of all hosts and the appropriate port

served from "gopher.server" enter:

jugtail -h gopher.server

5.      To traverse the server "gopher.server" excluding any ftp

items and items served from the same server out a different port
enter:

jugtail -X "ftp:*" -x gopher.server 6666 gopher.server

6.      To build the dataFile from all gopher servers in the

".utah.edu" domain excluding any ftp items and observe what is
looked at enter:

jugtail -mb dataFile -X "ftp:*" .utah.edu gopher.utah.edu

7.      To index the dataFile built in example 6 and observing the

status with the time required to build the tables enter:

jugtail -mB dataFile

8.      To start jugtail as a search engine on the prebuilt data

and tables and log to the file jugtail.log enter:

jugtail -Sl juhgead.log dataFile

9.      To rebuild the data used by jugtail, from sample 6, while

building a processed file that does not include any "Disallow" paths
which will be given to veronica, rebuild the index and hash tables, move the tables and files into place, and restart jugtail as a search engine do the following:

jugtail -Vb t/dataFile -X "ftp:*" .utah.edu gopher.utah.edu
jugtail -B t/dataFile
mv t/dataFile* ../
kill -HUP `/usr/ucb/head -1 /usr/local/etc/jugtail.pid`

FILES

/usr/local/etc/jugtail.pid is used to store the process id of jugtail and the command line that started jugtail as a search engine. This file is only created when jugtail is started with the -S option.

/usr/local/etc/jugtail.conf is used to override any variable defined in jugtail.conf.h. If this file does not exist the default values for these fields will be used. /usr/local/etc/jugtail.conf is read, if it exists, when ever jugtail receives the SIGHUP signal.  

BUGS

AUTHOR

The jugtail sources are available via anonymous ftp from:ftp.cc.utah.edu in /pub/gopher/GopherTools/jugtail as jugtail.VERS.tar.Z, where VERS is the current version of jugtail. Alternativly via gopher from gopher.utah.edu in: About the U of U Gopher/Gopher Tools/jugtail  

SEE ALSO

Index

NAME

SYNOPSIS

DESCRIPTION

Parameters

NOTES

EXAMPLES

FILES

BUGS

AUTHOR

SEE ALSO