	SWISH++
	Simple Web Indexing System for Humans: C++ version
	http://www.best.com/~pjl/software/swish/
	________________________________________________________________

	SWISH++ is a Unix-based file indexing and searching engine
	(typically used to index and search files on web sites).  It
	was based on SWISH-E although SWISH++ is a complete rewrite.
	SWISH++ was developed to circumvent my difficulties with using
	the SWISH-E package.

	SWISH++ has been ported to compile and run under Microsoft
	Windows by Robert J. Lebowitz <lebowitz@finaltouch.com>.

	________________________________________________________________

	Features

	 1. Lightning-fast indexing
	    SWISH++ attains its speed chiefly by doing two things:
	    using good algorithms and data structures and by doing fast
	    I/O.

		A. SWISH++ uses the C++ Standard Template Library's map
		   class that is typically implemented either as a
		   red-black or AVL tree for which the worst-case
		   running time is O(lg n).

		B. SWISH++ uses the mmap(2) Unix system call to read
		   files instead of using standard I/O. If you are
		   unfamiliar with mmap, it "maps" a file into memory
		   using the same virtual memory management mechanism
		   the operating system itself uses.  When the first
		   character of a file is read, a page fault occurs and
		   the operating system maps a page of the file into
		   memory. It is impossible to get faster access.
		   Additionally, because the file is in memory, the
		   characters in it are accessed via pointers using
		   simple pointer arithmetic rather than through
		   library function calls and input buffers.

	    Other factors contributing to SWISH++'s speed are that it
	    does very little explicit dynamic memory allocation, uses
	    function inlining, and makes very few function calls in
	    inner loops.

	 2. Indexes META elements, ALT, and other attributes
	    For HTML files, SWISH++ indexes words in META element
	    CONTENT attributes and associates them with the NAME
	    attributes. Meta names can later be queried against
	    specifically, e.g.:

		search author = hawking

	    SWISH++ also indexes the words in ALT attributes (for the
	    AREA, IMG, and INPUT elements), STANDBY attributes (for the
	    OBJECT element), SUMMARY attributes (for the TABLE
	    element), and TITLE attributes (for any HTML element).

	 3. Selectively not index text within HTML elements
	    Text within HTML elements belonging to specified classes
	    can be not indexed.  This is most useful not to index text
	    in common page headers, footers, and pop-up menus.

	 4. Apply filters to files on-the-fly prior to indexing
	    Based on filename extension, files can be filtered before
	    being indexed, e.g.: compressed files uncompressed, PDF
	    files converted to plain text, etc.

	 5. Index non-text files such as Microsoft Office documents
	    A separate text-extraction utility "extract" is included to
	    assist in indexing non-text files. It is a essentially a
	    more sophisticated version of the Unix strings (1) command,
	    but employs the same word-determination heuristics used for
	    indexing.

	 6. Index remote web sites
	    A separate utility "httpindex" is included that interfaces
	    SWISH++ to the wget(1) command enabling remote web sites to
	    be indexed. This is useful to be able to search all the
	    servers in your local area network simultaneously.

	 7. Handles large collections of files
	    SWISH++ automatically splits and merges partial indices for
	    large collections of files as it goes thereby not bringing
	    your machine to its knees by exhausting physical memory and
	    causing it to swap like mad.

	 8. Lightning-fast searching
	    The same mmap(2) technique used for indexing is used again
	    for searching. The generated index file is written to disk
	    such that it can be mmap'ed back into memory and binary
	    searched immediately, with no parsing of the data, also in
	    O(lg n) time.

	 9. Optional word stemming (suffix stripping)
	    SWISH++ allows stemming to be performed at the time of
	    searches, not at the time of index generation.  This allows
	    users to decide whether to perform stemming or not.

	10. Easy-to-parse results format
	    SWISH++ outputs its search results in the form:

		rank path_name file_size file_title

	    By placing the file_title, which may contain spaces, last,
	    you can easily parse it, e.g., in Perl:

		($rank,$path,$size,$title) = split( / /, $_, 4 );

	11. Generously commented source code
	    The source code is clearly written with lots of comments
	    including references to other works in case you want to
	    modify it under the terms of the GNU general public
	    license.

	________________________________________________________________

	Non-Features

	The following is a list of the features SWISH++ does not have
	that SWISH-E does. I wrote SWISH++ to solve my immediate
	indexing problems; therefore, I implemented only those features
	useful to me. If others can also benefit from the work, great.
	I may implement other features as time permits.

	1. Indexing and searching based on HTML tags
	   SWISH++ has no equivalent means for searching within
	   specific HTML tags (the SWISH-E -t option). I didn't have a
	   need for this feature.

	2. Document properties
	   This functionality can be achieved by using the extract_meta()
	   function in the included WWW Perl module.

	3. Crash and burn on files
	   SWISH++ will not crash while indexing any file.  Period.  If
	   it does, there's a bug and I'll fix it.

	________________________________________________________________

	Copyright (C) 1998 by Paul J. Lucas <pjl@best.com>
	SWISH++ is available under the GNU General Public License.
	This file last updated: July 26, 1999
