Netropolis - a program for searching a LAN
DESCRIPTION
Netropolis indexes files shared on a LAN and allows access to them through
a web-based front end. Only one computer running Netropolis can give everyone
on that LAN access to very fast network-wide searching.
Made using mod_perl, DBI and Samba.
MOTIVATION
True there are other programs out there that do similar things to this one.
I decided to write Netropolis after seeing that all the others either
a) required windows
b) transfered files through a central server
c) didn't have other features I wanted
d) were too slow
Netropolis is very fast, easy to configure, should run on any computer running apache+mod_perl and Samba, and is accessible and useful to users of any OS. Hopefully.
DOCUMENTATION
The documentation is in Netropolis::Viewer and Netropolis::Library in form of
POD format perldocs, as well as commenting throughout the modules.
To access the POD documentation, use:
perldoc Netropolis::Viewer
perldoc Netropolis::Library
INSTALLATION
1) Make sure the dependencies ( DBD, HTML::Template, Samba, and a SQL database ) are installed. You can find the perl modules via http://cpan.org. For a
database try http://www.postgresql.org/ or http://www.mysql.org
2) Install the perl modules:
cd netropolis-2.0
perl Makefile.PL
make
make install (must be root for this command)
3) Add a user 'apache' to the database with basic permissions:
adduser apache (Postgres)
4) Create a database:
createdb netro (Postgres)
Note: if you name your database something other than 'netro', you need to
change that in netropolis.conf
5) Setup the database tables:
psql -d netropolisl -f sql/netropolis.sql (Postgres)
- OR -
cat sql/netropolis.sql | psql -d netropolis (Postgres)
6) Make sure that Apache with mod_perl support is installed.
You should add lines similar to these in your httpd.conf file:
use Netropolis::Viewer;
PerlInitHandler Apache::StatINC
SetHandler perl-script
PerlHandler Netropolis::Viewer
7) Install the indexer script (must be done as root)
mv netropolis-2.0/bin/netropolis_indexer.pl /usr/local/bin
8) Move the template file to a good location, such as your home directory or
a directory within your httpd directory.
mv netropolis-2.0/netropolis.tmpl /home/httpd
Move netropored.png and netropoblue.png to a directory within your httpd
document root (probably something like /home/httpd/html)
mkdir /home/httpd/html/imgs
mv netropolis-2.0/*.png /home/httpd/html/imgs
9) Configure Netropolis by editing the file netropolis.conf in directory
where that was created when you uncompressed netropolis-2.0.tar.gz
vi netropolis.conf
-> values you need to update are db, password, smb_host, template,
and img_url (where you moved the .png files in step 8)
mv netropolis.conf /etc/netropolis.conf (must be done as root)
Note: netropolis.conf must be copied or moved to /etc
Test your configuration by running the script configuration_test.pl in the
bin directory
cd bin
perl configuration_test.pl
If you get an error, Netropolis will not work correctly until you fix it.
10) Depending on your needs, you may want to add a cron job to update the
index on a regular interval. You could add a line like this to your
/etc/crontab file (as root) to run the indexer script once a day:
01 4 * * * dbuser psql -d netro -f /path/to/netropolis/sql/reindex.sql
02 4 * * * root perl netropolis_indexer.pl
Note: You need to apply the reindex.sql file to the database before you
run netropolis_indexer.pl or you will get duplicate results.
11) Netropolis should now be ready to run. Restart apache and run the indexer
script
service httpd restart
- OR -
/usr/local/apache/bin/apachectl restart
perl netropolis_indexer.pl
12) Netropolis should now be running. Open your web browser and go to
http://localhost/netropolis
13) Optimize Netropolis
Methods:
A) Noise words: these are words that are very common and that many
files have in their file_name. You can prevent Netropolis from
both indexing and searching these words:
perl netropolis-2.0/bin/noise_words.pl
If the output from running this script makes you want to stop
Netropolis from indexing or searching certains words (if they have
a significantly high number, such as 1000+ or so) then you can add
them to your netropolis.conf file.
B) File type restrictions: in my experience, on my dorm network, there
was always at least one person who had their entire computer shared.
Indexing every single system file slowed the indexer script down
tremendously. So I decided to streamline the focus of my Netropolis,
and removed file types, such as .txt and .bmp and many others that
nobody really would want to use netropolis for, from my
netropolis.conf file. Netropolis will only index those file types
which are included in the netropolis.conf file.
Note: In my experience, it takes the indexer script about 20 min to index
just music and video files on my 170 computer network. This
was 181.2 Gb of data, but more importantly was just under 30,000
files. With more file types such as txt and gif and bmp in my
netropolis.conf I was getting about 220 Gb and 60,000+ files. Having
Netropolis index only the files you'll actually use it to search for
will drastically improve its performance.
CONTACT INFO
This program was written by Gabriel Burt (gburt@uiuc.edu). For more
information visit the Netropolis project web site at
http://netropolis.sourceforge.net
LICENSE
Netropolis: Network indexer and searcher.
Copyright (C) 2002 Gabriel A. Burt
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.