How to use Larbin
How to compile ?
untar the package
If you need specific settings, gmake accept several options :
- gmake : build the standard executable.
- gmake prof : enable profiling, ie compile with -g
-DPROF (the execution is very slow).
- gmake debug : compile with -g, in order to enable
debugging with gdb.
For more customizations see this page
How to run the program ?
Once compiled, you can launch the program simply by typing
'./larbin'. Use the "-c conf_file" to define the configuration file
you want to use (larbin.conf by default).
You can follow the work of Larbin thanks to its web server : just
take your favorite browser and submit it the following url :
http://localhost:8081/ (with no proxy).
If you want to submit urls to larbin, by hand or through a program,
connect to the port 1976 (tcp) of your computer. The first thing to do
is to enable input (add the line "inputPort 1976" in larbin.conf).
The first line you write explains larbin what to do with the urls you
give it. This line should look like this : "priority:1 depth:3
Then you just have to write one url per line (or one int, one space
and one url if options.h
includes "#define URL_TAGS"). You can try it with a simple 'telnet
- Set priority to 1 if the urls you submit need to be treated
before usual urls. If you submit a big number of urls, set priority to
0 so are stored urls on disk, thus avoiding to run out of memory.
- Depth tells larbin how many links it should follow (inside a
site). 0 means no links should be followed. 1 means just download
chidren. More than 5 is probably not a good idea.
- Set test to 1 if you want larbin to make sure it did not already
see this page before downloading it. set test to 0 if you want to
download this page even if you already fetched it before.
How to restart after a crash (or an intentional stop)
Larbin saves its internal state on disk frequently. This allows larbin
to restart where it last stopped after an intentional stop or a
crash. If you want to restart from scratch, use the -scratch option.
This feature is only usable if options.h includes "#define
RELOAD". For restarting from previous state, there must be at least 2
fifo files (fifoxxxxxx in working directory).