can ne body give me an insight on java cralers and what it takes to code web crawlers???
-
-
know what data you are going to extract
make an algorithm of it
beware of Object manipulations( specially for a 24/7 System, it consumes huge memory and increasing if not handled properly )
on threads, be careful to handle passed-by-reference instances
Select what database system you are going to use for storage on your extracted data
good luck
Comment