aary@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acf to Technology@lemmy.ml · 4 months agoI want to find users of Heritrix3 enginemessage-squaremessage-square1linkfedilinkarrow-up11file-text
arrow-up11message-squareI want to find users of Heritrix3 engineaary@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acf to Technology@lemmy.ml · 4 months agomessage-square1linkfedilinkfile-text
I have such setup, but with some problems: it always does full crawl (doesn’t do deduplication) I am unable to control crawl order (don’t have enough knowledge)
minus-squareВежливый_Лис@xn--e1aghfa.xn--e1aqbccjfc.xn--p1acflinkfedilinkarrow-up0·4 months agokreuzcrawl — a high-performance Rust web crawling engine.
kreuzcrawl — a high-performance Rust web crawling engine.