@algernon I'm wondering if it'd be more effective (in terms of impact/service cost) to serve finite trees that update periodically, like a real web site? I guess this way you could make the bot come back (poison it for a longer time) and constrain the expected bandwidth per visit?