@algernon I get that there's a lot of nuance here, that's why I asked for "consideration" that can include e.g. allowing standard crawlers.
Apparently building an index is much bigger effort than I expected (based on the struggles of EU and alternative providers), so I don't think that will happen in the near future.
LLM performance will degrade for sure, but I don't think it will restore trust in traditional search or otherwise move ppl away from assistants once they became dependent.
Btw. my post was less about your work, and more about e.g. GitHub where content is no longer properly searchable either via web search or their internal search :)