infosex.exchange <3
You are probably looking for the infosec.exchange Mastodon instance
This host is mostly for my random stuff, and in little part acts like a well-intentioned placeholder for the typosquatted domain.
Discoverability and Archiving
Currently I'm using this host for saving the items from my own feeds to the Wayback Machine and provide in-links for search engines. I hate that I have to do this, but the non-sense ideology of Mastodon pretty much ruined the search feature for Fediverse as a whole, and this wasn't changed by the fact that they owned their mistake and implemented search eventually.
Yes, I (or anyone else) could do similar things with other peoples published feeds, regardless of the tantrum. No, you can't defederate this, because the process doesn't rely on an instance.
Gluttony Section for Search Engines
@stf Thanks for the reminder, I never had the opportunity to use it! My goal is specifically to dump datasets from Wayback Machine for specific domains, so browser-based solutions are less useful for me now.
@ciaranmak Got you! I'd say that hitting paywalls and even some JS-based UI monstrocity is the "normal" these days which I'd expect (and probably use Selenium or similar to grab it). But in case of the Wayback Machine I'd expect a friendlier API...
@ciaranmak I'm not sure I follow. Are you doing this via the CDX API? If there is RSS what requires tweaking? The RSS feeds don't include the whole content so you have to scrape them for archiving?
TBF I face much more challenges saving data _from_ the WaybackMachine using the CDX API than most of the sites I've scraped:
Most tools for offline archiving simply don't work, and although I'm *really* slow with my requests I get throttled all the time :P
About once in a year I have to look at some Ruby stuff, and it's always getting worse.
Not only can't I install fresh versions with rbenv anymore, but even the ones that are available are broken.
Next Page