The Internet Archive’s “Wayback Machine” archive of the Web is what I consider to be a very valuable resource. Sites come and go, but if they’re successfully archived by the “Wayback Machine”, they’re still viewable and the information they contain isn’t lost.
I’ve been concerned about the effectiveness of the archive for several years, though, due to it making pages unavailable when a new version of the page either contains an explicit prohibition in the page’s “robots.txt” directive to search engine spiders, or if the file is missing or incorrectly configured. It caused a large number of previously-archived pages to be hidden from view for several years. Often, the simple presence of a “parking” page on an old site’s domain could cause the history of an old site’s domain to be effectively locked away.
Earlier this year, the Internet Archive has, thankfully, taken action to change the “Wayback Machine’s” behavior with respect to old sites versus the “robots.txt” file. Many sites which had been blocked from displaying their archived copies are now working again. Of course, it can only archive publicly-accessible content; content behind login prompts such as forums and members-only areas of websites can’t be archived, just as they generally can’t be indexed by search engines.
For MSTS users, that means that archived copies of long-gone MSTS-related pages are becoming available again. Some of the archives are complete, or nearly so, including freeware downloads on some pages. For instance, the freeware USRA Light Mountain by Train Artisan is a popular base component of various freeware steam locomotives found in TrainSim.com’s file library. It’s once again available here on the “Wayback Machine” after an absence of two years or more.