12-23-2022, 02:04 PM
Off-site attachments (such as dropbox links) weren't downloaded, but the local stuff that we host on server should be. Some of those HTML files of same names (-2, -3, and whatnot) are *exact* same pages in duplication. One link might be for http://whatever.com, another for https://whatever.com and a third for www.whatever.com -- all the same place, all the same page, but duplicates for the archive as each has a different link to access them!
Our last 2 forums weren't downloaded as they're private and password protected. The web crawler didn't have an account with us, they weren't signed in, so they only did mirroring of the publicly available forums.
Our last 2 forums weren't downloaded as they're private and password protected. The web crawler didn't have an account with us, they weren't signed in, so they only did mirroring of the publicly available forums.