Does lemmy have any communities dedicated to archiving/hoarding data?
I would add in some rom collections and book repositories as well. The whole library of Nintendo games is under a gig and would go a long way for entertaining people.
Book repos? I didn’t know such a thing existed. Can you share more?
Project Gutenberg has a large collection of public domain books
Thank you kindly
Sorry, I’m out of the loop. Is there something particular that triggered this that I missed?
I saw that post about texas requiring app stores (specifically says mobile devices, so not the typical distro repository… yet) to have age verification. If they expand that, it would mean all linux distros, while maybe leaving the windows .exe downloads (ugh, shudder) alone. Wikipedia is probably more relevant in most folks’ minds for having a backup though.
Well for starters, teachers have had to start telling students that .gov websites are no longer considered credible sources for research.
Nothing in particular that I’m aware of, just a growing recognition that things are very much not well in the US these days.
The broad censorship of government data in the US, combined with the recent political attacks on Wikipedia caused me to download the whole English Wikipedia earlier this year. Guessing OP is similar
Not sure why they’d download Debian with all packages though
Edit: I should mention it’s less about a potential loss of Wikipedia as it is a personal source of truth on politically sensitive topics that get censored, or turned to propaganda by bots
For example the Wounded Knee Massacre. Pete Hegseth has recently been calling it the, “Battle of Wounded Knee”. I wouldn’t be surprised if the current administration went to war with Wikipedia and forced them to 1) Change articles they disagree with, and 2) Hide those changes from history
I had downloaded the full (no pictures) Wikipedia earlier this year for exactly this reason. This thread told me about kiwix, which is awesome, so I downloaded the “Wikipedia .08” using kiwix, which is the best 45,000 articles from Wikipedia with pictures and it’s 7G, very manageable, has most topics anyone would care about.
My rationale with Debian is that distros are kind of like portals to entire compendiums of free and open-source software. With the increasing attacks on vpns in particular right now, I’m concerned there are any number of programs we take for granted that we might not have access to soon.
The internet is already deeply enshittified. There is a real possibility that it will no longer be a free and open web in any capacity soon. So it’s past time to make archives, and start setting up meshnets.
gestures broadly
Yeah I wonder too.
Years ago I bought a physical encyclopedia. I remember having one as a kid and using it for school reports. Also just looking through it can be cool. Learning about something you never knew existed is just a unique experience and doing it through a physical book just deepens the whole experience.
I also learned the practice of printing a physical encyclopedia is going out of fashion. I think there is only one company the still prints a yearly encyclopedia and it’s not Encyclopedia Britannica of all things. Might have change since I bought my copy but go give some physical media some love if you can.
I would love to have a small Wikipedia browser that can survive the apocalypse.
E-ink display, mini keyboard and touchpad, multiple ways/ports to transfer info, All wrapped up in a heavy duty equipment case that’s able to survive a building collapses and burns in an earthquake, that’s shielded from EMP.
Sounds like the beginning of a proper Hitchhikers Guide to the Galaxy.
Actually having something telling me Don’t Panic is big friendly letters would help my mental health…
You mean like the wiki reader:
I used it as an ebook reader until the screen gave out.
I would love to have a small Wikipedia browser that can survive the apocalypse.
I’ve got the full 120 GB Wikipedia dump running in Kiwix on a Raspberry Pi Zero. Works great (surprisingly)
E-ink display, mini keyboard
Have been using a Minimal Phone for a few months now which has both of those. Can connect to the Pi easily.
multiple ways/ports to transfer info,
Add a USB-C hub (or add a hub to the Pi) and you’re set
All wrapped up in a heavy duty equipment case that’s able to survive a building collapses and burns in an earthquake, that’s shielded from EMP.
And that’s where I’m limited - My 3D printer can only do so much lol. 😆
I’ve been working on a side project this week with a Orange Pi Zero 2W (Pi Zero “clone” but with better specs). It’s got the Kiwix+Wikipedia like my older Pi (described above) plus a bunch of other neat stuff. It’s kind of a combination travel router, portable web app server, party box, and extremely over-engineered bluetooth speaker all-in-one. Hoping to put together a show-and-tell post about it when I get the last of it squared away.
Very interested in your setup for that opi2w. I have one that is being retired from pihole duty that I’ll be doing similar to. Also want to add an sdr to it so it can pull ghostnet js8call and the like.
Ooh, I haven’t tried RTL-SDR on it yet, but I think I’m nearing capacity on what it can do at once lol.
Here’s the block diagram for it (in spoiler below). Everything’s up and running except the Bluetooth Receiver -> Snapcast (it works on the bench but I don’t have the scripting/automation done yet). I’m also adding an SMA connector for an external antenna, but the new base part is still printing. Photo shows it “as is” of this writing.
SSL for the web apps was a PITA since I wanted real certs. Had to make a wildcard domain under my main hobby domain, so all my apps are like “https://{APP_NAME}.mobile.mydomain.xyz/”
As soon as I can get the Bluetooth + Pulseaudio scripting done, I’m gonna try to do a write up and maybe a show/tell post.
Block Diagram
Current Case
There should be.
Official numbers here https://www.debian.org/mirror/size
About 4.4TB, but that’s all architectures and (I believe?) all distributions (stable, testing…).
If you only want source+all+amd64+arm64, and only want stable, it will be smaller of course.
Not nothing, but at $10/TB or so, it’s not much.
And if you’re following 3-2-1, I’m pretty sure the “1” is already handled for you :)
Kinda curious where you’re getting $10/TB from
I thought the whole point of torrenting was to decentralise distribution. I use torrents to get my distros.
In my own little bubble, I thought that’s how most people got their distro.
@Maroon I thought torrent technology to be a godsend for package managers.
Why none of them use it?
I mean, damn.
Torrents are often used for installers, but for packages it tends to be more trouble than what it’s worth. Is creating a torrent for a 4k library worth it?
Turns out hosting a bunch of files is very cheap.
What happens when they just cut the underwater cables? Torrent over carrier pigeon for a linux distro would take ages
Tiny jump drives on pigeons is low key excellent imo
Pigeon latency is horrible, but the bandwidth is pretty great. You could probably load up an adult pigeon with at least 12TB of media.
https://en.wikipedia.org/wiki/IP_over_Avian_Carriers
Just gonna leave this here for whoever wants to read more on the methodology and potential risks.
Over a 30-mile (48 km) distance, a single pigeon may be able to carry tens of gigabytes of data in around an hour, which on an average bandwidth basis compared very favorably to early ADSL standards, even when accounting for lost drives.
Compared to what I use at home now, this sounds great
We need some more community wifi projects
Community Wisps are cool
A good way to see what the future of places like the U.S are is to look at places like North Korea, where they do exactly this, move files around on flash media to avoid the state censors.
Sneakernet to the rescue. Some of you are too young to know about walking around with boxes full of disks.
A wise man once said
Never underestimate the bandwidth of a station wagon full of tapes hurtling down the highway.
It was trading CD-R’s during my high school days… good times. Napster was just starting to take off by the time we had a CD-R trading network set up, Napster just increased the amount of CD’s that got passed around.
Neither are that bad honestly. I have jigdo scripts I run with every point release of Debian and have a copy of English Wikipedia on a Kiwix mirror I also host. Wikipedia is a tad over 100 GB. The source, arm64 and amd64 complete repos (DVD images) for Debian Trixie, including the network installer and a couple live boot images, are 353 GB.
Kiwix has copies of a LOT of stuff, including Wikipedia on their website. You can view their zim files with a desktop application or host your own web version. Their website is: https://kiwix.org/
If you want (or if Wikipedia is censored for you) you can also look at my mirror to see what a web hosted version looks like: https://kiwix.marcusadams.me/
Note: I use Anubis to help block scrapers. You should have no issues as a human other than you may see a little anime girl for a second on first load, but every once and a while Brave has a disagreement with her and a page won’t load correctly. I’ve only seen it in Brave, and only rarely, but I’ve seen it once or twice so thought I’d mention it.
I rarely get bounced by Anubis, but oddly enough it has happened to me a couple times in FF, I suspect it’s the fingerprinting resistance settings that cause this to happen? Hasn’t happened in a while though
I bought a 14tb drive just for backups of all my other drives… and I got a shitload more space.
Might store it on an external HDD. I got plenty.
Yeah not gonna lie, i think i heard someone in a youtube video a while back talk about how the entirety of wikipedia takes up like 200 gigs or something like that, and it got me seriously considering to actually make that offline backup. Shit is scary when countries like the uk are basically blocking you from having easy access to knowledge.
UKGOV haven’t started on things like Wikipedia yet. They know kids use it for school and blinded by ideology though they are, even they can see there’d be an enormous backlash if they blocked it any time soon.
If that’s going to happen at all, I doubt it would be before the next election. That’s whether Labour get re-elected or the Tories make an unexpected comeback. You can tell how far Labour have fallen in the eyes of their party faithful when they’ve taken a Tory-drafted policy and made it their own.
Ironically, the up and coming third option fascist party, have said they’re going to repeal the Online Safety Act. They have other fish to fry if they get in, and they’ll want to keep their preferred demographic(s) happy while they do it.
I assume that eventually something like the OSA would come back to “protect the children”. They love the current US President.
None of this is hopeful. Take this as more of a rant.
Every day it seems the entire west is gonna bee a fascist hellhole in a decade
I’m certain that when UK forces DigitalID upon the nation it will be a requirement for access to every website
Yeah, it’s surprisingly small when it’s compressed if you exclude things like images and media. It’s just text, after all. But the high level of compression requires special software to actually read without uncompressing the entire archive. There are dedicated devices you can get, which pretty much only do that. Like there are literal Wikipedia readers, where you just give it an archive file and it’ll allow you to search for and read articles.
if you remove topics you are not interessed it can shrink even more
Sure, but removing knowledge kind of goes against what creating a Wikipedia backup is about…
Well, i doubt i will ever need to know anything about a football player or a car
“Fellow survivors, oh my God! What are your names?”
“I’m OJ Simpson. This is my friend Aaron Hernandez. And this is his car, Christine.”
If my experience with mashing the random article button is any indicator, you could reduce the size by 30% just by removing articles on sports players. I doubt I’ll need those
I keep a wiki copy as well as Reddit pre-fuckuspez. A Debian archive copy sounds like a good idea.
I’m also curious about the reddit archive. Did you copy it yourself or is this available somewhere?
I got it from Archive.org. There was a monthly dump. I can’t easily find it but that’s where I got it from.
the whole reddit? how big is it?
Ask OpenAI, since it is in their dataset 😂
Speaking of, how do I back up the entirety of ChatGPT 4? I’ve got a couple of spare SD cards lying around.
I also recommend downloading “Flashpoint archive” to have flash games and animations to stay entertained.
There is a 4gb version and a 2.3TB version.
There is a 4gb version and a 2.3TB version.
That’s quite the range
When I downloaded it years ago it was 1.8TB. It’s crazy how big the archive is. The smaller one is just so it’s accessible to most people.
Is that Flash exclusive or do they accept other games from that era?
I’m not sure, but I do think it’s just flash
I stumbled across this sort of fascinating area of doomsday prepping a few weeks back.
A nice addition to that, don’t just make it a USB, but a raspberry pi. So you’d have a reasonably low-powered computer you could easily take with you.
Not suggesting this one as it seems a bit expensive to me, but https://www.prepperdisk.com/products/prepper-disk-premium-over-512gb-of-survival-content?view=sl-8978CA41
at this point why not just use a phone running postmarketos?
You’d first have to buy a phone that can run postmarketos and these are much rarer than I wish they were. Is there even anything new that can run it? Pine64 stopped making phones and said they’ll make a new one when they can make it RISC-V.
Fairphone maybe I guess. 4 is listed as a supported device, but someone has gotten it working on 6 too.
there’s lots of devices it runs on iirc, something like the pixel 3a can be had for less than a new rpi3b+ where I live
Last I checked (3 years ago) postmarketOS drained the pinephone battery in record time :(
Cause if ya wanna go overboard like I did, 1TB of NVME storage, can add with SD Card if necessary. 16GB RAM. Very little learning curve for my part as I use SBCs often. Plus almost every Docker container and program I want works on RPi without any hassle.
There’s also more robust guides and community for RPi.
Just my thoughts.
Just built one of these myself. I went NVME M.2 instead of SD Card to avoid data corruption. I know SD Cards are fine if you don’t write to them a lot but if you wanna update or add your own stuff, scares me. Plus NVME is just so much faster.
How would you access the info if electricity permanently goes out?
Pretty much what Sinthesis said; USB power brick and/or solar panels. Both at the ready and tested. Also got a big ass battery backup that will charge off solar panels.
You only need 20 watts of power. One of those dinky fold up solar panels would work. Add a USB power brick for cloudy days.
2W for a RPi Zero with data on a microSD
You’re going to need a monitor as well.
I have a PaPiRus ePaper eInk e.g. https://media.digikey.com/pdf/Data Sheets/Pi Supply PDFs/PaPiRus_ePaper_Web.pdf and even though I don’t know the watts for a refresh but I assume it’s one of the lowest solution you can use.
PS: FWIW if you don’t refresh the display can keep the information on for months, if not years.
Eink displays are pretty awesome for this sort of thing, I repuposed a kobo ereader as a household info display and it worked nicely. Those PaPiRus screens look easier to interface with, but a little small for reading wikipedia articles. They’d do in a pinch, but the eyestrain would have me looking for a bigger solution.
You find a generator, or solar panels, or wind mill, or water turbine, or a bicycle hooked up to a generator.
If electricity permanently goes out then we’re in a scavenger situation and it is time to start taking apart things that are no longer necessary to build the things that are.
start taking apart things that are no longer necessary to build the things that are
Hey finally a good use for all of those cars, grab the alternators out for small generators (since bicycles are the ultimate apocalypse vehicle: simple, small, easy to maintain and don’t require complex fuels)
I saw that Wikipedia was having funding problems, what happened to Debian?
They lie. Wikipedia has plenty of money. Do not give those parasites any more.
https://en.wikipedia.org/wiki/Wikimedia_Foundation#Spending_and_fundraising_practices
