Germany · de.indymedia · · 3h
Critical infrastructure: There is a lack of strategy when it comes to web archiving
Deutsch (original) · Auto-translated to English
Web archives are critical infrastructure, says historian Jens Crueger. He calls for a plan for archiving in Germany and explains what politicians should do - and what each individual can do.
Electricity pylons, hospitals, submarine cables – the list of things without which our daily coexistence would not function is long. We also know it under the collective term “critical infrastructure”. But when it comes to important infrastructure, there is a lack of attention and a strategy, warns digital historian Jens Crueger at the Data Traces Conference in Dresden.
As a historian, Crueger has been working on the topic of web archiving for many years. Thehttps://de.wikipedia.org/wiki/Jens_Crueger">frformer member of the Bremen citizenshipis an editor athttps://irights.info/author/jenscrueger">irights.infoand is doing his doctorate at the Faculty of Health Sciences in Brandenburg on the history of the autistic movement in the 1990s. “One of the things I do is look at how autistic people organized themselves on the web back then.” It's not always easy for him.
Why we need web archives
Contrary to popular belief that the Internet never forgets anything, large parts of the web disappear quite quickly. According to digital historian Crueger, 38 percent of websites from 2013 were no longer accessible a decade later. The average lifespan of a website is only around 100 days; pages with dynamic web content change every minute. “The web is ephemeral,” says Crueger.
In addition to potentially important information disappearing as a result, this is also a historical loss. The result: “The early web is hardly preserved anymore,” says Crueger. Anyone looking for information from this period often has to rely on analogue means. “A printed screenshot in a print journal is sometimes more permanent than digital.”
Web archives store pages with as many of their elements as possible, from links to interactive graphics. This is not only relevant for historians and nostalgics. Journalists research the historical data, lawyers argue it in court and AI companies use it to train their models.
Don't leave everything to the Internet Archive
Currently, web archiving takes place primarily in the USA, at the Internet Archive, a non-profit foundation, says Crueger. The Internet Archive began archiving in 1996, and last year it saved its billionth website. A undoubtedly successful project. Nevertheless, we shouldn't rely on this one project, says Crueger.
Because the Internet Archive is always under pressure. The mass data retrievals from AI companies – some but not all of which would pay for the data – were already pushing the Internet Archive to its technical limits. Politically, the location in the USA represents a risk. “Since Trump took office, one of the Internet Archive server locations has been outside the USA, in Canada,” says Crueger in his lecture.
But even regardless of these problems, the question arises as to whether web archiving should be left largely in the hands of a single non-profit foundation.
–––
The hut is on fire! We're going in.
We oppose fascisation. We fight for a free, open and solidarity-based society. We defend basic rights for everyone.https://netzpolitik.org/spenden/?via=rss">Thisonly possible with your support.
–––
Because archiving is always linked to the decision of how to collect and what to preserve. An important question: How do you deal with disinformation? Crueger reports that during the Corona pandemic, disinformation continued to circulate even when the actual website had already been deleted - as a screenshot from the Internet Archive. Disinformation in the archives is also a problem because AI models are trained with this data. On the other hand: “There is a lot of research being done on fake news sites,” reports Crueger.
What Germany archives
For the historian, web archiving is also a government task. “We do the same thing with other archiving jobs,” says Crueger in an interview with netzpolitik.org. The German National Library (DNB) has onehttps://www.dnb.de/DE/Professionell/Sammeln/kommen_node.html#sprg733752">legal mandate for web archiving, but “only does what she has to do,” says Crueger.
There was only one systematic web crawl of all .de domains once, over 10 years ago. The DNB also regularly saves the content of selected German websites and operates “event harvesting”, i.e. it specifically collects German web content on specific events such as the 2024 Olympic Games or the war in Ukraine. The DNB collection includeshttps://www.dnb.de/SharedDocs/Downloads/DE/Ueber-uns/zahlenFakten.pdf?__...statisticsover 82,000 entries from almost 10,000 different websites.
Get out of the reading room
But even this comparatively small collection is not easily accessible. Because the copyright law provides for a “terminal barrier”. So if you want to browse through the entire web archive, you have to go to the DNB reading rooms in Leipzig or Frankfurt am Main, or to a library that cooperates with the DNB.
Crueger is calling for this terminal barrier to be removed. “As long as the terminal barrier exists, there is no incentive for politicians to invest more resources in web archiving.” If the DNB's web archive were more accessible, perhaps more people would see how little is currently stored there and ask questions.
The next step would then be more money: “The budget for web archiving must be increased so that you can regularly crawl the .de web,” demands the digital historian. For comparison: the DNB's last .de crawl in 2014 costhttps://www.dnb.de/SharedDocs/Downloads/DE/Ueber-uns/zumSammelvertragDNB...Euro plus 50,000 eurosannual hosting costs for the stored content. The Internet Archive had issues of about the same timehttps://projects.propublica.org/nonprofits/organizations/943242767">15.6million US dollars in a year.
A strategy is needed
For Crueger, however, the central problem is different: "There is no really well-defined collection mandate. There is no good strategy. The whole thing is still in its infancy."
–––
The work of netzpolitik.org is financed by donations from our readers.
Become part of this unique community and support our public interest-oriented, advertising- and tracking-free journalismhttps://netzpolitik.org/spenden/?via=rss">nowwith a donation.
–––
Such an archiving strategy must include different perspectives in order to ask which things are worth preserving: "Archivals, journalists, historians - all groups that have a possible interest in it. And, if possible, citizens selected by lottery," suggests Crueger.
In Ireland, he says, the archive there also asked citizens in 2016 which pages particularly depicted life in Ireland. “It was really done with a very participatory approach,” enthuses Crueger. Experts would probably have made a different selection of sites. “That’s what makes the method so exciting.”
Archive yourself!
Initially, Crueger's main aim is to create more awareness of the topic. He has had good experiences showing people in clubs earlier versions of their websites. “If you start directly from personal experience, you have a chance that people will take the topic seriously.”
Crueger's lecture is also a plea for people to archive themselves. “No part of the American web is as well archived as that of marginalized groups.” Especially when there is little trust in the state, the incentive to store important things yourself increases.
Technically inexperienced users can access websites with one clickhttps://web.archive.org/">atInternet Archivessave. Crueger also refers to a freely available manual that...https://nestor.sub.uni-goettingen.de/handbook/nestor-handbook_23.pdf">Gr...long-term archivingexplained.
Thehttps://talks.datenuren.de/ds26/talk/WQGLCL/">Lecture“Web archives are critical infrastructure!”by Jens Crueger took place at the data trails in Dresden on Friday, September 18, 2026. He will shortlyhttp://media.ccc.de/c/DS2026">onmedia.ccc.de availablebe.
–––
The work of netzpolitik.org is financed by donations from our readers.
Become part of this unique community and support our public interest-oriented, advertising- and tracking-free journalismhttps://netzpolitik.org/spenden/?via=rss">nowwith a donation.
Read the full story at the source
Source: de.indymedia