borje justrell director ict department national archives, sweden 2005
DESCRIPTION
The Digital Black Hole Reflections on Costs for Long Term Storage of Digital Information. Based on an Article by Jonas Palm, Head of Department of Preservation. Borje Justrell Director ICT department National Archives, Sweden 2005. “I’ve invented a machine which can answer all questions!”. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/1.jpg)
The Digital Black Hole
Reflections on Costs for Long Term Storage of Digital Information. Based on an
Article by Jonas Palm, Head of Department of Preservation
The Digital Black Hole
Reflections on Costs for Long Term Storage of Digital Information. Based on an
Article by Jonas Palm, Head of Department of Preservation
Borje JustrellDirector
ICT departmentNational Archives, Sweden
2005
Borje JustrellDirector
ICT departmentNational Archives, Sweden
2005
![Page 2: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/2.jpg)
“I’ve invented a machine which can answer all questions!”
![Page 3: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/3.jpg)
…it knows all the answers but I can’t come up with the questions….
![Page 4: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/4.jpg)
The digital black hole
![Page 5: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/5.jpg)
$ €SEK
DKK
![Page 6: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/6.jpg)
$
GBP
YEN SEK€
AUD
BTN
![Page 7: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/7.jpg)
![Page 8: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/8.jpg)
…the digital information may be lost for ever. It will be as secretly kept as anything captured by the
Black Holes in space.
![Page 9: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/9.jpg)
![Page 10: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/10.jpg)
![Page 11: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/11.jpg)
![Page 12: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/12.jpg)
Initiatives - lots of energy
The Life Cycle of Digitization
projects
Life on the Web
PlanningFunding
Project up and running
The Good Ideas
Termination
Survival of the fittest
“Which
project…?”
![Page 13: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/13.jpg)
Digitization project costs…
1. The transfer to a digital format
2. Preserving the digital information
and keeping it accessible
![Page 14: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/14.jpg)
Scanned material
Born digital
Analog audio and video magnetic tape
Should be stored digitally
Must be digitized to survive
Stored analog on film for future use
Left in the digital domain until it “dies”
Stored digitally for future use
Some reflections…
![Page 15: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/15.jpg)
Storage demands for different kinds of
information; for comparison - approx 1 minute of
informationFile size of approx 1 minute digitally stored information
0
10
20
30
40
50
60
1
Format
Mb
Text A4
Image RGB 8bit 300dpi A4
Audio (CD quality) 1min, 16Bit 44100sampl freq
Video+sound (DVDmovie) 1 min
![Page 16: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/16.jpg)
Harvard University Library, 2004
Comparison of costs for storing the same information on different media
including the original on paper
![Page 17: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/17.jpg)
National Archives, Sweden 2005
Comparison of costs for storing the same information on different media
including the original on paper
Costs for storing the same information on different media
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
1Medieformer
COM 35 mm film
331 pages/ 165,5 sheets) inarchival box
693 Mb digital image files (1A4 = 600 dpi b/w (1-bit))
1,35 Gb digital image files(1 A4 = 300 dpi greyscale(8-bit))
![Page 18: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/18.jpg)
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
9.00
COM 332 pages 1-bit 600 dpi 8-bit 300 dpi
RA
Harvard
Comparison of costs for storing the same information on different media
including the original on paper
National Archives and Harvard University Library
USD
![Page 19: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/19.jpg)
Decrease in storage costs for 1 Mb
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
År
€
Decrease in price for computer processing same kind of information
0.0
500.0
1000.0
1500.0
2000.0
2500.0
År
€
Expected decrease in price of HSM-system
050000
100000150000200000250000300000350000400000
År
€
![Page 20: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/20.jpg)
Storage Costs in HSM systemall costs in SEK
Storage Costs in HSM systemall costs in SEK
Storage costs in HSM system 1st year 2nd year 3rd year 4th year 5th year 5 years Cost/year 1 HSM storage system price 2003 + 3% interest/ year 948189 923791 899392 874994 850595 4496961 Price 4066430: - spread over 5 years; 813286: -/year
Storage medium 40 Tb/år 179300 112959 71164 44833 28244 436500 Staff 60% , operation, à 400000 :- incl all costs 240000 252000 264000 276600 289800 1322400 Staff 40% , input of data, à 400000 :- incl all costs 160000 168000 176000 184400 193200 881600
Premises 100m2 à 1260:- 126000 129150 132379 135688 139080 662297
Service/s upport 227000 289000 289000 289000 289000 1383000 Yearly Storage Cost (equipment, operation, service) 1541189 1593941 1584771 1576282 1568475 7864658 Yearly storage cost (equipment, operation, service) Gb/ år 38.53 19.92 13.21 9.85 7.84 89.35 Yearly Input Cost (staff, storage medium) 339300 280959 247164 229233 221444 1318100 Yearly input cost (staff, storage medium) Gb/ år 8.48 7.02 6.18 5.73 5.54 32.95
Tota l cost Gb/ år 47.01 26.95 19.39 15.58 13.38 122.31
To al cost 1 GB for 5 years 122.31
![Page 21: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/21.jpg)
Costs for equipment and storage medium
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
1000000
1styear
2ndyear
3rdyear
4thyear
5thyear
SEK
Equipment
Storage medium 40Tb/år
Storage costs per Gb
0
5
10
15
20
25
30
35
40
45
50
1 2 3 4 5
Year
SEK
Yearly storage cost(equipment,operation, service)Gb/år
Yearly input cost(staff, storagemedium) Gb/år
Total cost Gb/år
Operating Costs
0
100000
200000
300000
400000
500000
600000
1 2 3 4 5
Year
SEK
Staff à 400000:-incl all costs
Premises 100m2 à1260:-
Service/support
![Page 22: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/22.jpg)
Distribution of costs for storage in HSM
system at the National Archives, Sweden
Distribution of costs for storage in HSM
system at the National Archives, Sweden
![Page 23: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/23.jpg)
![Page 24: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/24.jpg)
![Page 25: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/25.jpg)
…but the scanning…?What does that cost?…but the scanning…?What does that cost?
![Page 26: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/26.jpg)
Distribution of scanning costs
Costs based on production costs at the scanning facility MKC, National Archives,
Sweden
Activities involved in feed scanning 1-bit 600 dpi A4 page
4%
3%
17%
33%12%
2%
14%
15%Registration
Archiving
Preparation
Scanning
Image control
Corrections
Extras
Administration
![Page 27: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/27.jpg)
Distribution of scanning costs
Costs based on production costs at the scanning facility MKC, National Archives,
Sweden
Activities involved in scanning 8-bit 297 dpi greyscale of A2 page
2%
2%
2%
65%
4%
1%
5%
18%
RegistrationArchivingPreperationScanningImage controlCorrectionsExtrasAdmnistration
![Page 28: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/28.jpg)
Total cost ≈€ 90,000,000
Digitizing costs for A/V as suggested in Swedish report covering 4,5 million hours on tape
30% audio tapes = 1,350,000 h = 810Tb in CD-quality
70% videotapes = 3,150,000 h = 7,245Tb in DVD quality
“Due to condition and technical circumstances transfer should be made
within the next ten years”
Total of 8,055Tb if digitized to CD- and DVD format
Facts from report published by The Ministry of Culture, Sweden, 2004, Bevara ljud och rörlig bild (SOU 2004:53)
(Preserving sounds and images)
![Page 29: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/29.jpg)
Digitizing costs for A/V (1 year)
Digitizing A/V material 805,000Gb/year
€ 9,000,000
€ 11.18/Gb
Facts from report published by The Ministry of Culture, Sweden, 2004, Bevara ljud och rörlig bild (SOU 2004:53)
(Preserving sounds and images)
Activities involved in digitizing A/V objects
2%3%10%
75%
1%
2%
5%
2%
Registration
Archiving
Preparation
Transfer to digital
Control
Corrections
Extras
Administration
![Page 30: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/30.jpg)
Scanning in Gb in Swedish scanning projects
Amount of Gb in Swedish digitization projects including A/V
0
100000
200000
300000
400000
500000
600000
700000
800000
900000
Gb
A4 1-bit images
A2 8-bit greyscale images
A/V
![Page 31: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/31.jpg)
Scanning costs costs per Gb
0
5
10
15
20
25
30
35
€
prod cost € per Gb 1-bit
prod cost € per Gb 8-bit
prod cost € per Gb A/V
Scanning costs compared
![Page 32: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/32.jpg)
Swedish project will allow lesser quality digitization
e.g.
One person should be able run up to 12 taperecorders
Transfer from analog to digital at double speed
Facts from report published by The Ministry of Culture, Sweden, 2004, Bevara ljud och rörlig bild (SOU 2004:53)
(Preserving sounds and images)
![Page 33: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/33.jpg)
Huge investments in digitizing must be met with adequate arrangements for preserving the work done
![Page 34: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/34.jpg)
$
GBP
YEN SEK€
AUD
BTN
![Page 35: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/35.jpg)
The Future
![Page 36: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/36.jpg)
![Page 37: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/37.jpg)
![Page 38: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/38.jpg)
2D bar code2D bar code
May store any kind of digital information on film - text, image, audio, video
![Page 39: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/39.jpg)
2D bar code2D bar code
1 m of 16mm film can today store 30Mb
![Page 40: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/40.jpg)
The Rosetta stone
Greek text
Demotic text (Egyptian text)
Hieroglyphs
With 2D bar code solution each film will begin with plain readable text on what the film contains and what is needed to access its information content.
![Page 41: Borje Justrell Director ICT department National Archives, Sweden 2005](https://reader035.vdocuments.net/reader035/viewer/2022062519/56814cf3550346895db9f5fb/html5/thumbnails/41.jpg)
Conclusions…
…depends on your agenda...