BBC Domesday Preservation Project
About the Domesday ProjectThe BBC Domesday Project was a partnership between Acorn Computers Ltd, Philips, Logica and the BBC. It marked the 900th anniversary of the original Domesday Book, an 11th century census of England.
It was compiled between 1984 and 1986 and published in 1986. It featured a 'survey' of the United Kingdom, in which people (mostly school children) wrote about the geography, history or social issues in their local area or simply about their daily lives. This was linked with maps, colour photos, statistical data, video and 'virtual walks'. It is estimated that over 1 million people participated in the project.
As well as data from public input, the project also included professionally-prepared video footage, virtual reality tours of major landmarks and other prepared datasets such as the 1981 census.
It is often cited as an example of 'digital obsolescence' as the physical medium used for data storage (LV-ROM) was very quickly superceded meaning there are now very few LV-ROM players available and in working order and the discs themselves are also now starting to suffer.
Preservation of the BBC Domesday Data
With the addition to the collection of another Acorn Domesday System, volunteers at the Centre for Computing History set about the task of preserving the precious data stored on the LVROM discs that the system uses.
With the addition to the collection of another Acorn Domesday System, volunteers at the Centre for Computing History set about the task of preserving the precious data stored on the LVROM discs that the system uses.
Although we now had two complete systems, neither of them worked properly, but with a little swapping of components in the Philips LVROM player, and an easy battery pack re-make for the Acorn BBC Master Turbo, we soon had one fully working system.
Whilst we are fully aware that other projects have attempted to archive data from the Domesday System with varying degrees of success, none have made it publicly accessible via the internet. We feel that having the information available to all is very much in the original spirit of the Domesday Project and we hope that any copyright issues can be easily resolved.
 Original Domesday Brochure |
There has been much talk online regarding copyright issues that prevent the information contained on the LVROM disks to be re-published online. We are looking into this issue.
We are currently taking the view that we would be publishing this content on a strictly non-profit basis and for educational use only. We are publicly requesting anyone that objects to this to contact us. We will of course respect any infringement of copyright claim and work with the claimant to remove the offending data.
We are currently taking the view that we would be publishing this content on a strictly non-profit basis and for educational use only. We are publicly requesting anyone that objects to this to contact us. We will of course respect any infringement of copyright claim and work with the claimant to remove the offending data.
People Involved
Tim Porter first experienced the Domesday System on his first visit to the museum on a Volunteer Day in May 2010. Taken with the unique way that the system works and fact that the information can only be accessed via this obsolete and endangered hardware, Tim offered his valuable time to help preserve this unique insight into UK history. He has been responsible for the drive and fast pace of the project.
Jason Fitzpatrick is the founder of the Centre for Computing History and has always been keen to undertake preservation projects of this nature. With Tim offering to reverse engineer the data, Jason took on the web interface side of the project.
Diary of Domesday Preservation Project
June 2010 - Transferring the Data
The first hurdle was to work out how to get the program and data files from the LVROM onto a media that could be easily ported to a PC. We knew that people had written programs to read data from LVROM and then squirt it through the RS423 serial port to a PC, but we opted to use a more modern approach!
The first hurdle was to work out how to get the program and data files from the LVROM onto a media that could be easily ported to a PC. We knew that people had written programs to read data from LVROM and then squirt it through the RS423 serial port to a PC, but we opted to use a more modern approach!
We purchased the external version of the RetroClinic DataCentre and connected it to the beeb. I have to say that I was extremely impressed with the quality of the product. Not only with the build, but the first class presentation and detail of the supporting documentation. The manuals were presented like the original Acorn manuals with proper wire spiral binding, in colour, and they were very well written. I can highly recommend it.
The one problem we encountered with the DataCentre (which was not at all a fault of the DataCentre) was the fact that it required two ROMs to operate. The first ROM effectively updated the ADFS to include the extra code to operate the DataCentre and the second RAMFS ROM offered a set of utilities. The problem was that the Domesday System software required the spare ROM sockets in the beeb to be configured as Sideways RAM meaning they were not available for the DataCentre ROMS.
Fortunately we had also bought the RetroClinic Switchable MOS which also had the updated ADFS code. So we installed the switchable MOS in place of the original MOS chip leaving the other two sockets free to be used as sideways RAM. This meant that we were unable to install the RAMFS ROM, but fortunately, for the purposes of ripping the data from the LVROM, it was not required.
The CFCARD that came with the kit also had some utilities on it. One very useful one was called "TreeCopy". It allows you to copy files between file systems, so copying the files from VFS to ADFS *should* be a fairly straight forward affair ...
Tim Porter & Jason Fitzpatrick - In Action!
11th July 2010 - Volunteer Day & Ripping The Discs
Sunday saw another busy Volunteer Day. While others worked on stuffing Amiga 1200's with loads of demoscene software, we set about extracting data from obsolete LV ROM discs!
Sunday saw another busy Volunteer Day. While others worked on stuffing Amiga 1200's with loads of demoscene software, we set about extracting data from obsolete LV ROM discs!
We started the "TreeCopy" application at 8:30am, it was still copying the data at 10:30pm... We decided to let it roll over night and by 8:30am the next morning we had a complete dump of one side of the National Disc on a CF Card. However, this was only half the battle. The next problem was capturing the video content of the discs...
An image of the CF Card was uploaded to our web space ready for Tim to start reverse engineering. While he was doing that we started the long process of ripping the other discs. Fortunately, the second side of the National Disc appeared to be just video so that'll be a straight video capture process. Over the next couple of days we ripped both the Northern and Southern sides of Community Disc. So now we have all of the 'digital' data preserved from those two important discs. Later we will do the same for the Eco Disc and the Countryside Disc. If anyone else has any other discs, please do get in touch with us.
12th July 2010 - Reverse Engineering the DataTim has spent a huge amount of time looking at the raw data files contained on the LV ROMs. He has made some great break-throughs early on which is very encouraging. He has dumped a considerable amount of text from the files and worked out some of the de-limiting codes. He has also possibly worked out how the articles link to the relevant images ...
19th July 2010 - First Demo Website
Tim has spent what must have been most of the weekend extracting data and splitting up the captured video into individual JPG images. Using some clever PHP, Tim has created an index of
over 9600 articles from the Community Discs. Each article links to a detailed page containing text and images if present. The images folder contains over 85,000 images although some of those appear to be blank frames. There's some very interesting articles in there and the images have a lovely 1986 feel to them. It's a flat export though with no interactivity. So I guess it's now my turn to create some kind of front-end for it!
Tim has spent what must have been most of the weekend extracting data and splitting up the captured video into individual JPG images. Using some clever PHP, Tim has created an index of
over 9600 articles from the Community Discs. Each article links to a detailed page containing text and images if present. The images folder contains over 85,000 images although some of those appear to be blank frames. There's some very interesting articles in there and the images have a lovely 1986 feel to them. It's a flat export though with no interactivity. So I guess it's now my turn to create some kind of front-end for it! 20th July 2010 - Front End Thoughts
Have been thinking about how to present the data from the Domesday System. Should we try to emulate the experience of using the Domesday System or simply preserve and present the data that it contained. We have a working system so anyone wanting to use the original system can do. Technology allows us to make much better use of the data now, so we probably should use that technology.
Have been thinking about how to present the data from the Domesday System. Should we try to emulate the experience of using the Domesday System or simply preserve and present the data that it contained. We have a working system so anyone wanting to use the original system can do. Technology allows us to make much better use of the data now, so we probably should use that technology.
Who will be the end-user? Bearing in mind that the 1986 Domesday Project was a snap-shot of life in the UK in the mid-eighties, it is fair to assume that most of the the end users will be people studying and researching UK history. They will be more interested in the data than the technology of the system.
For this reason, and at this point in time, we have decided to present the data in a user-friendly way rather than try to emulate the original user interface. However, given time, we hope to include features that will show the original system in use.
22nd July 2010 - LVROM Thoughts
Although most people seem to think the Domesday System equipment is somewhat delicate, it seems to me that it is actually rather hardy. I have been using the stuff out of it over the past couple of months with 14 hour continuous data transfers, sequential frame displays and the like. It seems to hold up just very well! Anyway, I was thinking about the eventuality of our LVROM player popping its clogs which lead me to thinking perhaps we could create a solid state version of the LVROM player? It would be a case of a SCSI interface, a microcontroller and an MMC or CF memory device. Some jiggery-pokery would be needed to give it a video output to display the images, but it is not impossible...
Perhaps we should just get the first part of the project done first...
4th Sept 2010 - Been Busy...
There's been quite a bit going on but not a lot to show for it. Tim has been cutting up the video files we captured from the LV discs into individual JPEG images that can then be referenced by the code we have online. There's 44,300 images for the Northern side of the Community Disc and 41,801 images for the Southern Side. Not all of them are actual images, some are blank frames from the video capture, but still... that's quite a lot of images!
We've still got to capture the video from the second side of the National Disc which is BBC news footage from 1980 to 1986...
25th Sept 2010 - Reverse Engineering Graphs
We're assembled again at the museum to crack on with the project. Alasdair has been putting a huge amount of effort into decoding the data contained on the National Disc. Specifically, he has been looking at how the data is stored for the huge number of graphs and charts contained within. Armed with his laptop of raw data and the real Domesday System he has managed to make sense of quite a lot of the data but there is still much more that remains a mystery.
4th Sept 2010 - Been Busy...
There's been quite a bit going on but not a lot to show for it. Tim has been cutting up the video files we captured from the LV discs into individual JPEG images that can then be referenced by the code we have online. There's 44,300 images for the Northern side of the Community Disc and 41,801 images for the Southern Side. Not all of them are actual images, some are blank frames from the video capture, but still... that's quite a lot of images!
We've still got to capture the video from the second side of the National Disc which is BBC news footage from 1980 to 1986...
25th Sept 2010 - Reverse Engineering Graphs
We're assembled again at the museum to crack on with the project. Alasdair has been putting a huge amount of effort into decoding the data contained on the National Disc. Specifically, he has been looking at how the data is stored for the huge number of graphs and charts contained within. Armed with his laptop of raw data and the real Domesday System he has managed to make sense of quite a lot of the data but there is still much more that remains a mystery.
Tim & Alasdair comparing the real world events with the raw data.
Tim has made a huge leap forward on the web interface and navigation of the maps which in turn link to the individual articles. Its still very much a debugging interface but still very useable. It's quite interesting navigating the map to specific areas to see what information it has about it...
I have been busy with other aspects of running the museum, but have today captured some better quality video using a Sony Lasermax Laservision Videodisc Player (LDP-3600D). It seems to give a cleaner image with less artefacts and sparklies. We'll have the video online soon. It looks like if we want to do this in the best way we can we'll have to re-capture all the video and image frames again!!! Doh!While we have everything set-up we've been looking at what other Laserdiscs we have. There's some very interesting Anglia TV recordable Laservision discs, Ferranti IIS, Marconi VORAS Disc, INMERC Project, Acorn IV Robotics Disc (Including 5.25" Floppy Discs) amongst other Learn to Play Golf and general educational discs! Given time, we'll investigate these discs further...
27th September 2010 - Promo Video Online
 |
| Watch the Domesday System Promotional Video Presented By Nick Ross |
It's taken a while to get round to it, but I finally got the promotional video laserdisc captured and put online. It's a very interesting watch and does an excellent job of providing a detailed overview of the Domesday System and what it is capable of. It's presented by Nick Ross (of CrimeWatch fame!)
Click on the image on the right to watch it online ...
5th April 2010 - Walk Throughs Cracked
Tim and Alasdair have now cracked the way the walk-throughs work on the community disc. We have a basic interface working online - we just need to make the UI more attractive.
Click on the image on the right to watch it online ...
5th April 2010 - Walk Throughs Cracked
Tim and Alasdair have now cracked the way the walk-throughs work on the community disc. We have a basic interface working online - we just need to make the UI more attractive.
18th April 2010 - On Display at Gadget Show Live
We had one of our real working Domesday Systems on display at the Gadget Show Live. It generated a lot of interest and prompted a number of people to get in touch with us regarding the project. We demonstrated the online system as it currently stands.
Good Pictures : Tim Porter, Poor Pictures : Jason Fitzpatrick
