International Journal of Naval History

Historical Research in the Digital Era:

Techniques, and the More Obvious Pros and Cons

Howard J. Fuller,

King’s College, London—Department of War Studies

[The IJNH would like to thank the author and 

the Journal of Maritime Research for their permission to reprint this article]

My Discovery:

I have tried something different in the course of my doctoral research.  It was not recommended by anyone; as far as I knew no one had attempted it, and perhaps it was only recently that digital research was feasible.  I began my archival research taking notes by hand on paper, transcribing them onto my laptop computer, and finally, making photocopies of documents when they seemed most necessary.

It was not long before written notes seemed like scraps of paper.  Microsoft Word documents constituted the bulk of my work, and happily so.  I could save my material, make backup copies.  Type in a few words and I could utilise key word searches to locate that one note amongst hundreds, more and more.  I could organise everything into virtual ‘Folders’ on my laptop, subdivided into more Folders—and carry it all in a briefcase, everywhere.

The photocopies were another matter.  They are expensive in the United Kingdom!  Forty-pence per A3-copy at the Public Record Office at Kew (they will not consider an A4 sheet, in order to fit the proper citation in the margin, and documents cannot be ‘shrunk’ to fit.)  If I encountered a ‘crucial’ printed Admiralty report, for example, I had two choices: either spend a long time transcribing what I felt I needed from the report, or photocopy it; time or money.  The problem with any transcribing in an archive, however, whether by hand or typing into a laptop, is the element of human error we are all familiar with.  Was that sentence the most important one in the paragraph after all?  Am I neglecting the larger context?  What if what I felt was important then has since changed?  Was that really a colon, or a semicolon?

Even if I felt had money, if not time, to spare for photocopies—to give me the whole document without any future doubts as to what it said or not—I began to accumulate another stack of paper.  Remember several hundred documents at hand, organised in a file cabinet, spread around your room or office during the chaotic process of actual writing?  These are hard memories, and in hindsight alone perhaps, even ‘fond’ ones.  But I confess I am not so sentimental.  Information recorded on paper is vulnerable.  Lose your time-consuming notes or costly photocopies, and then what?  However, information stored on computers can be just as vulnerable.  I was wisely instructed by my school to always make backups of my work on computer.  If the computer crashed, if it was stolen, if I left it on the tube (everyone seems to do that now and then!) all my work was gone too.  The same can be said for all my material.

My thesis has involved research abroad as well.  Since most of my work was on laptop, backed up on floppy discs (and then, as it piled up, on zip-discs of 100+ megabyte capacity) all I needed to take with me was my laptop.  But what of all the photocopies?  Either I left them behind or dragged them through the airport.  What of the photocopies I was sure to make elsewhere—to drag back as well?  At this point I invested in a scanner.  The most important photocopies could be scanned into my laptop, hard-drive memory space permitting, and then also, backed up on discs for protection.  Now I could take even more with me to and from anywhere, all on a single laptop.  This was the beginning of my ‘digital research’:

1. To facilitate mobility.

2. To cut down on the sheer bulk of materials; now all easily organised and accessible on a PC or laptop.

3. To enable full and perfect multiple copies and backups, made within minutes.

I will add here another implicit advantage: once the material was digitised onto my laptop, I could send copies of material to colleagues either on floppy disc, recordable CD-ROM (CDRs or CDRWs, “re-writables”) or email documents to anyone, anywhere, instantly.  Furthermore, I could always print this digitised material off on a simple bubblejet printer; so could anyone who received a copy from me via email or disc.

But scanning photocopies or even handwritten pages of notes was not enough, especially when researching abroad.  The biggest practical concern for a researcher abroad is money, if not time.  Every day you spend transcribing material in an archive is a night spent lodging.  How many of us have found ourselves pulled away from a letter collection because we simply did not have the time or resources to explore more fully than we wished!  The ‘game’ has always been trying to extract as much material as quickly as possible, hoping that we do not overlook something in the process, to have to return to again at ‘another time’.  Digital research is therefore more than simply digitising historical documents; it is perhaps more importantly about getting the documents in the first place.  As such, I soon realised that a scanner was simply not enough; scanning in documents by night which I photocopied by day, at my own expense.  About this time, the market for digital cameras did two wonderful things: it introduced the 2-million ‘mega-pixel’ range camera (or higher), and dropped the prices.

A word about digital cameras:  It is a misnomer in my opinion to consider them ‘cameras’.  They are more like hand-held scanners, or even photocopiers.  The first generation digital cameras gave you a one mega-pixel range resolution; nice for a family vacation, perhaps, but not very good for picking up sufficient detail to legibly reproduce a hand-written letter, or say, newspaper print.  I know, I tried.  Pro and con: digital cameras never require film, there are no developing costs, ever, because every image produced with the click of the shutter is digital information stored onto the camera’s memory card; however, you cannot operate a digital camera—or realistically own one for that matter—unless you also have a computer.  When your digital camera’s memory card fills up, it has to be downloaded into a computer.  This is done in at least two ways; normally through a USB cable connection from the camera directly to the PC or laptop, or, much more conveniently (because there is no wire and it is even faster), through a PCMCIA-port adaptor on the side of a laptop computer.  In the latter case, the memory card fits into a slot in the credit-card size adaptor, which is then inserted into the PCMCIA port.  My camera can download 40+ images (each one saved in the ‘JPEG’ graphic format) into my laptop in approximately 7 seconds.  The entire process of ejecting the memory card, inserting it into the adaptor in the laptop, downloading, disabling the card for ejection, and inserting it back into the digital camera for more photography, takes me 40 seconds on average.  “Electronic Developing” is therefore instantaneous; you can check your image on the spot before moving on.

Another important distinguishing characteristic about digital cameras is that they ordinarily do not require a flash, especially when ‘shooting’ documents at a table with standard lighting.  A useful analogy might be digital camcorders, which do not require lighting except in very low-lit rooms or at night.  The technology is the same, including the digital ‘LCD-screens’ on both digital camera and camcorders which allow you to see the image you are shooting, as well as looking through a shutter.  (I always use the LCD, to help frame the image better.)

More particulars: digital cameras have variable resolution settings, and a few ‘special effects’.  My camera[1] gives me the option of ‘Quality’ and ‘Resolution’, I am still not sure what exactly the differences is, but after no small amount of testing, settled for ‘Low Quality’, ‘High Resolution’ to take perfectly legible photographs of letters and newspaper print, for example.  Since colour is rarely an issue when considering documents, I also save on memory (and therefore the size of each JPEG image-file/shot) by shooting in the optional ‘Black and White;” more on this later.

Now, with such a device, I could go into an archive (permitting!), consult documents, and reproduce them if need be on the spot within seconds—at no charge.  On a good day I was scooping up nearly 300 pages.  There was no more nightly scanning; the transfer from ‘camera’ to laptop was part of the copying process.  Frankly, it took weeks for me to snap out of a rather giddy delirium, I could not believe my luck!  I had previously spent too much time, too much money, and too much effort in the course of my research by comparison.  I was now capable of covering four, even five times’ the amount of material, in the same amount of time, with unquestionable precision.  In fact, the ‘treasure’ I had gradually discovered—what no one else apparently had before—was incalculable:

After having touched,  felt, examined these treasures, Edmond rushed through the caverns like a man seized with frenzy; he leaped on a rock, from whence he could behold the sea.  He was alone—alone with these countless, these unheard-of treasures!  Was he awake, or was it but a dream?[2]

The Technique:

Since I was clearly pioneering into a form of historical research, utilising the latest market technology, I had little idea what to expect.  Of course I was aware of the advantages, but what about any possible disadvantages?  The first problem I encountered was power supply.  My digital camera, like most I believe, did not include an AC-chord.  To order this off the website would cost me over £30, if I remember correctly.  The alternative was four AA-batteries, which did come with the camera; but digital cameras (LCD-screen activated or not) consume a lot of power.  Those standard AAs went cold after four hours.  Nor do I like the idea of a power chord dangling from my camera, over a crowded table with precious documents and my laptop...  So, the best solution for me was to invest in two sets of four rechargeable AA-batteries, and a battery charger.  Nickel-hydride batteries are much better than nickel-cadmiums, incidentally.  You can purchase chargers now that accommodate four AAs at a time, with four AA nickel-hydride batteries, for around £15.  But you will always want at least two sets: one in the camera, the other always charging at a nearby outlet.  Fortunately most national archives, public libraries, or local county record offices have accessible power outlets.  If you want your digital camera shooting for upwards of 6-8 hours, then a third, previously-charged set, is also recommended.  When you get home, begin charging your drained sets immediately, to be ready the next morning.  No electrical power means no digital research!

I have already given my opinion on the actual memory-downloading process, preferring PCMCIA-port adaptors for laptops rather than another dangling (USB) chord in a workspace.  Keep your digital camera mobile.  A much more serious problem concerns memory.  I settled for a resolution-setting which was the best combination of memory-economy and image quality.  Even so, the average size of each JPEG image-file/shot was 200k, or, a quarter of one megabyte (which equals 1,000k).  This makes sense; as I said before, I averaged about 40 shots before my digital camera’s 8-megabyte memory card ‘filled up’.  If I could manage 300 such photos per day, I was therefore dumping 60 megabytes of data into my laptop’s hard-drive.  After a week this meant some 300-360 megabytes—over a quarter-gigabyte of hard-drive memory.  Do this for a couple of months and you will simply run out of space; at least that was the case with my laptop which (originally) had what was then regarded as a gigantic 6-gigabyte memory capacity.  Even though 20-gigabytes is standard now on new laptops, let alone PCs, you see the danger.

The problem was not so much stretching my laptop’s memory with this flood of digitised historical research material, but retaining the ability to back it up.  For this, even a zip-drive was not enough, unless I was willing to spend over an hour making copies on a dozen or so zip-discs.  (Backing up a gigabyte of digital data onto old floppies, by the way, will require some 714 discs!)  Greedy and in a hurry as usual, I was not.  Digital- and computer- technology seem to balance themselves out quickly enough, however.  Just as digital cameras became more affordable, CD-writers or ‘burners’ appeared on the market as well.  Your standard recordable CDR has a 700-megabyte capacity.  Only by now, the hardware required for digital research has begun to add up:

2-(million) mega-pixel range camera (or higher),

Two to three sets (of 4) AA rechargeable nickel-hydride batteries, plus charger,

CD-Writer (standard now with new PCs and some laptop-computers).

Another concern is the strength of your laptop itself (I assume laptop here if you plan on spending a day or two at a remote archive site) itself.  If your machine is an older model, it will at least need to have 64 megabytes of RAM, preferably 128, to actually manipulate all these JPEG files.  Hard-drive memory capacity should be at least 6 gigabytes to start with.  The processor itself should be Pentium-III, 650-megahertz (equivalent) or greater.  Digital-based research material, pouring nearly everything onto your laptop or PC, will likely push its capacity to the limit.  It can go either way, depending on how new your machine is, and what its ‘specs’ are.

Let me now explain another problem.  In the past several years my handy digital camera (or ‘scanner’, or ‘photocopier’) has taken over 26,000 photos, at least 22,000 of which are research-related.  How does one make sense of 20,000+ files?  Organisation is the key.  You simply cannot rely upon separate JPEG-files for document consultation on your computer.  Let’s say I found a four-page letter in an archive of which I wanted a full copy.  I take four shots to capture each page, making four JPEG-files, ‘DCP_0001’, ‘DCP_0002’, ‘DCP_0003’, ‘DCP_0004’.  I want the document in a multi-page file, however, that I can find and open when I want, rather than having to open four separate files to read that one letter.  Each JPEG-file would have to be renamed at any rate, making organisation of this research four times as tedious.

Of all the software programs I currently had, or since, the one I found the most suitable for this purpose was IMAGING for Windows (by Kodak).  First of all, it is a standard Windows program[3], like ‘Calculator’, ‘Notepad’, or ‘Paint’, found by going:

Start>Programs>Accessories

Secondly, it has a relatively easy interface, no more functions than you really need.  These include cutting, pasting, rotating and magnifying an image; as well as ‘Annotation’ tools so you can highlight passages within the image or even insert ‘Notes’ on pages within the document.  Going backwards and forwards to different pages is also straightforward.  But how do you get JPEGs into the Imaging format (which are saved as TIFF files)?  After much experimenting, I finally devised a technique:

Open your Folder with the saved JPEG-files, taken from your digital camera.  Minimise the screen to its thinnest size, and align it to the left of your screen.  The JPEGs should be listed in a long column, going down your screen, in a separate window.

Double-click a JPEG-file, opening it with whatever Photo Editor program your computer has.  In my case, I settled for Microsoft Photo Editor, which is a standard program with Microsoft Office 2000.  Again, minimise that window so that it aligns with the edge of your JPEG Folder window on the left on one side, and the right edge of your screen on the other.  You can pull or ‘stretch’ the window down about a third of the way down your screen.  You should now have two windows open, minimised, at the same time on your screen. Open Imaging; minimise that window to fit into the leftover space on your screen, filling most of it.

There are several things I should mention here before going on.  If you have a JPEG Folder with, say, 100 separate JPEG-files of documents you shot, you will need to go through those files in advance of the above procedure and indicate where one document ends and the next begins.  This ‘preparation’ phase of organisation is important because you do not want to wonder which JPEGs in your list relate to one document (ex. a letter) or another while you are reformatting them into Imaging.  I RENAME only the first ‘page’ of each document; so that JPEG-file “DCP_1127” becomes “DCP_1127a”.  What I am left with therefore is a list of JPEG-files, some of which have an “a” at the end, telling me how many pages constitute the final historical document itself, whether it’s one page or sixty.

The next thing I advise for preparation is creating a series of blank Imaging ‘template’ files.  When you open Imaging it starts you with a 1-page file.  Since most of your documents will be multi-paged files, you need to insert extra pages, and save each file separately as “1”, “2”, “3”, and so on, to as many blank Imaging templates you might think you will ever need.  I settled for up to “30” such files.  To create these blank Imaging templates:

(In Imaging) go to “File” (top toolbar)>New> hit “Ok”.

Go to “View” (top toolbar)>Page and Thumbnails.

Go to “File” again>Save As> type in “1”> hit “Save”.

Close this File.

Repeat Steps 1-2 above, starting with a new Imaging File.

Go to “Page” (top toolbar)>Append>Existing Page.  Double-click the Imaging File “1” just created.

Repeat Step 3 above, saving as “2”.

Repeat Steps 6-7, adding more and more pages to separately-saved Imaging Files, “1” to whatever number you wish.

Going back to the JPEG transfer technique above, you should open on Step 3 one of the pre-made blank Imaging template files just described, depending on the number of pages your historical document will contain—according to the indications you have already made in your JPEG-file Folder list (ex. “DCP_4137a” to “DCP_4147a”, or a ten-page document; therefore open your blank Imaging template file “10”.)  Now:

Right-click on the image on your Photo Editor program window at the top>Copy.

Right-click on the blank image (page 1) of your Imaging program window>Paste.

Rotate the image on your page of your Imaging program window if necessary (either by the toolbar, or “Page” (top toolbar)>Rotate Page>(Left or Right).

Click on the arrow to go the next page in your Imaging program window (either by the toolbar, or “Page” (top toolbar)>Next.

Double-click on the next JPEG-file in your JPEG-file Folder window.  It should open on top of the previous image in your Photo Editor program window.

Repeat Steps 1-5 as necessary.

When you have reached the last page of your document, go to “File”>Save As in your Imaging program window and type in a name for your new multi-page Imaging file.  Click Save.

How you name the new Imaging-file is also vital.  You want to be able to browse through a possibly long list of such completed files when consulting them, or be able to type in key-words when trying to find a particular Imaging-file/document.  You can do the latter by:

Start>Find>Files or Folders, and typing in appropriate key words

             In the case of letters, I settled for date; names; and a brief description of contents. 

Example:

“27-10-1862; Palmerston to Russell; French ironclad weaknesses; armour plate trials”

Keep your file names as succinct as possible, the longer the string, especially when combined with the Folder names they are in, the more likely your computer system will experience errors over time.  Also, you may experience short-term memory problems when conducting this type of multi-window copying and pasting.  I cannot explain why this happens, but on several occasions (on my laptop) I was shocked to see my hard-drive memory rapidly ‘dwindle away’ after an hour or so of this exercise non-stop.  Close down any unnecessary programs that may be running first; stop the operation for about 5 minutes; and magically enough (thank you Microsoft!) the hard-drive memory should reappear.

A much more serious warning: the process of carefully formatting all your digital camera JPEG-files, organising them as you please, and organising them still further into Folders, sub-Folders, and so forth, will take time, and can be extraordinarily monotonous!  My only consolation for this was that I could do it when, where and how I pleased, and the ‘beauty’, if you will, of the finished product.

Considerations:

What does this mean?  Personally-speaking, I now have some 7,700 separate historical documents, all on my laptop, all formatted as described above, all named for quick and easy reference, and all backed up (in several ongoing copies) conveniently enough on CD.  Nor is it just a matter of documents themselves; I have several hundred books, extracts of books, and articles; theses; photographs/art; anything that might be photocopied, scanned, or shot with a regular camera all on a laptop computer, courtesy of a digital camera.  In my opinion it is nothing short of a Revolution in how we can obtain, access, and store material for historical research.  Even despite the labour-time involved in file organisation (any less than going through mountains of paperwork?) it is much faster, cheaper, and more accurate than any previous technique.  It is easily duplicated, easily and instantly transmittable, and safer for the integrity of the sources themselves.

A common objection to ‘cameras’ for archive, record office or library research is that the flash damages the documents.  Digital cameras, as stated above, do not require a flash.  They are whisper-quiet in an archive, record office or library—much quieter than the person sitting next to you punishing his keyboard while he transcribes material.  Whenever a photocopy of a manuscript is made, it must be placed physically face down on the photocopier and subjected to a light more prolonged and intense than any camera flash.  Books or bound volumes have to be placed in a manner inevitably damaging to their binding.  A researcher does not handle the manuscript in question with a digital camera.  The document lays on a table, just as if the researcher were taking hand-written notes or transcribing into a nearby laptop-computer; a quiet, flashless shot; the page is turned as normal; followed by another. 

What happens when researchers begin generally substituting the old ways with the new one?  Historical knowledge is disseminated like never before.  Instead of collections of arbitrary handwritten notes or photocopies, whole sections of archive collections are preserved in private hands.  Some authorities may see this as a ‘threat’.  But I imagine if ever the P.R.O. is flooded, or another fire takes the British Library away from us, they will be grateful when the public can freely donate these collections back to a central depository.

There is, finally, another “Con” associated with historical research in the ‘Digital Era’.  Actually there are two: does making research easier necessarily make it better; and are we ready to write history off a computer monitor?  I cannot say, of course, but can only continue to offer my own opinion:

It is better to ‘over-research’ than otherwise.  Research to me is more a matter of skilful organisation of resources, not how many I have.  With all of the material literally at the touch of a finger, I can choose, as often as I need to, what is significant about a particular document, for example, and what is not.  Sometimes my opinions change.  But the documents themselves do not.  I will take my chances that this excess of ‘treasure’ will not spoil me too much.  In the end I would rather have that choice.

We are already ‘turning digital’.  Whether it’s reading news headlines, conducting daily correspondence, watching movies or even playing games, computers, the internet, and digital CDs are a reality.  Right now I can open up to 20 separate documents at a time on my laptop; their names are lined up along the bottom of my screen; I can switch backwards and forwards to them, or, splitting windows, view several simultaneously if need be, whether they are letters, newspaper or journal clippings, secondary sources, or my own note files.  So far, no screaming headaches, no disfigured back, and no loss of vision (though some do wonder about my attention span.)

The renowned American maritime and naval historian Samuel Eliot Morison was quoted as saying “History is to the community what memory is to the individual.”[4]  In the ‘Digital Era’ that sense of memory, and therefore also historiography, is everyday becoming a matter of new tools with greater capacity than ever before.  As always, the tools are only as good as the individual historian, always willing to ply his vocation for an insatiable, globalising community in a big hurry.