Menu Close

Category: historical research

After The Games Are Over: Olympic Archival Records

2 Comments

What does an archivist ponder after she turns off the Olympics? What happens to all the records of the Olympics after the closing ceremonies? Who decides what to keep? Not knowing any Olympic Archivists personally, I took to the web to see what I could find.

Olympics.org uses the tag line “Official Website of the Olympic Movement” and include information about The International Olympic Committee’s Historical Archives. The even have an Olympic Medals Database with all the results from all the games.

The most detailed list of Olympics archives that I could find is the Olympic Studies International Directory listing of Archives & Olympic Heritage sites. It is from this page that I found my way to records from the Sydney Olympic Park Authority.

The Olympic Television Archive Bureau (OTAB) website explains that this UK based company “has over 30,000 hours of the most sensational sports footage ever seen, uniquely available in one library”  and aims to provide “prompt fulfilment of your Olympic footage requirements”.

Then I thought to dig into the Internet Archive. What a great treasure trove for all sorts of interesting Olympic bits!

First I found a Universal Newsreel from the 1964 Olympics in Tokyo (embedded below).

I also found a 2002 Computer Chronicles episode Computer Technology and the Olympics which explores the “high-tech innovations that ran the 2002 Winter Olympic Games” (embedded below).

Other fun finds included a digitized copy of a book titled The Olympic games, Stockholm, 1912 and the oldest snapshot of the Beijing 2008 website (from December of 2006). Seeing the 2008 Summer Games pages in the archive made me curious. I found the old site of the official Athens summer games from 2004 which kindly states: “The site is no longer available, please visit http://www.olympic.org or http://en.beijing2008.com/”. The Internet Archive has a bit more than that on the athens2004.com archive page – though some clicking through definitely made it clear that not all of the site was crawled. Lucky for us we can still see the Athens 2004 Olympics E-Cards you could send!

Then I turned to explore NARA‘s assorted web resources. I found a few photos on the Digital Vaults website (search on the keyword Olympics).  A search in the Archival Research Catalog (ARC) generates a long list – including footage of the US National Rifle Team in the 1960 Olympics in Italy.

My favorite items from NARA’s collections are in the Access to Archival Databases (AAD). First I found this telegram from the American Embassy in Ottawa to the Secretary of State in Washington DC (Document ID # 1975OTTAWA02204) sent in June 1975:

 1. EMBASSY APPRECIATES DEPARTMENT’S EFFORTS TO ASSIST CONGEN IN CARING FOR VIPS WHO CERTAINLY WILL ARRIVE FOR 1976 OLYMPIC GAMES WITHOUT TICKETS OR LODGING. HAS DEPARTMENT EXPLORED POSSIBILITY OF OBTAINING 4,000 TICKETS ON CONSIGNMENT BASIS FROM MONTGOMERY WARD, WITH UNDERSTANDING THAT, AS TICKETS ARE SOLD, PROCEEDS WILL BE REMITTED? PERHAPS SUCH AN ARRANGEMENT COULD BE WORKED OUT WITH FURTHER UNDERSTANDING THAT UNSOLD TICKETS BE RETURNED TO MONTGOMERY WARD AT SOME SPECIFIED DATE PRIOR TO BEGINNING OF EVENTS.

2. EMBASSY WILL FURNISH AMOUNT REQUIRED TO RESERVE SIX DOUBLE ROOMS FOR PERIOD OF GAMES. AT PRESENT HOTEL OWNERS AND OLYMPIC OFFICIALS ARE IN DISAGREEMENT AS TO AMOUNTS THAT MAY BE CHARGED FOR ROOMS DURING OLYMPIC PERIOD. NEGOTIATIONS ARE CURRENTLY BEING CARRIED OUT AND AS SOON AS ROOM RATES HAVE BEEN ESTABLISHED, QUEEN ELIZABETH HOTEL MANAGER WILL ADVISE US OF THEIR REQUIREMENTS TO RESERVE THE SIX DOUBLE ROOMS.

Immediately beneath that one, I found this telegram from October 1975 (Document Number 1975STATE258427):

SUBJECT:INVITATION TO PRESIDENT FORD AND SECRETARY
KISSINGER TO ATTEND OLYMPIC GAMES IN AUSTRIA,
FEBRUARY 4-15, 1976

THE EMBASSY IS REQUESTED TO INFORM THE GOA THAT MUCH TO THE PRESIDENT’S AND THE SECRETARY’S REGRET, THE DEMANDS ON THEIR SCHEDULES DURING THAT PERIOD WILL NOT MAKE IT POSSIBLE FOR THEM TO ATTEND THE WINTER GAMES. KISSINGER

There are definitely a lot of moving parts to Olympic Archival Records. So many nations participate.  New host countries with the option to handle records however they see fit. I explored this whole question two years ago and came up against the fact that control over the archival records produced by each Olympics was really in the hands of the hosting committee and their country. A quick glance down the list of Archives & Olympic Heritage sites I mentioned above gives you an idea of all the different corners of the world in which one can find Olympic Archival Records in both government and independent repositories. Given that clearly not all Olympic Games are represented in that list, it makes me wonder what we will see on this front from China now that the closing ceremony is complete.

I also suspect that with each Olympic Games we increase the complexity of the electronic records being generated. Would it be worthwhile to create an online collection for each games – as has been done for the Hurricane Digital Memory Bank or The September 11 Digital Archive, but extend it to include access to Olympic electronic records data sets? The shear quantity of information is likely overwhelming – but I suspect there is a lot of interesting information that people would love to examine.

Update: For those of you (like me) who wondered what Montgomery Ward had to do with Olympic Tickets – take a look at Tickets For The ’76 Olympics Go On Sale Shortly At Montgomery Ward over in the Sports Illustrated online SI Vault. Sports Illustrated’s Vault is definitely another interesting source of information about the Olympic Games. If my post above has made you nostalgic for Olympics gone by – definitely take a look at the current Summer Games feature on their front page. I couldn’t figure out a permanent link to this feature, but if I ever do I will update this post later.

Read more →

THATCamp 2008: Text Mining and the Persian Carpet Effect

5 Comments

alarch: Drift of Harrachov mine (Flickr)I attended a THATCamp session on Text Mining. There were between 15 and 20 people in attendance. I have done my best to attribute ideas to their originators wherever possible – but please forgive the fact that I did not catch the names of everyone who was part of this session.

What Is Text Mining?

Text mining is an umbrella phrase that covers many different techniques and types of tools.

The CHNM NEH-funded text mining initiative defined text mining as needing to support these three research functions:

  • Locating or finding: improving on search
  • Extraction: once you find a set of interesting documents, how do you extract information in new (and hopefully faster) ways? How do you pull data from unstructured bulk into structured sets?
  • Analysis: support analyzing the data, discovery of patterns, answering questions

The group discussed that there were both macro and micro aspects to text mining. Sometimes you are trying to explore a collection. Sometimes you are trying to examine a single document in great detail. Still other situations call for using text mining to generate automated classification of content using established vocabularies. Different kinds of tools will be important during different phases of research.

Projects, Tools, Examples & Cool Ideas

Andrea Eastman-Mullins, from Alexander Street Press, mentioned the University of Chicago’s ARTFL Project and these two tools:

  • PhiloLogic: An XML/SGML based full-text search, retrieval and analysis tool
  • PhiloMine: a extension being developed for PhiloLogic to provide support for “a variety of machine learning, text mining, and document clustering tasks”.

Dan Cohen directed us to his post about Mapping What Americans Did on September 11 and to Twistori which text mines Twitter.

Other Projects & Examples:

Some neat ideas that were mentioned for ways text mining could be used (lots of other great ideas were discussed – these are the two that made it into my notes):

  • Train a tool with collections of content from individual time periods, then use the tool to assist in identification of originating time period for new documents. Also could use this same setup to identify shifts in patterns in text by comparing large data sets from specific date ranges
  • If you have a tool that has learned how to classify certain types of content well… then watch for when it breaks – this can give you interesting trails to things to investigate.

Barriers to Text Mining

All of the following were touched upon as being barriers or challenges to text mining:

  • access to raw text in gated collections (ie, collections which require payment to permit access to resources) such as JSTOR and Project MUSE and others.
  • tools that are too difficult for non-programmers to use
  • questions relating to the validity of text mining as a technique for drawing legitimate conclusions

Next Steps

These ideas were ones put forward as important to move forward the field of text mining in the humanities:

  • develop and share best practices for use when cultural heritage institutions make digitization and transcription deals with corporate entities
  • create frameworks that enable individuals to reproduce the work of others and provide transparency into the assumptions behind the research
  • create tools and techniques that smooth the path from digitization to transcription
  • develop focused, easy-to-use tools that bridge the gap between computer programmers and humanities researchers

My thoughts
During the session I drew a parallel between the information one can glean in the field of archeology from the air that cannot be realized on the ground. I discovered it has a name:

“Archaeologists call it the Persian carpet effect. Imagine you’re a mouse running across an elaborately decorated rug. The ground would merely be a blur of shapes and colors. You could spend your life going back and forth, studying an inch at a time, and never see the patterns. Like a mouse on a carpet, an archaeologist painstakingly excavating a site might easily miss the whole for the parts.” from Airborne Archaeology, Smithsonian magazine, December 2005 (emphasis mine)

While I don’t see any coffee table books in the near future of text mining (such as The Past from Above: Aerial Photographs of Archaeological Sites), I do think that this idea captures the promise that we have before us in the form of the text mining tools. Everyone in our session seemed to agree that these tools will empower people to do things that no individual could have done in a lifetime by hand. The digital world is producing terabytes of text. We will need text mining tools just to find our way in this blizzard of content. It is all well and good to know that each snowflake is unique – but tell that to the 21st century historian soon to be buried under the weight of blogs, tweets, wikis and all other manner of web content.

Image credit: Drift of Harrachov Mine by alarch via flickr

As is the case with all my session summaries from THATCamp 2008, please accept my apologies in advance for any cases in which I misquote, overly simplify or miss points altogether in the post above. These sessions move fast and my main goal is to capture the core of the ideas presented and exchanged. Feel free to contact me about corrections to my summary either via comments on this post or via my contact form.

Read more →

Of Pirates, Treasure Chests and Keys: Improving Access to Digitized Materials

1 Comment

Key to Anything by Stoker Studios (flickr)Dan Cohen posted yesterday about what he calls The Pirate Problem. Basically the Pirate Problem can be summed up as “there are ways of acting and thinking that we can’t understand or anticipate.” Why is that a ‘Pirate Problem’? Because a pirate pub opened near his home and rather than folding shortly thereafter due to lack of interest from the ‘very serious professionals’ who populate DC suburbs – the pub was a rousing success due to the pirate aficionados who came out of the woodwork to sing sea shanties and drink grog. This surprising turn of events highlighted for him the fact that there are many ways of acting and thinking (some people even know all the words to sea shanties without needing sheet music).

Dan recently delivered the keynote speech at a workshop at the University of North Carolina at Chapel Hill. The workshop brought together dozens of historians to talk about how the 16 million archival documents of the Southern Historical Collection (SHC) should be put online. He devoted his keynote “to prodding the attendees into recognizing that the future of archives and research might not be like the past” and goes on in his post to explain:

The most memorable response from the audience was from an award-winning historian I know from my graduate school years, who said that during my talk she felt like “a crab being lowered into the warm water of the pot.” Behind the humor was the difficult fact that I was saying that her way of approaching an archive and understanding the past was about to be replaced by techniques that were new, unknown, and slightly scary.

This resistance to thinking in new ways about digital archives and research was reflected in the pre-workshop survey of historians. Extremely tellingly, the historians surveyed wanted the online version of the SHC to be simply a digital reproduction of the physical SHC.

Much of the stress of Dan’s article is on fear of new techniques of analysis. The choppy waters of text mining and pattern recognition threaten to wash away traditional methods of actually reading individual pages and “most historians just want to do their research they way they’ve always done it, by taking one letter out of the box at a time”.

I certainly like the idea of new technologically based ways of analyzing large sets of cultural heritage materials, but I also believe that reading individual letters will always be important. The trick is finding the right letter!

And of course – we still need the context. It isn’t as if when we digitize major collections like the SHC that we are going to scan and OCR each page without regard to which box it came out of. We can’t slice and dice archival records and manuscripts into their component parts to feed into text analysis with no way back to the originals.

I like to imagine the combination of all the new technology (be it digitization, cross collection searching, text mining or pattern recognition) as creating keys to different treasure chests. Humanities scholars are treasure hunters. Some will find their gems through careful reading of individual passages. Others will discover patterns spread across materials now co-existing virtually that before digitization would have been widely separated by space and time. Both methods will benefit from the digitization of materials and the creation of innovative search and text analysis tools. Both still require an understanding of a material’s origin. The importance of context isn’t going anywhere – we still need to know which box the letter came from (and in a perfect world, which page came before and which came after). I want scholars to still be able to read one page from the box – I just want them to be able to do it from home in the middle of the night if they are so inclined with their travel budget no worse for wear.

Dan ties his post together by pointing out that:

… in Chapel Hill I was the pirate with the strange garb and ways of behaving, and this is a good lesson for all boosters of digital methods within the humanities. We need to recognize that the digital humanities represent a scary, rule-breaking, swashbuckling movement for many historians and other scholars.

In my opinion, the core message should be that we just found more locked treasure chests – and for those who are interested, we have some new keys that just might open those locks. I enjoyed the Pirate metaphor (obviously) and I appreciate that there are real issues here relating to strong discomfort with the fast changing landscape of technology, but I have to believe that if we do something that prevents historians from being able to read one letter at a time we are abandoning the treasure chests that are already open for the new ones for which we haven’t yet found the right keys. I am greedy. I want all the treasure!

Image credit: key to anything by Stoker Studios via flickr

Read more →

Pondering Structured Data About Archives: Archives Wiki, Freebase and OCLC’s World Map & WikiD

6 Comments

This week both Dan Cohen’s blog and ArchivesNext posted about the new Archives Wiki sponsored by the American Historical Association (AHA). The AHA blog summarizes the goals of this wiki as:

…we hope that by harnessing this (relatively) new technology for collaboration on the web, we can draw on the collective interests of thousands of researchers and archivists to develop a rich resource for anyone venturing into new archives for the first time.

The AHA post goes on to express the hope that the wiki “will provide a deeper level of information than the rather general information on most archival web sites”. Setting aside the question of if this wiki will reach critical mass with regard to contributions, the idea of collecting lots of information about archives and their collections got me thinking again about Freebase.com.

My earlier post, Metadata World Building: Freebase.com and OpenLibrary.org, considers the potential of using Freebase to build a set of structured data about archival institutions. I believe in the spirit behind the Archives Wiki, but I wish that the rich set of information that is going to be captured was being stuck into multiple attributes rather than free-form wiki text. I know that they have contributor guidelines , but that isn’t enough for me.

Why Structured Data Is So Cool

Why am I so hung up about structured data? This is the sort of thing that needs a good example – and thanks to Level 1 Librarian‘s post OCLC maps the world, I found my way to the amazing OCLC WorldMap project. The WorldMap itself is a Flash based application that lets you explore data about both WorldCat holdings and other related statistics for countries around the world.

Once you get into the application, click on any two countries (I chose Russia and Australia) and then click on the ‘Compare’ button. For those especially interested in Archives, click on the ‘Cultural Institutions’ button (3rd from the bottom). If you move your mouse over each of the bars on the bar graphs you can see the actual numbers driving them. For example, on my Cultural Institutions comparison chart for Russia vs Australia I can see that Russia has 112 Archives while Australia has 42. The data source for both of these numbers is listed as the International Directory of Archives. To see the sources for the data, click on one of the tabs labeled with a country name and then click on any number to see it’s source. If my instructions are lacking, take a look at the beautiful and thorough Key to the WorldMap.

If people are going to go to all this effort entering data about archives and their collections, I wish it were being collected in such a way that we could then build new and more fabulous tools for accessing, manipulating and exploring the information.

As an example – if we collected the hours of each archives in a structured way we could figure out how many hours a week the Illinois State Archives is open (Monday–Friday, 8:00 a.m.–4:30 p.m.; Saturday, 8:00 a.m.–3:30 p.m. = 50 hours) and contrast that with the weekly hours of the Missouri State Archives (8 a.m. to 5 p.m. Monday through Wednesday and Friday; 8 a.m. to 9 p.m. on Thursday; and 8:30 a.m. to 3:30 p.m. on Saturday = 56 hours) . We could figure out how many state archives have evening hours or weekend hours. How about a map that showed the historian visiting a new city which archives were open late on the one night he has off from his conference? This is just a tiny example – but I hope it lights a spark for people about the promise of collecting this simple kind of data in a structured way.

Freebase’s whole point is to build data-sets that can drive interesting applications – like WorldMap. This just makes me want to race back to Freebase and figure out how to capture what I wish were being captured by the ArchivesWiki within the confines of Freebase’s model.

OCLC’s WikiD

I was just about to end this post when I realized I ought to check to see if someone has already tackled this problem of adding structured data to a wiki. I found my way to OCLC’s WikiD (Wiki/Data) project. The project’s home page states: “WikiD (Wiki/Data) extends the Wiki model to support multiple WikiCollections containing arbitrary schemas of XML records with minimal additional complexity.” From a brief look around, I am not clear how this would integrate in with the more traditional wiki style of the Archives Wiki – nor am I convinced that this project is still moving forward (the most recent dates I see on presentations are from 2006) – but the idea of a wiki that includes structured data is definitely there. Anyone out there have any more information about WikiD or any other tool that supports wiki style ease with the ability to structured data?

Final Thoughts

Again, I love the vision inherent in the Archives Wiki. I know that even getting a project like this off the ground is a big deal. I found this old AHA blog post from October of 2006 that discusses the original proposal for it and why it should be done. All the reasons are sound. But (you knew there was a but) the database geek in me just goes nuts when I see structured data being typed in free-form.

Read more →

Capa’s Found Images and Thoughts on Digital Photographers’ Sketchbooks

2 Comments

Robert Capa by Gerda Taro (May 1937)In the Washington Post article There Are No Black-and-White Answers in War — Then Lost Negatives Turn Up (February 1, 2008), we learn that three cardboard boxes of negatives were recently delivered to the International Center of Photography (ICP) – possibly including as many as 4,000 images. This collection of black-and-white film, consisting predominately of photos shot by Robert Capa during the Spanish Civil War, was long thought lost during World War II and will join the already existing Robert Capa Archives. The boxes also contain negatives from two other famed photographers associated with Capa, Gerda Taro and David Seymour (known by the pseudonym Chim – pronounced shim).

There are many reasons these boxes are exciting for historians and Capa researchers. They hold the promise of answering some long standing questions. Where certain famous photos were staged? Are the current credits given for various photos are correct? But what caught my eye in this article was the following quote from ICP curator Brian Wallis:

“Capa was really adept at creating a whole story in one day: Here are the characters, here is the beginning, the action shots, the end, and the effect on civilians. If you look at his work not as great individual shots, but as stories, you get a completely different picture of him, and I think a more accurate and valuable picture.

“These negatives will further amplify that story, not just a few stories but dozens of stories that went out. It is like a sketchbook — he was trying out various ideas, and some worked and some didn’t.”

What About Digital Photographer’s Sketchbooks?

If you have ever used a digital camera, you have almost certainly enjoyed the instant gratification of being able to preview your photos on the tiny screen. The next temptation is to click the delete button. Sometimes you delete because the photo is clearly not what you were after – other times you delete to make room for some much more crucial photo.

I don’t know what the standard best practices are for professional photographers. Part of me hopes that they keep everything – at least until they can view the images on a big screen. But there is clearly a much easier path to deleting the ideas that didn’t work out. It leaves me wondering what the scholars of the future will be missing by not being able to see the failed experiments. The ‘sketchbooks’ of digital photographers could easily be perceived as at risk records. That said, many creative individuals (artists, architects, photographers… etc) do not care to share their failed experiments with the outside world. One of the issues facing those preserving digital records of the design community is the strong desire of designers to not share their work in progress and only share the final product (see my post SAA2007: Preserving Born Digital Records of the Design Community (Session 106) for more thoughts on this).

Image Overload

Of course there will be the photographers who do keep everything. Hard drive space is getting cheaper with every passing day. Perhaps my fears are misplaced and instead we should be worrying more about the flood of photographs that will overwhelm archivists and researchers. The time needed to discover the ‘good’ and ‘important’ photographs in a collection of thousands of images could be extreme.

I shoot all my photos digitally now. I no longer live in a world where there are only 36 shots on a single role – I don’t need to choose each photo carefully. I cheerfully tell my friends “Photos are free!”. Even that doesn’t stop me from deleting the ones that I really dislike. Sometimes the 2 GB card in my camera gets full before an event is done, so the on the spot weeding of photos occurs as well. But when I compare the number of ‘good’ photos that I have uploaded to share online (currently 5,000+ and counting) with the number of photos I have on my hard drive (20,000+) it is clear to me that I am keeping plenty of ‘sketch photos’. It is also interesting to note that I will often realize that there are photos I really like now that I didn’t appreciate immediately after they were taken. While something at the time made me NOT include it as a photo to share, now I see something in the image that catches my eye in a new way. The more this happens, the less I delete as I download, organize and tag my photos.

Metadata and the Exchangeable Image File Format (EXIF)

Of course the situation with digital photographs is not all bad. When digital cameras record a photo, they also record a set of metadata in the exchangeable image file format (EXIF) format. The metadata recorded usually includes camera make and model, date, time, and camera settings. Some cameras can even record GPS generated location information. Because there is no way to know the time zone (at least without location information), the value of the time setting is more useful for relating photos from within a set to one another than in establishing the actual time a photo was taken.

Adobe has contributed their own proprietary metadata format called Extensible Metadata Platform (XMP).

The most common metadata tags recorded in XMP data are those from the Dublin Core Metadata Initiative, which include things like title, description, creator, and so on. The standard is designed to be extensible, allowing users to add their own custom types of metadata into the XMP data. (Wikipedia Entry: Extensible Metadata Platform )

The magic of both XMP and EXIF is that the metadata is embedded in the file itself. There is no chance of losing the connection between a photo and the information about it – it is akin to writing on the back of an analog photograph. Embedded metadata provides the greatest tool for rediscovering the original order in which a series of photographs were taken, as well as providing access to metadata entered by the photographer at the image level.

The archivist of today accessioning born digital images must be comfortable with tools for viewing and updating embedded metadata. I mention updating because any information that is currently known about an image that could be added to the embedded metadata is more information that cannot later become accidentally separated from the images in question. This of course assumes that we will still have the proper technology in the future with which to access all this embedded metata.

Embedded metadata can be updated before it reaches the controlled environment of an archive. Data found as embedded metadata must be evaluated in the same manner that any information about photographs would be evaluated. For example, it would be a lot easier to modify metadata on digital photos to make the images appear to have been taken in a different order than it would be to do the same change with a strip of analog negatives. If this in fact was done – the fact that it was would likely be as interesting to researchers as the original order (assuming of course that you could ever figure out that such a modification had been made!).

Not all methods of organizing photos results in embedded metadata, so there is plenty of room for the standard challenges of old software that you can’t get to run but that holds the key to information about a hard drive of thousands of images. Sophisticated photograph management tools often now include workflow features that could also provide insight into the decision making and processing steps taken by a photographer. Much of this type of information is very unlikely to be embedded in the photos themselves – but still would represent interesting digital records related to the everyday work that a professional photographer performs.

Final Thoughts

I do feel that something is being lost via the ease with which one may delete experimental ‘sketchbook’ photos, but I suspect that the lure of virtually infinite hard drive space, image organization/tagging software tools and the clues provided by embedded metadata will balance the scales. Those who study photographers and their work will certainly have more to say about far in the future. There will be hard choices over the next decades – what can we do to guarantee access in that distant time to the full digital bodies of work of the Capas of today? I think the answers start with building strong lines of communication between prominent digital photographers and archivists. I know that this is just a special case of the challenges we see with digital records across professions, but each field adds its own special issues that must be sorted through and figured out one at a time. So, are there archivists out there working with professional digital photographers?

For more images related to this story, see the New York Time’s slideshow about Robert Capa’s Lost Negatives [UPDATE: New images available in the slideshow Inside the Mexican suitcase, posted April 29, 2009]

Image credit: Photographer Robert Capa during the Spanish civil war, May 1937. Photo by Gerda Taro. If the logic on this Wikimedia Commons page is to believed, this photo is in the public domain in the United States because the photographer died in 1937 (ie, more than 70 years ago).

Read more →

Using WWI Draft Registration Cards for Research: NARA Records Provide Crucial Data

1 Comment

NARA: World War I photograph, 1918 (ARC Identifier: 285374)

In the HealthDay article Having Lots of Kids Helps Dads Live to 100, a recent study was described that examined what increased the chances of a man living past 100.

A young, trim farmer with four or more children: According to a new study, that’s the ideal profile for American men hoping to reach 100 years of age. The research, based largely on data from World War I draft cards, suggests that keeping off excess weight in youth, farming and fathering a large number of offspring all help men live past a century.

The article mentions that this research was “spurred by the fact that a treasure trove of information about 20th-century American males has now been put online”. The study was based out of the University of Chicago’s Center on Aging. The paper, New Findings on Human Longevity Predictors, includes the following reference:

Banks, R. (2000). World War I Civilian Draft Registrations. [database on-line]. Provo, UT, Ancestry.com.

With an account on Ancestry.com, you too could examine the online database of World War I Draft Registration Cards. This Ancestry.com page notes the source of the original data as:

United States, Selective Service System. World War I Selective Service System Draft Registration Cards, 1917-1918. Washington, D.C.: National Archives and Records Administration. M1509, 4,582 rolls

NARA’s page for the World War I Selective Service System Draft Registration Cards, M1509 includes similar background information to what can be found on the Ancestry.com page, but of course – no access to the actual records.

It is frustrating to a study based on archival records that is making the news, but that does not make it clear to the reader that archival records were the source for the research. As I discussed at length in my post Epidemiological Research and Archival Records: Source of Records Used for Research Fails to Make the News, I feel that it is very important to take every opportunity to help the general public understand how archival records are supporting research that impacts our understanding of the world around us. I appreciate that partnering with 3rd parties to get government records digitized is often the only option – but I want people to be clear about why those records still exist in the first place.

Photo Credit: US. National Archives, World War I Photographs, 1918. Army photographs. Battle of St. Mihiel-American Engineers returning from the front; tank going over the top; group photo of the 129th Machine gun Battalion, 35th Division before leaving for the front; views of headquarters of the 89th Division next to destroyed bridge; Company E, 314th Engineers, 89th Division, and making rolling barbed wire entanglements. NAIL Control Number: NRE-75-HAS(PHO)-65

Read more →

SAA2007: Archives and E-Commerce, Three Case Studies (Session 404)

4 Comments

George Washington US DollarDiane Kaplan, of Yale University Library’s Manuscripts and Archives unit, started off Session 404 (officially titled Exploring the Headwaters of the Revenue Stream) by thanking everyone for showing up for the last session of the day. This was a one hour session that examined ways to generate new funds through e-commerce . Three different e-commerce case studies were presented, followed by a short question and answer period.

University of Wyoming’s American Heritage Center

Mark Shelstad‘s presentation, “Show Me the Money: Or: How Do We Pay for This?”, detailed the approach taken by the University of Wyoming‘s American Heritage Center (AHC) to find alternate revenue streams. After completing a digitization project in the fall of 2004, the AHC had to figure out how to continue their project after their original grant money ran out.

Since they didn’t have a lot of in-house resources, they chose Zazzle.com for their effort to profit from their existing high resolution images. They can earn up to 17% from the sales through a combination of affiliate sales and profits from the sale of products featuring American Heritage Center images.

They had a lot of good reasons for choosing Zazzle.com. Zazzle.com already had an existing ‘special collections’ area, meaning that their images would have a better chance of being found by those interested in their offerings (for example – take a look at the Library of Congress Vintage Photos store). Zazzle.com also did not require an exclusive license to the images. The American Heritage Center Zazzle on-line store opened in 2005.

Currently they are making about $30 a month in royalties from 200 images. Mark pointed out that everyone needs to keep in mind that the major photo provider, Corbis, has yet to turn a profit in online photo sales. He also mentioned a website called Cogteeth.com that lets you click on any image and use those images on t-shirts, mugs.. etc.

Near the end of his talk, Mark shared an amazing idea to create a non-profit that would be a joint organization for featuring and selling products using archival images. I love it! It is easy to see that many archives are small and don’t have the infrastructure to create and run their own e-commerce websites. At the same time, general sites that let anyone set up a store to sell items with custom images on them threaten to loose the special nature of historical images in the shuffle. Even the special collections section of Zazzle lumps the American Heritage Center and the Library of Congress collections with Disney and Star Wars. I would love to see this idea grow!

Minnesota Historical Society

Kathryn Otto of the Minnesota Historical Society (MHS) spoke next. She first gave an overview of traditional services provided by MHS for a fee, such as photocopies, reader-printer copies, microfilm sales, media sales, inter-library loan fees, classes and photograph sales. MHS also earned income via standard use fees and research services.

The first e-commerce initiative at MHS was the sale of Minnesota State Death Certificates from 1904 – 2001. Made available via the Minnesota Death Certificate Index they provide the same data as Ancestry.com, but the MHS index provides a better search interface. They have had users tell them that they couldn’t find something on Ancestry.com – but that they were able to find what they needed on the MHS site.

To their existing Visual Resources Database, MHS also added a buy button for most images. Extra steps were added into the standard buy process to deal with the addition of a use fee depending on how the purchaser claims the image will ultimately be used. One approach that did not work for them was to offer expensively printed pre-selected images. The historical society sells classes online and can handle member vs non-member rates. TheVeterans Graves Registration Index is a tiny database that was created by reusing the interface used for the death certificates.

The Birth Certificate Index provides “single, non-certified copies of individual birth certificates reproduced from the originals” via the website.. while “[o]fficial, certified copies of these birth certificates are available through the Minnesota Department of Health.” The MHS site provides much faster and easier service than the Department of Health as can be seen from this page detailing how to order a non-certified copy of a birth record from the DOH – which requires printing, filling out and either faxing or snail mailing a form.

Features to keep in mind as you branch into in e-commerce:

  • Statistics – Consider the types of statistics you want. Their system just gave them info about orders – not how much they made.
  • Sales tax – Figure out how is it handled
  • Postage/Handling fees – Look at the details! The MHS Library-Archives was stuck with the Museum Store’s postage rates because the e-commerce system could not handle different fees for different types of objects.
  • Can’t afford credit card fees? Consider PayPal.
  • Advertise what you are selling on your own website.

Godfrey Memorial Library, Middletown, CT

The final panelist was Richard Black, Director of the Godfrey Memorial Library in Middletown, Connecticut. The Godfrey is a small, non-profit, genealogical research library with approximately 120,000 genealogical items. They currently have 5 full time staff and 60 volunteers.

Services they provide:

About 3 years ago they had exhausted all of their endowment money and faced the strong possibility of closing the doors. They were down to one full time librarian and a few volunteers and were dependent mostly on donations and some minor income from other sources/services.

They had only a few options open to them:

  • find more money from other sources
  • merge with another library
  • close the doors
  • sell some of the content
  • others??

The first approach to raise funds was to create a subscription website. The Godfrey acquired Heritage Quest census records and added other databases as resources allowed. Subscriptions were sold for $35 a year. The board thought they might be lucky to get 100 subscriptions.. but they actually got approximately 14,000!

Now the portal provides access to sites for which a premium has been paid (so that subscribers don’t have to pay), sites that are available free on the Internet (but made easier to find) and sites unique to Godfrey, including digitized material in the library and other material that has been made available to them. They just added 95,000 Jewish grave-sites – brought to them by a local rabbi. Another recent addition was a set of transcriptions of a grave-site made as an Eagle Scout project. They also negotiated to have their books digitized for them for free. The company performing the digitization will pay a royalty to Godfrey as the books are used.

The costs to acquire data for the portal includes $60,000 a year for access to premium sites, the cost to digitize and transcribe unique content (there are opportunities to partner and reduce costs) and the cost to acquire patrons. The efforts of the Godfrey staff and volunteers is ‘free’ – but costs time.

The Godfrey subsequently lost access to the Heritage Quest material. This was like taking the anchor store out of the corner of a mall. It forced them to diversify their revenue streams and watch for new opportunities.

Current revenue source distribution:

  • online portal 45%
  • annual appeal 10%
  • patron requests 5%
  • contract services 35% (OCLC analytical cataloging that they do)
  • misc 5%

The endowment funds have been restored and the Godfrey’s staff is now growing again.

Questions

Question: Did you meet resistance in your institutions?
Answer: No.. Minnesota said they had such success that the 2 questions they here now are A) What do we put online next? B) How long can they protect their income from the rest of the institution?

Question: (From someone from a NJ archives) Is there a way to do e-commerce with government records and not have the money ‘stolen’ from them?
Answer: Minnesota – The department of health was happy for death and birth certificates business to go away? They do worry about the future when they might try to make a marriage index – because that territory is already ‘owned’ by a group that wants to keep that income.

Question: When you charge for use fees – are there people who don’t pay them?
Answer: Minnesota: Probably – no way to really know.
Mark (American Heritage Center): Our images are public domain – they can do what they like with them.

Question: Do you brand your images?
Answer: Mark: Yes.. a logo and URL goes with the images.

My Thoughts

I was particularly impressed by how much information was conveyed in the course of the 1 hour session. My personal highlights were:

  • As I mentioned above, I want Mark’s idea for a non-profit to sell co-located products based on archival images to gain support and momentum.
  • I was pleased by the point that the MHS makes money from their Minnesota Death Certificate Index partly due to their improved and powerful search interface. The data is available elsewhere – but they made it easier to find information, so they will become the destination of choice for that information.
  • The Godfrey’s story is inspirational. In an age when we hear more and more often about archives and libraries being forced to cut back services due to funding shortfalls, it is great to hear about a small archives that pulled themselves back from the brink of disaster by brave experimentation.

These three case studies gave a great glimpse of some of the ways that archives can get on the e-commerce bandwagon. There is no magic here – just the willingness to dig in, figure out what can be done and try it. That said – there is definitely lots of room to learn from others successes and mistakes. The more real world success and failure stories archives share with the archival community about how to ‘do’ e-commerce, the easier it will be for each subsequent project to be a success.

As is the case with all my session summaries from SAA2007, please accept my apologies in advance for any cases in which I misquote, overly simplify or miss points altogether in the post above. These sessions move fast and my main goal is to capture the core of the ideas presented and exchanged. Feel free to contact me about corrections to my summary either via comments on this post or via my contact form.

Read more →

Controversial Photos, Archvists’ Choices and Journalism

New York Times Magazine Cover: January 1995The New York Times Magazine published The Great Ivy League Nude Posture Photo Scandal in January of 1995. Still available online, it is a fascinating tale that took reporter Ron Rosenbaum on a wild hunt through multiple archives in a quest for long lost photographs. I spotted a link to the article in a post on Boing Boing – and once I started reading it I couldn’t stop.

The story includes thorough coverage of the research (and the footwork and the paperwork) it took to find the final resting place of some very controversial photographs. Taken as part of the orientation process of new students at Ivy League and Seven Sisters school campuses predominately during the 1940’s, 50’s and 60’s, these photos were theoretically taken to screen for students who needed remedial posture classes. William Herbert Sheldon was a driving force behind many of the photos. Best known for assigning people into three categories of body types in the 1940s, Sheldon based his categories of endomorphic, mesomorphic, and ectomorphic on measurements done using the student photographs. Rosenbaum’s quest was to find the real story behind the photos and to discover if any of the photos survived the purging fires at that occurred at many of the schools involved.

His first stop was Harvard’s archives:

Harley P. Holden, curator of Harvard’s archives, said that from the 1880’s to the 1940’s the university had its own posture-photo program in which some 3,500 pictures of its students were taken. Most were destroyed 15 or 20 years ago “for privacy scruples,” Holden said. Nonetheless, quite a few Harvard nudes can be found illustrating Sheldon’s book on body types, the Atlas of Men. Radcliffe took posture photos from 1931 to 1961; the curator there said that most of them had been destroyed (although some might be missing) and that none were taken by Sheldon.

A major turning point in Sheldon’s project came in 1950. He went to the University of Washington to further his plans to make an Altas of Women. The families of a few photographed females students at the university questioned the real purpose of the photographs. The resulting upheaval culminated in the destruction of many photographs. A Time article dated September 25, 1950, Revolt at Washington, documents the events in Washington and notes that over 800 photos were burned.

Rosenbaum’s article goes on mention that thousands of photos were subsequently burned at Harvard, Vassar and Yale in the 60’s and 70’s – but he continued to hunt for the ones that some believed had escaped into Sheldon’s private archives. A chain of contacts led Rosenbaum to Sheldon’s former associate Roland D. Elderkin. An elderly gentleman of 84 at the time of the story’s publication, Elderkin spent years assisting Sheldon. He took many of the photographs. And after being turned down by many archives, he found Sheldon’s records, photos and negatives a home in the National Anthropological Archives.

In 1987, the curators of the National Anthropological Archives acquired the remains of Sheldon’s life work, which were gathering dust in “dead storage” in a Goodwill warehouse in Boston. While there were solid archival reasons for making the acquisition, the curators are clearly aware that they harbor some potentially explosive material in their storage rooms. And they did not make it easy for me to gain access.

On my first visit, I was informed by a good-natured but wary supervisor that the restrictive grant of Sheldon’s materials by his estate would permit me to review only the written materials in the Sheldon archives. The actual photographs, he said, were off-limits. To see them, I would have to petition the chief of archivists. Determined to pursue the matter to the bitter end, I began the process of applying for permission.

In their online guide to collections I found the entry for SHELDON, WILLIAM HERBERT (1898-1977), Papers. It notes that the collection is 150 linear feet. It also includes a line that reads “RESTRICTION: The photographic material is not available for research.”

While Rosenbaum’s hunt was for the photographs, some of his most interesting discoveries came from the papers themselves. During his three month wait for permission to view the photos, he reviewed boxes of letters and notes. See Rosenbaum’s article for details – but it was Sheldon’s own words in those papers that revealed he held racist views and that he seemed more concerned with his research than with the psychological impact of his research on the girls whose photos he arranged to take.

When finally Rosenbaum was given the opportunity to review some 20,000 negatives of the photos (no photos and no names) we read:

A curator trundled in a library cart from the storage facility. Teetering on top of the cart were stacks of big, gray cardboard boxes. The curator handed me a pair of the white cotton gloves that researchers must use to handle archival material.

I love it – gray cardboard boxes and white cotton gloves. He even mentions the finding aids and gives examples of how the groups of photos are described. I also appreciate the earlier acknowledgment of the “solid archival reasons for making the acquisition”.

Rosenbaum looked through a lot of the negatives, mostly to verify that what the finding aids claimed were present were in fact in those gray boxes. He was struck by the contrast between the expressions on the mens’ and womens’ faces.

For the most part, the men looked diffident, oblivious. That’s not surprising considering that men of that era were accustomed to undressing for draft physicals and athletic-squad weigh-ins. But the faces of the women were another story. I was surprised at how many looked deeply unhappy, as if pained at being subjected to this procedure. On the faces of quite a few I saw what looked like grimaces, reflecting pronounced discomfort, perhaps even anger. I was not much more comfortable myself sitting there in the midst of stacks of boxes of such images. There I was at the end of my quest. I’d tracked down the fabled photographs, but the lessons of the posture-photo ritual were elusive.

He found the missing photos – but no easy answers. This is a great combination of a compelling story and a realistic representation of archives and archivists. The records don’t always hold the answers to the question you thought you were asking – but sometimes they hold secrets you hadn’t expected.

So many elements tie back to the choices made by individual archivists – sometimes made in the heat of the moment or under great community pressure. I think this story is a particularly poignant example of the downstream effects of these sorts of hard choices. It isn’t often that we can see cause and effect this clearly.

What would you have done? Would you have burned the photos or stored them away? Would you have stepped forward to take Sheldon’s records? If something like this happened today – what do you think the future of these photos might be?

Read more →

International Environmental Data Rescue Organization: Rescuing At Risk Weather Records Around the World

2 Comments

iedro.jpgIn the middle of my crazy spring semester a few months back, I got a message about volunteer opportunities at the International Environmental Data Rescue Organization (IEDRO). I get emails from from VolunteerMatch.org every so often because I am always curious about virtual volunteer projects (ie, ways you can volunteer via your computer while in your pajamas). I filed the message away for when I actually had more time to take a closer look and it has finally made it to the top of my list.

A non-profit organization, IEDRO states their vision as being “.. to find, rescue, and digitize all historical environmental data and to make those data available to the world community.” They go on to explain on their website:

Old weather records are indeed worth the paper they are written on…actually tens of thousands times that value. These historic data are of critical importance to the countries within which they were taken, and to the world community as well. Yet, millions of these old records have already perished with the valuable information contained within, lost forever. These unique records, some dating back to the 1500s, now reside on paper at great risk from mold, mildew, fire, vermin, and old age (paper and ink deteriorate) or being tossed away because of lack of storage space. Once these data are lost, they are lost forever. There are no back up sources; nothing in reserve.

Why are these weather records valuable? IEDRO gives lots of great examples. Old weather records can:

  • inform the construction and engineering community about maximum winds recorded, temperature extremes, rainfall and floods
  • let farmers know the true frequency of drought, flood, extreme temperatures and in some areas, the amount of sunshine enabling them to better plan crop varieties and irrigation or drainage systems increasing their food production and helping to alleviate hunger.
  • assist in explaining historical events such as plague and famine, movement of cultures, insect movements (i.e. locusts in Africa), and are used in epidemiological studies.
  • provide our global climate computer models with baseline information enabling them to better predict seasonal extremes. This provides more accurate real-time forecasts and warnings and a better understanding of global change and validation of global warming.

The IEDRO site includes excellent scenarios in which accurate historical weather data can help save lives. You can read about the subsistence farmer who doesn’t understand the frequency of droughts well enough to make good choices about the kind of rice he plants, the way that weather impacts the vectorization models of diseases such as malaria and about the computer programs that need historical weather data to accurately predict floods. I also found this Global Hazards and Extremes page on the NCDC’s site – and I wonder what sorts of maps they could make about the weather one or two hundred years ago if all the historical climate data records were already available.

There was additional information available on IEDRO’s VolunteerMatch page. Another activity they list for their organization is: “Negotiating with foreign national meteorological services for IEDRO access to their original observations or microfilm/microfiche or magnetic copies of those observations and gaining their unrestricted permission to make copies of those data”.

IEDRO is making it their business to coordinate efforts in multiple countries to find and take digital photos of at risk weather records. They include information on their website about their data rescue process. I love their advice about being tenacious and creative when considering where these weather records might be found. Don’t only look at the national meteorological services! Consider airports, military sites, museums, private homes and church archives. The most unusual location logged so far was a monastery in Chile.

Once the records are located, each record is photographed with a digital camera. They have a special page showing examples of bad digital photos to help those taking the digital photos in the field, as well as a guidelines and procedures document available in PDF (and therefore easy to print and use as reference offline).

The digital images of the rescued records are then sent to NOAA’s National Climatic Data Center (NCDC) in Asheville, North Carolina. The NCDC is part of the National Environmental Satellite, Data and Information Service (NESDIS) which is in turn under the umbrella of the National Oceanic and Atmospheric Administration (NOAA). The NCDC’s website claims they have the “World’s Largest Archive of Climate Data”. The NCDC has people contracted to transcribe the data and ensure the preservation of the digital image copies. Finally, the data will be made available to the world.

IEDRO already lists these ten countries as locations where activities are underway: Kenya, Malawi, Mozambique, Niger, Senegal, Zambia, Chile, Uruguay, Dominican Republic and Nicaragua.

I am fascinated by this organization. On a personal level it brings together a lot of things I am interested in – archives, the environment, GIS data, temporal data and an interesting use of technology. This is such a great example of records that might seem unimportant – but turn out to be crucial to improving lives in the here and now. It shows the need for international cooperation, good technical training and being proactive. I know that a lot of archivists would consider this more of a scientific research mission (the goal here is to get that data for the purposes of research), but no matter what else these are – they are still archival records.

Read more →

ArchivesZ: Visualizing Archival Collections

8 Comments

Announcing ArchivesZ – a tool for visualizing archival collections. This prototype is the final project for my information visualization class. It is a web based tool designed to support exploration of aggregated data about archival collections – inspired by the availability of structured data in EAD encoded finding aids.

For visual thinkers who just want to see what this is about – take a look at our 5 minute video demonstration. I don’t yet have a version online for folks to play with – but that is in the works.

This is the official blurb we came up with to describe the project:

ArchivesZ is an information visualization tool designed to support search, understanding and exploration of of archival and manuscript collections. The tool addresses one of the major challenges facing those who work with archival records – the need to understand the scope and quantity of available records. Since archival collections are unique, vary dramatically in record quantity and are organized based on the records creators it can be a great challenge for users to gain perspective concerning the available records across multiple collections. ArchivesZ leverages a unique dual sided histogram to support exploration of the multiple subjects assigned to each collection. As subject terms are selected, the dual sided histogram chart is generated to display related subjects. The tool combines the dual sided histogram with a more traditional histogram displaying year data to permit tightly coupled, multi-dimensional browsing of subject and time period metadata. By representing the distribution of subjects and time periods using the metric of total aggregate linear feet of associated collections, ArchivesZ permits users to get a better sense of total available research materials than they would by viewing a standard search result list.

If you are curious about what a ‘dual-sided histogram’ actually is (or just want to read more about the process and ideas that led us to the current incarnation of ArchivesZ) take a look at our final paper about ArchivesZ.

There is a very long list of features I would like to add or improve but of course there is only so much you can do in the few weeks available for a project like this. Some of our ideas are detailed at the end of the paper I linked to above. I plan to continue working on ArchivesZ and I welcome all feedback – either as comments to this post or via email to jeanne AT spellboundblog.com.

Read more →