Menu Close

Category: search

WorldCat Identities: Visualizing Library Catalog Data

The folks over at OCLC have opened the doors on the beta version of WorldCat Identities a few weeks back. It provides a new way to look at the publications by and about specific authors and other ‘content creators’ such as musicians, playrights and movie producers.

The main page starts you off with the top 100. This is not merely a list of the 100 most prolific authors, rather creators have been ranked based on the combination of their own publications and other works about the creator. For example, Shakespeare lists 40,460 total works. A timeline displays the number of publications per 5 year period stretching from 1550 through to the present day. The only gap in publications occurred from 1556-1561 (how is that for a great trivia question?). The most prolific period of publications was actually from 2000-2005 showing 11,897 publications. We then get to see lists of the most widely held works about and by Shakespeare (Tales from Shakespeare and The complete works of William Shakespeare), an audience level, a list of related names and a couple of useful links (including a link to the Authority File and to Shakespeare’s Wikipedia entry). But wait – there is more! In the right sidebar we get to see a list of alternate names for Shakespeare. For example, the list includes 4 different versions of Shakespear’s name in Hebrew. We also get to see the list of Languages into which the works have been translated along with the number of works published in that language. Runner-up to English’s top value of 96,923 is German clocking in at 5,538.

Beyond the top 100 – you can search for any author you like. I love that the focus isn’t so much on book sales or ‘best sellers’ – but rather on library holdings and the aggregate of all the information our cataloging of books has assembled. The timeline visualization does a great job of giving you the big picture of when works (and those about the creator) were produced.

On my wishlist would be the ability to view the list of publications ranked in order by translations. It is no surprise to me that The complete works of William Shakespeare is the most widely held with “2,503 editions published between 1623 and 2007 in 25 languages and held by 47,606 libraries worldwide” – but it was tantelizing to see that Hamlet has been translated into 67 languages while MacBeth has only been translated into 53. What about learning which works have been repeatedly published over the longest span of years? Or seeing timelines by subject so you could look for changes in the type of works being published?

Overall, this is a great example of the direction I hope the archives community will move in permitting users to explore aggregations of structured data. By ‘structured data’ I mean those nice predictable units of information such as dates, locations, subjects and language. As more and more archivists furiously generate EAD finding aids full of these nice tidbits, we will have more and more data to explore and display in new and interesting ways.

Read more →

Understanding Born Digital Records: Journalists and Archivists with Parallel Challenges

3 Comments

My most recent Archival Access class had a great guest speaker from the Journalism department. Professor Ira Chinoy is currently teaching a course on Computer-Assisted Reporting. In the first half of the session, he spoke about ways that archival records can fuel and support reporting. He encouraged the class to brainstorm about what might make archival records newsworthy. How do old records that have been stashed away for so long become news? It took a bit of time, but we got into the swing of it and came up with a decent list. He then went through his own list and gave examples of published news stories that fit each of the scenarios.

In the second half of class he moved on to address issues related to the freedom of information and struggling to gain access to born digital public records. Journalists are usually early in the food chain of those vying for access to and understanding of federal, state and local databases. They have many hurdles. They must learn what databases are being kept and figure out which ones are worth pursuing. Professor Chinoy relayed a number of stories about the energy and perseverance required to convince government officials to give access to the data they have collected. The rules vary from state to state (see the Maryland Public Information Act as an example) and journalists often must quote chapter and verse to prove that officials are breaking the law if they do not hand over the information. There are officials who deny that the software they use will even permit extractions of the data – or that there is no way to edit the records to remove confidential information. Some journalists find themselves hunting down the vendors of proprietary software to find out how to perform the extract they need. They then go back to the officials with that information in the hopes of proving that it can be done. I love this article linked to in Prof. Chinoy’s syllabus: The Top 38 Excuses Government Agencies Give for Not Being Able to Fulfill Your Data Request (And Suggestions on What You Should Say or Do).

After all that work – just getting your hands on the magic file of data is not enough. The data is of no use without the decoder ring of documentation and context.

I spent most of the 1990s designing and building custom databases, many for federal government agencies. There are an almost inconceivable number of person hours that go into the creation of most of these systems. Stakeholders from all over the organization destined to use the system participate in meetings and design reviews. Huge design documents are created and frequently updated … and adjustments to the logic are often made even after the system goes live (to fix bugs or add enhancements). The systems I am describing are built using complex relational databases with hundreds of tables. It is uncommon for any one person to really understand everything in it – even if they are on the IT team for the full development life cycle.

Sometimes you get lucky and the project includes people with amazing technical writing skills, but usually those talented people are aimed at writing documentation for users of the system. Those documents may or may not explain the business processes and context related to the data. They will rarely expose the relationship between a user’s actions on a screen and the data as it is stored in the underlying tables. Some decisions are only documented in the application code itself and that is not likely to be preserved along with the data.

Teams charged with the support of these systems and their users often create their own documents and databases to explain certain confusing aspects of the system and to track bugs and their fixes. A good analogy here would be to the internal files that archivists often maintain about a collection – the notes that are not shared with the researchers but instead help the archivists who work with the collection remember such things as where frequently requested documents are or what restrictions must be applied to certain documents.

So where does that leave those who are playing detective to understand the records in these systems? Trying to figure out what the data in the tables mean based on the understanding of end-users can be a fool’s errand – and that is if you even have access to actual users of the system in the first place. I don’t think there is any easy answer given the realities of how many unique systems of managing data are being used throughout the public sector.

Archivists often find themselves struggling with the same problems. They have to fight to acquire and then understand the records being stored in databases. I suspect they have even less chance of interacting with actual users of the original system that created the records – though I recall discussions in my appraisal class last term about all the benefits of working with the producers of records long before they are earmarked to head to the archives. Unfortunately, it appeared that this was often the exception rather than the rule – even if it is the preferred scenario.

The overly ambitious and optimistic part had the idea that what ‘we’ really need is a database that lists common commercial off-the-shelf (COTS) packages used by public agencies – along with information on how to extract and redact data from these packages. For those agencies using custom systems, we could include any information on what company or contractors did the work – that sort of thing can only help later. Or how about just a list of which agencies use what software? Does something like this exist? The records of what technology is purchased are public record – right? Definitely an interesting idea (for when I have all that spare time I dream about). I wonder if I set up a wiki for people to populate with this information if people would share what they already know.

I would like to imagine a future world in which all this stuff is online and you can login and download any public record you like at any time. You can get a taste of where we are on the path to achieving this dream on the archives side of things by exploring a single series of electronic records published on the US National Archives site. For example, look at the search screen for World War II Army Enlistment Records. It includes links to sample data, record group info and an FAQ. Once you make it to viewing a record – every field includes a link to explain the value. But even this extensive detail would not be enough for someone to just pick up these records and understand them – you still need to understand about World War II and Army enlistment. You still need the context of the events and this is where the FAQ comes in. Look at the information they provide – and then take a moment to imagine what it would take for a journalist to recreate a similar level of detailed information for new database records being created in a public agency today (especially when those records are guarded by officials who are leery about permitting access to the records in the first place).

This isn’t a new problem that has appeared with born digital records. Archivists and journalists have always sought the context of the information with which they are working. The new challenge is in the added obstacles that a cryptic database system can add on top of the already existing challenges of decrypting the meaning of the records.

Archivists and Journalists care about a lot of the same issues related to born digital records. How do we acquire the records people will care about? How do we understand what they mean in the context of why and how they were created? How do we enable access to the information? Where do we get the resources, time and information to support important work like this?

It is interesting for me find a new angle from which to examine rapid software development. I have spent so much of my time creating software based on the needs of a specific user community. Usually those who are paying for the software get to call the shots on the features that will be included. Certain industries do have detailed regulations designed to promote access by external observers (I am thinking of applications related to medical/pharmaceutical research and perhaps HAZMAT data) but they are definitely exceptions.

Many people are worrying about how we will make sure that the medium upon which we record our born digital records remains viable. I know that others are pondering how to make sure we have software that can actually read the data such that it isn’t just mysterious 1s and 0s. What I am addressing here is another aspect of preservation – the preservation of context. I know this too is being worried about by others, but while I suspect we can eventually come up with best practices for the IT folks to follow to ensure we can still access the data itself – it will ultimately be up to the many individuals carrying on their daily business in offices around the world to ensure that we can understand the information in the records. I suppose that isn’t new either – just another reason for journalists and archivists to make their voices heard while the people who can explain the relationships between the born digital records and the business processes that created them are still around to answer questions.

Read more →

Spring 2007:Access and Information Visualization

5 Comments

I don’t often post explicitly about my experiences as a graduate student – but I want to let everyone know about the focus of my studies for the next four months. I am taking two courses that I hope will complement one another. One course is on Archival Access (description, MARC, DACS, EAD and theory). The other is on Information Visualization over in the Computer Science department.

My original hope was that in my big Information Visualization final project I might get the opportunity to work with some aspect of archives and/or digital records. I want to understand how to improve access and understanding of the rich resources in the structured digital records repositories in archives around the world. What has already happened just one week into the term is that I find myself cycling through multiple points of view as I do my readings.

How can we support interaction with archival records by taking advantage of the latest information visualization techniques and tools? We can make it easier to understand what records are in a repository – both analog and digital records. I have been imagining interactive visual representations of archives collections, time periods, areas of interest and so forth. When you visit an archives’ website – it can often be so hard to get your head around the materials they offer. I suspect that this is often the case even when you are standing in the same building as the collections. In my course on appraisal last term we talked a lot about examining the collections that were already present on the path to creating a collecting policy. I am optimistic about ways that visualizing this information could improve everyone’s understanding of what an archives contains, for archivists and researchers alike.

Once I get myself to stop those daydreams… I move on to the next set of daydreams. What about the products of these visual analytics tools? How do we captured interactive visualizations in archives? This seems like a greater challenge than the average static digital record (as if there really is such an animal as an ‘average’ digital record). I can see a future in which major government and business decisions are made based on the interpretation of such interactive data models, graphs and charts. Instead of needing just the ‘records’ – don’t we need a way to recreate the experience that the original user had when interacting with the records?

This (unsurprisingly) takes me back to the struggle of how to define exactly what a record is in the digital world. Is the record a still image of a final visualization? Can this actually capture the full impact of an interactive and possible 3D visualization? With information visualization being such a rich and dynamic field I feel that there is a good chance that the race to create new methods and tools will zoom far ahead of plans to preserve its products.

I think some of my class readings will take extra effort (and extra time) as my mind cycles through these ideas. I think that a lot of this will come out in my posts over the next four months. And I still have strong hopes for rallying a team in my InfoViz class to work on an archives related project.

Read more →

Amazing Visual Tools

1 Comment

This has been a great week for exploring new visual tools. This post will look at one new tool that can compare photos to find images that have similarities to one another, another that creates 3D photo montages and a third that presents grouped keywords for image browsing.

First there is Riya, a visual search engine still in beta. I was especially intrigued by the Riya Personal Search that will let you use “face and text recognition to auto tag your photos”. Imagine a collection of 10,000 photos that include many combinations of the same 25 people. instead of going through and hand tagging them all – you should be able to use this feature to select good examples of each of the 25 people’s faces and let Riya’s magic do the busywork for you.

Over on Research Buzz they talk about Riya’s first commercial venture Like.com in Visual Search Engine for Finding Items by Photo. Basically this site lets you find clothing and accessories based on images in celebrity photos. What about buildings? What about automatic tagging of location by recognition of landmarks with distinctive shapes?

That leads us to Photosynth from Microsoft Live Labs . Photos are compared and stitched together into 3D models of space. ( Read more about how here.) They claim that the version they have out there now is a ‘tech preview’ or a ‘sneak peak’ at what is being built for real back in the labs. I played with it a bit – and it is very interesting. I love the idea overall – but I wish that there were a mode in which you could see the 3D world with many photos showing at once. Right now it shows you a sort of 3D point plotted version of the physical structures and then ‘hangs’ the individual photos in the proper place to give you a bit of ‘skin’ on the 3D skeleton as you browse through the photos. Take a look and play with it to see what I mean.

My fascination with this (beyond the cool factor) is the idea of feeding in photos taken a long time ago to create a virtual 3D space to walk through. The lower east side of New York City in the early 1900s or New Orleans before Katrina. I also wonder at being able to create 3D environments with a tie into time so that another slider control on the screen could let you alter the time you were viewing such that you could view the same facade change in fast forward.

Finally I wanted to share the approach taken by CSA Images for their Visual Brainstorming Index. They have provided a way to view keywords grouped by topic. Their goal is to inspire their customers by showing them terms that might jump out as being useful. This appears a great way to give users a quick grasp of what sorts of information (in this case images) can be found on this site. If a standard tool could show classified sets of keywords for users to explore it could go a long way to easily communicating the kinds of information available in a digitized archive online – especially if the browsable terms evolved as records were added to the collection. I am sure there are other great ways to communicate the big picture, but I think that a way to browse keywords or tags would give users a handle on the records in a way that a well written finding aid might not manage. Ask 100 users what a finding aid is. Then ask the same 100 users what a keyword or tag is. Moving to a model that users are comfortable with can only increase the use of the records we are working so hard to put online. Of course this assumes that the records in question are assigned keywords, tags or subject terms of some sort – but I think there are many paths to that (including the aforementioned Riya tool).

All three of these creations are interesting steps forward in searching through, processing and understanding images. As archivists make more progress in smoothing out the digitization process (or management of existing digital records), they will finally have more time to consider the wide array of tools that might make accessing those records easier. I hope that it happens sooner rather than later… and that those just starting the process of making archival records available online for research might make their plans with innovative ideas like these in mind.

Read more →

Archival Context and Description – Taking It to the Next Level

In her post Describe, display, explain…, Jill Hurst-Wahl of the fabulous Digitization 101 talks about being more ambitious and committed to describing things well. This resonated very strongly for me, for so much of what I believe about enhancing access to and understanding of archival records online is tied up with describing them well. I don’t mean this solely in the traditional use of the term Description as it is used in the archival arena – but more in terms of the Five Ws.

They are not so very different really. The best finding aids and item level meta data can give you the same type of background information that would make any journalist proud. Who created these records? What are they? Why were they created? Where were they created? When? How?

This gives me a great chance to point people again to the Library of Congress American Memory Browse Collections page. I fell in love with it back when I was doing research for the paper that became my SAA 2006 Poster, but I couldn’t put my finger on how to explain why I liked it so much. I think it goes back to that basic journalistic approach to thinking about things. Am I interested in who created the records and why? Browse by topic. Am I interested in where the records were created? Browse by place. Am I interested in when the records were created? Browse by time period. You get the idea. (In a perfect world, there would be an advanced search option that would let me specify more than one of these interests at a time – but that is a topic for another day.)

So much of this goes back to some of the basics of web design. If you care about this sort of thing and you haven’t yet read Steve Krug’s Don’t Make Me Think – do yourself a favor and get your hands on a copy. It is an easy, quick and fun read – and it will leave you wishing that you could force it on the designers of every frustrating website you have ever been forced to muddle through when trying to get things done.

What would it be like if users of online archival collections didn’t need to learn new terminology in order to get the most out of the records? What if you never had to hunt to find the background history for a record because it was so obvious where to click? While I love the idea of thorough item level descriptions – I understand why that is not a reasonable expectation for most digitized archival collections. What I want is the context for any record an easy and obvious click away.

Of course if you want the dreamy version, then I would encourage folksonomy tagging and end user descriptions of digitized items. I know this would not work many places – but I suspect some experimentation with this kind of model with the right sort of collections would take the “Do you know who is in this photo” sorts of appeals to the next level.

I remember asking in my first Archives course if the archivists take information from the users of the records and add that information back into the finding aids. The answer was of the “maybe… sometimes” variety – followed up with an explanation of how archivists often have private files about a collection that are not shown to patrons. The thought was that perhaps an archivist might add a note there. My thinking was that since one of the biggest challenges with archival records is that it is possible that any single researcher is looking at a record for the first time since it was created (at least in any intent sort of way) – wouldn’t you want to harness that attention and use it to help others know what that first person found there?

This is already being done at the University of Michigan with the Polar Bear Expedition Digital Collections. They permit entry of comments at the item level and support search of the text in those comments. Every time someone adds a comment they are extending the description of the item. Take a look this comment on the Silver Parrish diary item (this was listed as the most recent comment just know when I looked at the site).

So thank you Jill for making me think about description again – from a new point of view. The more we do to describe everything fully, make it easy to find that information and let our users add more information to the mix – the more dynamic, usable and alive archival records will become.

Read more →

Squirl.info – an interesting option for putting collections online

1 Comment

I got an interesting email from the folks over at Squirl.info. They wanted me to take a look and see what I thought of their site. I explored it a bit and exchanged emails back and forth with one of the founders (John McGrath).

Okay – remember those big dreams of mine? Specifically relating to A Hosting Service for Digitized Collections and Archival Transcriptions? Well Squirl.info looks like an interesting option to explore with these ideas in mind. I will admit that the current collections are predominately individual collectors showing off what they have accumulated over the years – but you can start to get an idea of how this might work for digitized collections from looking at the Lewis Carroll Postcards and London Hospitals Postcards.

I had lots of questions for John. What about more than one image per item (I want to see the backs of those postcards!) – he says it is on the way. What about the clause in the terms of service about deleting inactive accounts – he says only empty ones.

I wanted to know how customized each set of meta data was based on the type of object you entered – had he considered any of the standards when setting them up. His answer (from an email to me):

In a previous life I wrote content management systems for publishing companies, so I’m aware of classification and taxonomic issues, and I made a conscious effort to play nice with the ones I was aware of when I built our system. Metadata can also be added to all items in the form of tags. We also have a feature that lets you export all your data in comma-separated files, letting you manipulate it however you’d like. Our general approach has been to give users as much flexibility with their data as possible, while trying to keep the site easy to use. And we’re wide open to suggestions.

So far so good. Then we get to the fun parts that are already part of what the folks at Squirl have already implemented:

  • RSS: Imagine providing an RSS feed to fans of your institution as an easy way to provide a steady flow of the latest additions to your online collections.
  • Blog Integration: Check out the Blog Widgets that will let you add a block of up to 10 of your most recently added items from any of your collection to your blog or other web page.
  • Easy Interface: They have done a nice job with making it easy and obvious how to get your content online.
  • Inexpensive: Three collections for free – unlimited collections for $10 a year. There is some mention of no ads with the Squirl+plus plan, but that seems to have more to do with your experience when adding content rather than your users’ experiences when viewing your collections.
  • Integration: Squirl’s collection view pages include links to not only the RSS feed, but also quick and easy icons to let those who view your collection to add the link to digg, del.icio.us, reddit and others.
  • Search: You get the free support of their built in search features.

My wish list is shorter:

  • an ad free mode for those viewing your collection
  • a direct URL for collections with a pretty name (maybe just a direct URL for an institution as the homepage for their collections – with a pretty name)
  • support for transcriptions (ie, to support the ideas I described in my Archival Transcriptions post)
  • support for groups adding content – and control of the privileges for those users (John says this one is in the pipeline, but no promises on when).
  • a way to export your ENTIRE collection (tags, images, meta data – the whole nine yards) – of course to a certain degree any web archiving software could do this for you to some degree

Go take a look at Squirl. See what you think. Create an account and experiment with your three free collections (you can keep them private if you like while you are experimenting). Ask the nice people for things you need – and see what they say. In a world where archivists are wishing for an easy way to get their collections online – this might be an answer for some of them. I suspect that some critical mass of archives using a single tool would help drive further support for features that archivists want and need.

Read more →

Interesting Interface for exploring E-mail: TrampolineSystem SONAR Platform

1 Comment

Conversations and articles about the problem of archiving and accessing e-mail are often accompanied by the wringing of hands or the shrugging of shoulders. It has often seemed to me that figuring out how to archive and facilitate access to e-mail is a challenge that most people would rather ignore because it seems so difficult (and because there are plenty other things that need work too).

“In October 2003 the US Federal Energy Regulatory Commission placed 200,000 of Enron‘s internal emails from 1999-2002 into the public domain as part of its ongoing investigations.” So says TrampolineSystems on their facinating website that lets you explore those 200,000 public domain e-mails using their SONAR platform (that stands for Social Networks and Relevance). I would highly recommend taking a look and browsing around the Enron e-mails.

It appears that SONAR somehow tags the emails without human intervention – though they do not state this specifically one way or the other. The implication from the SONAR PR page is that you plug in the platform – and you instantly have this new access to your information. It is my impression that this works for either a fixed collection of e-mails (as is the case with the Enron emails) – or for an active live e-mail collection that is changing over time.

I like the social network Visualizer and the way it shows you how people are related to one another as represented by their e-mail correspondence. I like the theme and people tag clouds. I like the ease with which I can search for and read emails. I like how clearly they specify what you searched on at the top of your e-mail result list – and how many e-mails, people and themes the list represents.

On the other hand, there are a number of things I wish I could do. I wish that it was clear to me what order the emails are listed in when I do a search on a term. I searched on the word ‘pager’ – and received 2012 emails in no obvious order (most likely relevance – but that is not at all clear). I would like to be able to re-sort the results (by date for example). I would like to be able to add together multiple tags and people to get a scoped list of emails between two people on a specific set of theme.

Just as in traditional archival collections – there is some non-unique information in the mix. I found a generic Hotwire promotional email while looking at the theme The Insider (4th hit on the list). While I suppose spam and legitimate e-mail ads (ie, ones you asked for) are interesting – perhaps software considering e-mail to retain permanently could block some of these somehow.

I like clicking on things in the Visualizer and seeing the social networks hidden within the e-mails – but that gets old quickly unless you are looking for something very specific. I found myself wanting more context. Who are these people? What are their jobs? How are they ‘officially’ related in the corporate hierarchy? How do these e-mails compare with a timeline of events? What about the content of attachments (they don’t seem to be part of this interface)? All of this information could be linked into this interface in such a way as to improve an outsider’s understanding of this amazing landscape of 200,000 e-mails.

All in all I think it is an excellent starting point and I applaud them for trying to find an answer to the email question rather than just ignoring the problem.

(Thanks to Boing Boing for the pointer to this site.)

Read more →

Archival Transcriptions: for the public, by the public

11 Comments

There is a recent thread on the archives listserv that talks about transcriptions – specifically for small projects or those that have little financial support. There is even a case in which there is no easy OCR answer due to the state of the digitized microfilm records.
One of the suggestions was to use some combination of human effort to read the documents – either into a program that would transcribe them, or to another human who would do the typing. It made me wonder what it would look like to make a place online where people who wanted to could volunteer their transcription time. In the case where the records are already digitized and viewable, this seems like an interesting approach.

Something like this already exists for the genealogy world over at the USGenWeb Archives Project. They have a long list of different projects listed here. Though the interface is a bit confusing, the spirit of the effort is clear – many hands make light work. Precious genealogical resources can be digitized, transcribed and added to this archive to support the research of many by anyone – anywhere in the world.

Of course in the case of transcribing archival records there are challenges to be overcome. How do you validate what is transcribed? How do you provide guidance and training for people working from anywhere in the world? If I have figured out that a particular shape is a capital S in a specific set of documents, that could help me (or an OCR program) as I progress through the documents, but if I only see one page from a series – I will have to puzzle through that one page without the support of my past experience. Perhaps that would encourage people to keep helping with a specific set of records? Maybe you give people a few sample pages with validated translations to practice with? And many records won’t be that hard to read – easy for a human’s eye but still a challenge for an OCR program.

The optimist in me hopes that it could be a tempting task for those who want to volunteer but don’t have time to come in during the normal working day. Transcribing digitized records can be done in the middle of the night in your pajamas from anywhere in the world. Talk about increasing your pool of possible volunteers! I would think that it could even be an interesting project for high school and college students – a chance to work with primary sources. With careful design, I can even imagine providing an option to select from a preordained set of subjects or tags (or in Folksonomy friendly environment, the option to add any tags that the transcriber deems appropriate) – though that may be another topic worthy of its own exploration independent of transcription.

The initial investment for a project like this would come from building a framework to support a distributed group of volunteers. You would need an easy way to serve up a record or group of records to a volunteer and prevent duplication of effort – but this is an old problem with good solutions from the configuration management world of software development and other collaboration work environments.

It makes a nice picture in my mind – a slow, but steady, team effort to transcribe collections like the Colorado River Bed Case (2,125 pages of digitized microfilm at the University of Utah’s J. Willard Marriott Library) – mostly done from people’s homes on their personal computers in the middle of the night. A central website for managing digitized archival transcriptions could give the research community the ability to vote on the next collection that warrants attention. Admit it – you would type a page or two yourself, wouldn’t you?

Read more →

Records Speaking to the Present: Voices Not Silenced

1 Comment

When I composed my main essay for my application to University of Maryland’s MLS program, I wrote about why I was drawn to their Archives Program. I told them I revel in hearing the voices of the past speak through records such as those at EllisIsland.org. I love the power that records can wield – especially when they can be accessed digitally from anywhere in the world. It is this sort of power that let me see the ship manifests and the names of the boats on which my grandparents came to this country (such as The Finland ).

All this came rushing back to me while reading the September 18th article 2 siblings reunited after being separated in Holocaust. The grandsons of a Holocaust survivor looked up their grandmother in Yad Vashem’s central database of Shoah Victims’ Names – and found an entry stating that she had died during the Holocaust. One thing led to another – and two siblings that thought they had lost each other 65 years earlier were reunited.

The fact that access to records can bring people together across time speaks to me at a very primal level. So now you know – I am a romantic and an optimist (okay, if you have been reading my blog already – this shouldn’t come as any surprise). I want to believe that people who were separated long ago can be reunited – either through words or in person. This isn’t the first story like this – a quick search in google news turned up others – such as this holocaust reunion story from 2003.

This led me to do more research into how archival records are being used to find people lost during the Holocaust.

The Red Cross Holocaust Tracing Center has researched 28,000 individuals – and found over 1,000 of them alive since 1990. The FAQ on their website states that they believe there to be over 280,000 Holocaust survivors and family members in the United States alone and that they believe their work may continue for many years. As much as I love the idea of finding a way to provide access to digitized records – it is easy to see why the Tracing Center isn’t going away anytime soon. First of all – consider their main data sources – lots of private information that likely does NOT belong someplace where it can be read by just anyone:

While the American Red Cross has been providing tracing for victims of WWII and the Nazi regime since 1939, impetus for the creation of the center occurred in 1989 with the release of files on 130,000 people detained for forced labor and 46 death books containing 74,000 names from Auschwitz. Microfilm copies released to the International Committee of the Red Cross (ICRC) by the Soviet Union provided the single largest source of information since the end of WWII.

The staff of the center have also forged strong ties with the ICRC’s International Tracing Service in Arolsen, Germany – and get rapid turnaround times for their queries as a result. They have access to many organizations, archives and museums around the world in their hunt for evidence of what happened to individuals. They use all the records they can find to discover the answers to the questions they are asked – to be the detectives that families need to discover what happened to their loved ones. To answer the questions that have never been answered.

The USC Shoah Foundation Institute for Visual History and Education consists of 52,000 testimonies of survivors and other witnesses to the Holocaust collected in 56 countries and 32 languages from 1994 through 2000. These video testimonies document experiences before, during and after the Holocaust. It is the sort of first hand documentation that just could not have existed without the vision and efforts of many. They say on their FAQ page:

Now that this unmatched archive has been amassed, the Shoah Foundation is engaged in a new and equally urgent mission: to overcome prejudice, intolerance, and bigotry – and the suffering they cause – through the educational use of the Foundation’s visual history testimonies… Currently, the Foundation is committed to making these videotaped testimonies accessible to the public as an international educational resource. Simultaneously, an intensive program of cataloguing and indexing the testimonies is underway. This process will eventually enable researchers and the general public to access information about specific people, places, and experiences mentioned in the testimonies in much the same way as an index permits a reader to find specific information in a book.

The testimonies also serve as a basis for a series of educational materials such as interactive web exhibits, documentary films, and classroom videos developed by the Shoah Foundation.

I guess I am not sure where I am going with this – other than to point out a dramatic array of archives that are touching the lives of people right now. Consider this post a fan letter to all the amazing people who have sheparded these collections (and in some cases their digital counterparts) into the twenty-first century where they will continue to help people hear the voices of their ancestors.

I have more ideas brewing on how these records compare and contrast with those about the survivors and those who were lost to 9/11, The Asian Tsunami and Katrina. How do these types of records compare with the Asian Tsunami Web Archive or the Hurricane Digital Memory Bank? Where will the grandchildren of those who lost their homes to Katrina go in 30 years to find out what street the family home used to be on? Who will give witness to the people lost in Asia to the Tsunami? Lots to think about.

Read more →

Just Promise to Put It All Online

As reported in Inside Higher Ed’s article Harming the Historical Record, the NEH Guidelines for Scholarly Editions Grants have been updated. The crucial passage is as follows:

In keeping with the goals of the NEH Digital Humanities Initiative, the Scholarly Editions Program requires that applicants employ digital technology in the preparation, management, and online publication of all critical and documentary editions. Projects that include TEI (Text Encoding Initiative) conformant transcription and offer free online access are encouraged and will be given preference. (emphasis mine)

Offering free online access is encouraged (not required) and the description of the Digital Humanities Initiative does sound inspiring. It includes this sentence:

[The] NEH is interested in fostering the growth of digital humanities and lending support to a wide variety of projects, including those that deploy digital technologies and methods to enhance our understanding of a topic or issue; those that study the impact of digital technology on the humanities–exploring the ways in which it changes how we read, write, think, and learn; and those that digitize important materials thereby increasing the public’s ability to search and access humanities information.

I love a lot of the sites that they list as having been sponsored by the NEH (such as Valley of the Shadow and Maryland Institute for Technology in the Humanities), and I am always one for “increasing the public’s ability to search and access humanities information”, but it is so frustrating that the glamour of digital access to records would cross over into requirements for funding Scholarly Editions. The core goal of these grants are described as “[to] support the preparation by a team of at least two editors and staff of texts and documents that are currently inaccessible or available in inadequate editions.”

I feel strongly that this sort of expectation for digitization is rarely set with an full understanding of all the other elements that need consideration, ongoing support and financial backing.

First of all – what does it mean to be ‘put online’. While I understand that they likely desire online access to some version of the scholarly edition created with the grant funding – the requirement is still very vague. One could easily wonder if we are talking about images of the records? Transcriptions of the text of the records? What sort of supporting data must be provided? It isn’t as if you can just upload 10,000 scans of records and create a single page with links to them and call it a day. Of course no-one would think that interface was useful, but it could certainly could be considered as being online. Will the grants provide some provision for supporting the online sites in subsequent years? Websites need hardware, bandwidth and support from IT personnel. Unfortunately there is no accepted, open-source, freely hosted solution for serving up digital records. Some institutions have been experimenting with using Flickr as a Digital Collection Host – but that entire topic (and all the issues inherent therein) is fodder for another post in the future.

Next let us consider copyright and privacy issues. Many archival collections are kept, supported and maintained by an archival institution that does NOT in fact retain the copyright to the records. To demand that a project promise to publish all records for free online would unfairly punish collections that do not have the right to publish all the records online even if they have secured the rights to publish records in books. On the privacy side – archivists must often restrict access to certain records or selected series of a collection due to the private information about individuals included in those records or series. This presents yet another challenge to blanket digitization requirements.

The Inside Higher Ed article went on to mention that by requiring free publication of records online, the NEH is removing creative ways for institutions to find additional funding to support their important work. “Virginia is a major player in archival series, publishing — among others — the Papers of George Washington and the Papers of James Madison . Much material from those projects is placed online, free, Kaiserlian said. But Virginia is also selling site licenses to libraries to enable them to have access to everything, while supporting the work that goes into the project.” As the sources for funding for humanities projects such as these are shrinking every day it is unfortunate that a grant might force institutions to consider the income they would loose if they apply for a grant with such strings attached.

Browsing through the rest of the NEH Digital Humanities Initiative website I did find a lot to be enthusiastic and hopeful about – such as the Digital Humanities Start-Up Grants :

NEH’s Digital Humanities Start-Up Grants will encourage scholars with bright new ideas and provide the funds to get their projects off the ground. Some projects will be practical, others completely blue sky. Some will fail while others will succeed wildly and develop into important projects. But all will incorporate new ways of studying the humanities.

I love it. I want to see what they fund. I want to participate. I want that grant to still exist when I am done with my graduate degree and have more focus to my ideas.

Browsing the sidebar of the main Digital Humanities Initiative page you can see how they are presenting all their grants in the context of being digital in some way. If I want to “create digital humanities tools for analyzing and manipulating humanities data” I should apply for either a Reference Materials Grant or a Research and Development Grant. If I want to “develop a Web site or other digital project for a general public audience” I should apply for a Special Projects Grant. And if I want to “create a digital or online version of a scholarly edition” I should apply for a Scholarly Editions Grant. In some ways it just feels as if they added a ‘digital’ element to all their grants without any other major restructuring, not that I am an expert on the history of NEH grants.

I wonder – if I had arrived at the NEH Digital Humanities Initiative page without prior knowledge of how these grants have been used in the past on existing projects, would I have ended up with a post with similar questions, but less frustration? Less stress over what appears to be perceived (if the quotes in the article at the start of this post are to believed) as a major change to the structure of a grant that many doing fine and important work have come to depend upon? That said – I still think that the issues of vague online access expectations, the challenges related to privacy and copyright and the lack of ongoing funding to support websites and their patrons are real and worth consideration.

Read more →