Contents of Presentation

• Persistent Identifiers at First Glance

• Tangled Terminology

• Persistence Aside, What about Complete Identification?

• Persistent Identifiers at Second Glance (adding what we've learned)

• So What If Some URLs Break?

• Setting User Expectations

• Who is Breaking URLs

• Persistent Identifiers at Third Glance (again adding what we've learned)

• The Archival Resource Key (ARK) Scheme

• ARKs in Action

• Persistent Identifiers at the California Digital Library (CDL)

• ARK Summary

• Postscriptum

Persistent Identifiers at First Glance

• Goal: long-term identifiers

• Problem: URLs break

• Why URLs break: because objects are moved, removed, or replaced

• Conventional hypothesis: use indirect names (PURLs, URNs, Handles) instead; what worked for DNS should work for digital object references

  • [ The DNS (Domain Name System) is what allows us to say www.library.ucsf.edu instead of 128.218.39.5. ]

• Why aren't we done yet?

Tangled Terminology

• Persistent identifiers? But we have them: ISBN, LCCN, ...

  • no - we really mean actionable persistent identifiers

• "Actionable" means we have widely available tools that, given an identifier, take us close to the object (e.g., within 0 or 1 clicks, or to a password challenge)

• What's an identifier (as opposed to a mere string of data)?

  • 1L039G81U -- is this an identifier?

  • http://www.cdlib.edu/staff/~greenstein -- is this an identifier?

• An identifier is not a string, but an association between a string and an object

  • now it makes sense to say "an identifier that breaks"

• By the way, URLs are locators, not locations

  • http://ntsrvr2.abc.org/docsfolder/version3.htm -- not even this one

Persistence Aside, What about Complete Identification?

• How do you know if you have an identifier at all?

  • only if a provider can vouch for an association

• What constitutes proof depends on how picky you need to be; examples

  • "click on it" returns something, anything at all

  • there exists a metadata record that one can compare to a delivered object to decide if it is the requested object

• In general, inspection and comparison are called for

  • this requires a provider-supplied metadata receipt or manifest

  • it always helps in the comparison process if the identifier in the metadata record is visible right on the returned object

• Providers who cannot supply a metadata record...

  • how well do you trust their ability to manage their objects at all?

Persistent Identifiers at Second Glance (adding what we've learned)

• Goal: long-term actionable identifiers

  • Requirement: that identifiers deliver you to objects (where feasible)

  • Requirement: that identifiers deliver you to object metadata

  • Desirable: each object should wear its own identifier

• Problem: URLs break (that is, associations between URLs and objects are not maintained)

• Why URLs break: because objects are moved, removed, or replaced

• Conventional hypothesis: use indirect names (PURLs, URNs, Handles) instead; what worked for DNS should work for digital object references.

• Why aren't we done yet?

So What If Some URLs Break?

• Providers rarely promise they won't

  • and many URLs really should break, especially if it costs anything to maintain the associations that they represent

• How to indicate the suggestion of a promise

  • typically with a distinct literal marker string

  • e.g., "purl", "doi:", "urn:", or "ark:"

• But promises aren't simple binary yes/no statements; instead they are faceted statements, including dimensions such as

  • how long the identifier (association) will be valid

  • how long the object (access) will be available

  • how invariant its content is (e.g., subject to revision)

Setting User Expectations

• No general way for users to tell durable URLs from fragile URLs

  • best prediction is based on who made what promise

• Providers of persistent identification play two main roles

  • name assigning authority (initial assignment)

  • name mapping authorities (to objects and metadata)

• Making a persistence commitment known...

  • a marker string (purl, doi:, urn:, ark:) is not enough

  • identifier must deliver you to a multi-faceted promise

Who is Breaking URLs

• Passive voice alert: associations don't break on their own

  • providers break them neglectfully or deliberately

  • changing identifier syntax won't fix the breakage, because an organization that doesn't maintain indirection tables now (HTTP redirects) does not spontaneously become the kind of organization that does

  • DOI/URN/PURL indirection tables are ripe for similar neglect

• Most providers neglect URL redirection tables now

  • because they don't care or can't afford persistence

• Persistence is for the stalwart few, and for only a few of their objects

• Persistence is purely a matter of service and luck

  • what's left is to link would-be users to basic persistence services

  • there are no guarantees in life, but we can help users make up their minds

Persistent Identifiers at Third Glance (again adding what we've learned)

• Goal: long-term actionable identifiers

  • Requirement: that identifiers deliver you to objects (where feasible)

  • Requirement: that identifiers deliver you to object metadata

  • Desirable: each object should wear its own identifier

  • Requirement: that identifiers deliver you to statements of commitment

• Problem: URLs break for some objects (that is, associations between URLs and objects are not maintained) and we have no way to tell which ones will or won't break.

• Why URLs break: because objects are moved, removed, or replaced - completely normal activities - and the provider in each case demonstrates insufficient commitment to update indirection tables, and/or to have planned identifier assignment carefully. Persistence is in the mission of few organizations.

• Conventional hypothesis: use indirect names (PURLs, URNs, Handles) instead; what worked for DNS should work for digital object references. Wrong. Indirection is spectacularly successful and elegant in DNS, but it's a side issue in the provision of digital object persistence.

• Why aren't we done yet? Because the service issues have not been squarely faced.

  • This is what the California Digital Library is pursuing, using the ARK scheme.

The Archival Resource Key (ARK) Scheme

• ARK (Archival Resource Key) meets previously stated requirements and more

• Anatomy of an ARK

  
            http://foobar.zaf.org/ark:/13030/654xz321
            \___________________/ \__/ \___/ \______/
              (replaceable)        |     |      |
                   |         ARK Label   |    Name (assigned by the NAA)
                   |                     | 
     Name Mapping Authority             Name Assigning Authority
            Hostport (NMAH)              Number (NAAN)
  
  

  • after the web, the core identity remains -- ark:/NAAN/Name

• NMA (Name Mapping Authority) makes an ARK actionable

  • identity-inert, disposable, brand-friendly booster rocket

• NAAN (Name Assigning Authority Number) identifies CDL

  • prototype NAAN registry at National Library of Medicine

ARKs in Action

• Services activated by appending '?' (description) or '??' (promise)

  
            http://foobar.zaf.org/ark:/13030/654xz321?
            http://foobar.zaf.org/ark:/13030/654xz321??
  

  • outputs are machine-readable and human-readable

  • automated probes of the 3 services can detect still-born ARKs

  • periods optionally reveal variants, slashes reveal containment

• NMAH replacement algorithm modeled on URN resolver discovery service

  • uses existing NAPTR record in DNS

  • can be bootstrapped using a small, replicated, plain text file registry

Persistent Identifiers at the California Digital Library (CDL)

• Principle 1: persistence is purely a matter of service

  • explore use of ARKs (PURLs for stuff outside CDL control)

• Principle 2: remove recognizable semantics from identifiers

  • whether intended or not, semantic rot is a liability

  • still room for limited, identity-inert branding

• Principle 3: identifier composition guidelines

  • apply check characters in generated ARKs

  • restrict character set to digits and non-vowels

  • don't generate more than two non-digits in a row

  • hyphens never generated, so received hyphens can safely be ignored

  • pseudo-random generation avoids "series semantics"

ARK Summary

• A specially constructed URL encapsulating a globally unique identity

  • hostname as branded booster rocket that doesn't affect comparisons

    
                 http://foobar.zaf.org/ark:/12025/654xz321
               http://sneezy.dopey.com/ark:/12025/654xz321
                                       ark:/12025/654xz321
    

  • access not just to an object, but to its metadata and to a promise

  • promises are compound (not binary) statements

  • carries provider promises, not irrelevant promises from defunct namers

• Complete technical specification:

  • http://www.ietf.org/internet-drafts/draft-kunze-ark-07.txt

Postscriptum

• What about these neat URLs doubling as ad hoc metadata containers?

  
          http://ajpcell.physiology.org/cgi/content/full/278/2/C391
  

  • hard to give up, because they're authoritative too

  • but semantics are better in real metadata, tightly bound to the identifier

  • ... and it makes for identifiers that age and travel well

• Tools under development

  • identifier generator service and open source software

  • packaged resolvers and binders