Matching hierarchy

[sckott]

• 1st cut
  • match on catalognumber & institutioncode
  • if no match, then ->
• 2nd cut
  • location, datetime, collector, identification/taxon name
  • if no match, then ->
• 3rd cut
  • ???

[sckott]

notes on each

catalognumber/institutioncode

• may want to match whole string & from front to middle, then from back to middle.

location/datetime/collector/tax name

• location
  • for lat/long, sig. dig. could vary a lot, hmmmm
  • for geographic place names, do matching on place names? or geocode, then match?
• datetime - first standardize dates, then attempt match
• taxonomic names
  • use name parser, then match on genus, sp. epithet

[mjcollin]

Rod Page post ideas bout how catalog numbers can be matched using sub-strings:

http://iphylo.blogspot.com/2012/02/how-many-specimens-does-gbif-really.html

[mjcollin]

Article on deduplicating across ALA and GBIF: A specialist’s audit of aggregated occurrence records

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3677402/

[mjcollin]

Need to figure out how to segment the dataset to avoid doing a complete cross join of the datasets. Can we compute entropy for each field type and go in increasing order?