(1) Errors, omissions, and missing historical content in Shepard's:
print v. online (See e-mails, infra, by Shannon Malcolm, Scott Burgh,
and Byron Hill, about mistaken statutory references and overlooked
statutory amendments in print Shepard's.)
I don't know whether Shepard's has a "quality control" program to catch
errors and omissions of the kind just reported. At any rate, it's far
from clear that Shepard's prevents errors or omissions any better in
Shepard's online than in print, even if it's presumably more
cost-effective to correct them online. It's equally unclear whether,
say, for an 1851 decision in the California Supreme Court, all of the
citing decisions from Shepard's volumes in the 1940s still show up
online. They don't appear to, but that is just my anecdotal observation.
Some related questions appear to warrant consideration, not just for
citator nerds like me, but even for mainstream users of the rival
services. I qualify my comments by pointing out what may be obvious: no
one relies exclusively on Shepard's or KeyCite to jump-start case-law
reseach, and quite often no one needs to use them for that purpose.
(2) headnote assignment: Shepard's v. KeyCite
One such question concerns differences in how Shepard's and KeyCite
assign (roughly) comparable headnotes to citing decisions. See Susan
Nevelow Mart's groundbreaking article on the subject, "Cite Checking: A
Brave New World," Legal Information Alert (April 2006).
Her article raises a broader question about precision and recall among
citing decisions that receive headnote assignment. In headnote
assignment, precision is the tendency to (a) make assignments that are
correct, or or that are plausible, though incorrect; and (b) to exclude
assignments that are silly. Recall is the tendency to make assignments
that are correct.* Compare KeyCite and Shepard's for headnote assignment
in frequently cited decisions. If you are a citation Puritan, and prefer
precision over an unmangeable (and questionable) recall, you may not
pardon Shepard's assignment of LexisNexis headnotes for heavily cited
decisions. Once assigned certain LexisNexis headnotes, citing decisions
often crash the party and turn it into a mass Bacchanalia for those
headnotes. Both KeyCite and Shepard's depend on algorithmic automation
of headnote assignment. But Shepard's assignment of LexisNexis headnotes
looks like automation run amok, because hundreds of citing decisions may
have assignment to specific headnotes, even for a given jurisdiction.
Are you hoping to use LexisNexis headnote assignment as a research tool
for relevant case law? Good luck finding the ones that are correct, much
less the ones that are correct and that matter.
(3) Shepard's editorial assignment of Thomson-West headnotes: hasta la
vista, baby?
Another question concerns Shepard's editorial assignment of Thomson-West
headnotes (in print and online) and KeyCite's automated assignment of
such headnotes. It has been claimed that KeyCite's automation achieves
greater recall in assignment of Thomson-West headnotes than Shepard's
editorial assignment, with a relatively small loss in precision.
[Shepard's contests the claim. See Elizabeth M. McKenzie, "Comparing
KeyCite with Shepard's Online," 17 Leg.Ref.Serv.Q. 85, 86-87 (1999).] If
- or when - Shepard's in print disappears, any advantages that Shepard's
editorial assignment may have will also disappear. Of course, we can
expect, when the coup de grace unfolds, that Thomson-West headnotes will
also disappear from Shepard's reports on Lexis.com.
(4) LexisNexis headnotes and (algorithmic?) goblins
LexisNexis headnotes can represent a strange animal. Anomalies among
them may even suggest a mythical creature. One finds them where one
would not expect to find them. That is, they look like mistakes, because
they match, not points of law, but dicta or factual issues. And one
doesn't find them where one might expect to find them - that is,
omissions arise for distinguishable points of law. Or LexisNexis
headnotes combine points of law that should be distinguished. These
problems strike me as (considerably and annoyingly) worse for LexisNexis
headnotes than for Thomson-West headnotes.
I am not, of course, a partisan of either type of headnote, or of either
corresponding form of headnote assignment by Shepard's or KeyCite. I
call the shots as closely as a nearsighted librarian sees them. I may
very well be wrong and would welcome correction. But if you think I am
wrong, I dare you to take the LexisNexis-headnote challenge. Compare
LexisNexis headnotes and Thomson-West headnotes for the same decision.
Choose any decision at random, and repeat. You would, at most, have
anecdotal evidence that LexisNexis headnotes can - at least sometimes -
look like a child's goblins under the bed. But even anecdotal evidence
might be cause to wonder about how well LexisNexis tracks points of law,
rather than phantoms.
Notwithstanding LexisNexis press releases, I find something unusually
mechanical about how LexisNexis headnotes are identified - as if
automation does much of the work, with perhaps random editorial checking
to provide some minimal level of quality control. That is a heretical
conjecture, and I am prepared to retract it as soon as I have evidence
to the contrary. Anyhow, my impression is that for the same decisions,
LexisNexis headnotes will tend to be more puzzling than Thomson-West
headnotes.
*Dan Dabney offers this definition of precision and recall in headnote
assignment.
For the record: I am stating just my own fallible opinion in these
matters. My view carries no other affiliation.
________________________________
From: owner-law-lib@ucdavis.edu
[mailto:owner-law-lib@ucdavis.edu] On Behalf Of Malcolm, Shannon
Sent: Wednesday, August 02, 2006 8:18 AM
To: law-lib@ucdavis.edu
Subject: RE: SHEPARD'S ERROR -- Are you monitoring the list,
Lexis?
Colleagues,
By no means do I excuse the publication of bogus material; it's
reprehensible.But I see a greater issue here: yet another reason
electronic citators are preferable to paper ones. The publishers refuse
to revise and reissue corrected material because, presumably, they find
it economically inefficient. An electronic citator, of course, if and
when it contains errors of this sort, is cheaply and rapidly corrected
with a few keystrokes from an editor.
I know most of my colleagues wouldn't touch Shepard's in paper
with a ten foot pole, save for the few things with no electronic
analogue (e.g., session law coverage, exactly what Scott and Byron are
addressing!); I'm not the first to decide that relying on it in practice
amounts to malpractice. Thus, my thought is that Elsevier realizes it's
paper product is not long for this world and, while they will keep
producing it as long as enough people buy it to turn a profit (a
representative said as much not long ago in the CRIV Sheet, I recall),
they aren't going to put much, if any, resources into fixing these kinds
of snags because not enough of the consumer base knows or cares about
them. (The problem also illustrates why I teach students, where
possible, to use Shepard's, KeyCite, GlobalCite, and anything else they
can to cross check their findings.)
My question to any lurking representatives of Lexis would be:
why not make Shepard's online do everything the print versions can and
then everyone will be happier: publishers can fix errors cheaply and
consumers will have accurate, up-to-date citators?
Shannon Malcolm
Reference Librarian
University of Illinois at Urbana-Champaign
142K Law Building
504 East Pennsylvania Ave.
Champaign, IL 61820
Phone: 217-244-3044
Fax: 217-244-8500
slewism@law.uiuc.edu
"When you paint a lot of pictures, if you're any good,
occasionally you paint something great." - Steve Hogarth
________________________________
From: Scott Burgh [mailto:sgburgh@ameritech.net]
Sent: Tuesday, August 01, 2006 12:05 PM
To: Hill, Byron C.; law-lib@ucdavis.edu
Subject: Re: SHEPARD'S ERROR -- Are you monitoring the list,
Lexis?
This is like the errors I pointed outed last month in Shepard's
Acts and Cases by Popular Name. They ran the wrong tapes and the 1992
changes in Illinois law are still wrong and they refuse to correct it
and hide behind an intellectualy dishonest letter they sent me. But
they have no problem collecting the subscription fees while the
integrity of the publication is suspect.
Why don't you find the direct editor and raise the question of
why these amendments are omitted?
Scott Burgh
Chief Law Librarian
City of Chicago Department of Law Library
"Hill, Byron C." <BHill@bowditch.com> wrote:
A Shepardization for any amendments to Chapter
327 of the Acts of 1979 in Massachusetts revealed no changes, through
the July, 2006 supplement. Yet Chapter 327 was clearly amended by
Chapter 380 of the Acts of 2004 (see:
http://www.mass.gov/legis/laws/seslaw04/sl040380.htm
<http://www.mass.gov/legis/laws/seslaw04/sl040380.htm> ). This is a
troubling error by a resource I used to feel I could count on.
Byron Hill, Librarian
Bowditch & Dewey, LLP
Worcester - Framingham - Boston, MA
Communications from our firm may contain or incorporate
federal tax advice. Under recently promulgated US Internal Revenue
Service standards (Circular 230), we are required to inform you that
only formal, written tax opinions meeting the requirements of Circular
230 may be relied upon by taxpayers for the purpose of avoiding
tax-related penalties. Accordingly, this communication is not intended
or written to be used, and it cannot be used, for the purpose of
avoiding tax-related penalties under the Internal Revenue Code.
This e-mail message is generated from the law firm of
Bowditch & Dewey, LLP and contains information that is confidential and
may be privileged as an attorney/client communication or as attorney
work product. The information is intended to be disclosed solely to the
addressee(s). If you are not the intended recipient, be aware that any
disclosure, copying, distribution or use of the contents of this email
information is prohibited. If you have received this email in error,
please notify the sender by return email and delete it from your
computer system. For more information about Bowditch & Dewey, please
visit our web site at www.bowditch.com <http://www.bowditch.com/>
____________________________________________
This message and any files or text attached to it are intended only for
the recipients named above, and contain information that may be
confidential or privileged. If you are not an intended recipient, you
must not read, copy, use or disclose this communication. Please also
notify the sender by replying to this message, and then delete all
copies of it from your system. Thank you.
This archive was generated by hypermail 2b29 : Wed Nov 14 2007 - 20:46:37 PST