Re: Accuracy assessment and spatial autocorrelation

From: Bob Powell (bob_powell@erinet.com)
Date: Thu May 06 2004 - 07:02:01 PDT

  • Next message: David B. Clark: "Rectifying IKONOS and Quickbird data"

    While I am by no means the voice of authority on these matters, I do have a
    certain amount of experience, having grappled with the same considerations
    in our study of the Cape Sable Seaside Sparrow (see references below).

    First of all, let me congratulate you on having got hold of Ikonos
    data. Like Phil Nott, I would like to know how you did that without
    leaving behind an arm and/or a leg.

    Secondly, the matter of spatial autocorrelation is simply unavoidable in
    almost any ecological research. Good habitat is almost always spatially
    autocorrelated; that is, if an ecological event takes place in a location
    with certain characteristics, there is usually a high probability nearby
    locations possess the same or highly similar characteristics. In our case,
    we usually found sparrows in pixels surrounded by pixels with the same
    spectral, vegetative and hydrological properties. In the case of
    validating a signature, I don't think it is a problem as the choice of a
    signature is mainly a spectral problem, not a spatial problem.

    I think the way you collected your training samples will stand up to
    scrutiny. I do think you need to be careful in properly locating your
    sample points, both in space and on the imagery. Using such high
    resolution imagery means you need location data accurate to within about 10
    cm. This can only be accomplished using differential corrections to your
    GPS readings. Similarly, I would not trust the rectification of the
    imagery as it comes to you. I suggest you do a very careful
    re-rectification using carefully GPSed locations of your ground control
    points. Again, you need to get your rms error down to centimeters or
    below. Otherwise, you will not be sure that a pixel on the image
    corresponds to the tree located on the ground.

    I infer from your message that the location data comes from USFS data. In
    other words, somebody else did it. I would give their methodology a very
    careful look and then resurvey a representative sample in order to satisfy
    myself that the data are accurate to the precision you need. I find that
    ecologists seldom understand the limitations of GPS data. They tend to
    believe the claims that are written on the box. Careful calibrations are
    absolutely necessary.

    I think there is no question that partitioning your data into a training
    set and an evaluation set is legitimate. In fact, you are fortunate in
    having so much data to work with. It opens up a great many possibilities
    for additional analysis. You can partition the data randomly a number of
    ways, use one partition as the training sample and see how well the
    classifier does on the other partition. I suspect you will find that there
    will not be much difference from one replication to the next. If there is,
    you will know that there is something weird going on and you can do a
    deeper investigation.

    I further infer that you are doing an exhaustive species map. If not, you
    need to be careful in using the maximum likelihood decision rule as it
    tends to classify every pixel as one thing or another. Also, as a matter
    of personal interest, I would like to know what you used for your
    priors. A reply off list would be appreciated.

    Cheers,

    Bob

    References:

    2003. Jenkins, C.L., R.D. Powell and S.L. Pimm. Demonstrating the
    destruction of the habitat of the Cape Sable Seaside Sparrow. Animal
    Conservation, 6:29-38

    2003. Jenkins, C.L., R.D. Powell and S.L. Pimm. Why sparrow distributions
    do not match model predictions. Animal Conservation, 6:39-46

    Robert D. Powell
    Congress Farm Research Institute
    Wilmington, Ohio, USA
    bob_powell@erinet.com

    Ludere cum sacris



    This archive was generated by hypermail 2b29 : Thu May 06 2004 - 07:05:01 PDT