Assessing Avian - Wind Power Interactions:
Sampling, Study Design and Statistical Issues
by
Kenneth H. Pollock, North Carolina State University
Interactions between birds and wind power facilities are complex to measure and assess but they have importance to all parties involved in setting policy. In this paper I present a discussion of sampling, study design and other statistical issues. The paper is written from the viewpoint of a scientist trained as a statistician with a strong biological background and a deep interest in modeling.
I begin by discussing the various sampling methods used to sample avian populations and how they relate to sampling theory. I emphasize methods involving counts and methods that involve capture and resighting (usually by radio-tagging).
The heart of the paper is a discussion of traditional experimental design, which depends on three principles. These are use of controls; randomization of "treatments" to experimental units; and replication of experimental units within treatments. This leads to a discussion of testing to compare treatments and the importance of the test having high power. Power depends critically on the number of replicates and the inherent variation in the experimental units. With the stage set, I then discuss the difficulties of assessing an environmental impact, such as that of a wind power facility. Often there is a lack of good control sites, no randomization is feasible, and replication is inadequate due to scale and cost considerations.
Given the difficulties with traditional experimental design, I discuss alternative approaches considered in the literature. These include the so called BACI design (Before, After, Control Impact), which assumes that it is possible to replicate experimental units on both control and impacted units both before and after the impact, even though it is not possible to randomize. I then briefly mention the more radical alternative of using mechanistic models in attempting to assess impact; this is not always popular with traditional statisticians. Modeling is considered in more detail in other papers in this conference.
The next part of the paper discusses several types of studies related to avian - wind power interactions. These include avian risk reduction studies, avian - wind farm interaction studies, sampling and modeling to assess population impacts, and preliminary population modeling exercises to aid in the design of all field studies. I emphasize the importance of funding and implementing more examples of each type of study so that we can get to understand their properties in more detail.
I conclude with a discussion and summary of the important issues raised. I emphasize key immediate and longer term decisions that need to be made. I also emphasize the need for good coordination to avoid wasting resources, and the need for future research.
A statistician may be involved in avian - wind power interaction studies at various levels. First, he or she may be involved in deciding how to measure variables of interest on one area or plot. This involves traditional sampling approaches to count or capture-resight methods. A second and more difficult type of involvement concerns assessment of environmental impact at a larger scale involving multiple plots or areas. Traditional experimental design has weaknesses that lead to alternatives like BACI designs or modeling (Fig. 1).
FIGURE 1. How statistical issues affect measurements of avian populations on one plot and assessment of impacts through measurements on multiple plots. The emphasis is on alter-na-tives to traditional comparative experimental designs, which are not very practical here.
Section 2 of this paper discusses sampling issues on one area or plot. Section 3 presents traditional experimental design concepts. Section 4 presents alternative designs that may be relevant to our work. These include BACI designs and modeling approaches. Section 5 presents some possible study designs for avian - wind interactions. Section 6 summarizes the most important recommendations.
2. Sampling Avian Populations
2.1 Finite Population Sampling Theory.-At the smallest scale, a scientist may want to measure certain variables related to avian - wind power interactions on a particular area of land. It might be an area around one turbine, around a string of turbines, or perhaps around a whole wind farm of similar turbines. In all these cases it will be necessary to measure variables using statistical sampling of a finite population (Cochran 1978; Thompson 1992) because it is often not possible to take a census of the whole area.
As an example, let us consider measuring raptor use of an area with a group of wind turbines for one year. In this case a multi stage design might be used (Thompson 1992, Chapter 13). It is necessary to sample some of the days in the year at random. Therefore, the days are what we refer to as primary sampling units. Within each day sampled it also might be necessary to have observers make counts at various randomly-selected times through the day so the observers do not have to count all day. These points in time would be referred to as secondary sampling units. The methods used here are very similar to instantaneous counts used in angler surveys (Pollock et al. 1994).
Whatever the sampling design used, the usual questions about precis-ion of estimators are present. The obvious way to increase precision is to take a larger sample. However, there may be more cost-effective ways to increase precision via stratifi-ca-tion and use of auxiliary variables in ratio or regression estimators (Thompson 1992).
Next we consider briefly methods of sampling birds either by counts or by capture and relocate (typically using radio-tagging).
2.2 Count Methods.-The easiest method of sampling dead or live birds is by carrying out counts in defined areas. When the area of interest is a long narrow strip, or is large enough to contain one or more of these strips, transect methods may be used. In other cases point counts may be used. Often counts of all the birds in an area can be made by searching the area systematically. Important assumptions usually made are that all birds are seen and that no birds are counted twice. The realism of these assumptions depends on the prac-tical aspects of the problem (live or dead birds, species of bird, etc.). A rather statistical treatment is given by Seber (1982). The first chapter of the book by Buckland et al. (1993) on distance sampling is relevant. However, when dealing with large birds like raptors, which are easily sighted, we often will not need the sophistication of the Buckland et al. approach, which takes account of varying detection probability vs. distances of observers from birds.
2.3 Capture-Resight Methods.-In some cases the area to be studied may be large and may include a valued population of birds that inhabits a wind farm (e.g. Golden Eagles in the Altamont Pass, California). One method of estimating survival of these birds is to capture and mark the birds and then resight (or relocate) them. In many cases, the only practical way to do this relocation is with radio tagging (White and Garrott 1990). Impor-tant papers on survival analysis for radio-tagging studies are Heisey and Fuller (1985), Pollock et al. (1989a,b, 1995), and Bunck et al. (1995).
3. Traditional Experimental Design
3.1 Fundamentals.-First, it is important that a statistician or bio-metric-ian should be involved in all aspects of ecological field studies. Following Hurlbert (1984) I suggest that the important aspects are
(a) Specifying the objectives: This involves deciding on hypotheses to be tested, the variables to be measured, the treatments to be compared, the resources available, and so on.
(b) Study design: This involves the structure of the treatments, controls, replication, randomization and blocking if appropriate. (These terms are defined below.)
(c) Study execution: This involves actually implementing the study design in the field.
(d) Statistical analysis: This might involve formal tests or just graphs, tables etc. depending on the study.
(e) Interpretation of results: Are the statistical procedures used valid? Has too much been made of the results in an attempt to make the study seem more important than it actual-ly is? Statistical ideas will be crucial to valid interpretation of the results of any study.
The following definitions are critical in developing experimental design concepts:
Treatment: A set of experimental conditions of special interest to the scientist. Usually there is a need to compare two or more treatments, or a treatment vs. an untreated "control" or reference situation.
Experimental unit: This is the experimental material to which the treatments are applied. An example might be an experiment comparing two types of wind turbine on avian mortality. The two types of wind turbine would be the two treatments and the experimental units could be areas of land where strings of turbines of each type were constructed.
Moore and McCabe (1993), and many other statistics books going back to Fisher (1935) in his classic Design of Experiments, consider the following principles crucial to a compara-tive experiment. These principles could be said to define the traditional experimental design paradigm.
(A) Control: The scientist tries to control (standardize) as many variables as possible except for those associated with the different treatment conditions that are to be compared.
(B) Randomization: The scientist randomly allocates treatments to experimental units so that variables not controlled are allocated equally over units (at least on average).
(C) Replication: Each treatment is allocated to multiple experi-mental units so that unexplained or inherent variation can be quantified. Information about the amount of inherent variability is needed for valid statistical testing procedures.
(D) Blocking: To increase precision for a fixed number of repli-cates, the scientist may randomly allocate treatments within homo-geneous blocks of experimental units if such blocks can be identified in the real experi-ment at hand. Blocking is important but not essen-tial, unlike principles (A, B, C). I emphasize that the blocking factors chosen should be in-de-pend-ent of the treatments being tested (i.e. no interaction between treatment and block effects). Alternatively, it may be possible to improve precision by analysis of covar-i-ance (Steel and Torrie 1980, p. 401) if auxiliary variables are available. An auxiliary var-i-able is one whose relationship to the dependent variable can be defined by a regres-sion model.
3.2 Hypothesis Testing.-For illustration let us consider a simple experiment with two treatments, each replicated n times in a completely random design. The treatments may be two types of wind turbine and each experimental unit may be a string of 10 wind turbines of one type. The variable measured could be number of dead raptors/year.
The classic test for use on this problem is the two sample t-test (Steel and Torrie (1980, p. 96), which is based on a normality assumption, or the corresponding two sample rank sum test, the Mann-Whitney U-test (Steel and Torrie 1980, p. 542). The null hypothesis being tested is that the population mean responses of the two treatments, as estimated by the mean number of dead raptors/year per 10-turbine string, are equal. The alterna-tive hypoth-esis is that the population mean responses are not equal.
The power of the test is the probability that the test will reject the null hypothesis when it is false. Ideally, the power should rapidly approach one as the population means for the two treatments become more and more differ-ent. For a fixed difference between the two population means, the power obviously can be increased by increasing the number of replicates (here, the number of 10-turbine strings being monitored per treatment). An alternative way to increase the power might be to use an alternative paired t-test, or its nonparametric equivalent, if homogeneous pairs or blocks of experimental units can be found (Steel and Torrie 1980, p. 102). Randomization, unlike replication and blocking, does not directly affect the power of the test, but randomization is fundamental to ensuring the validity of the test.
3.3 Difficulties.-The three fundamental components of traditional designs-random-iza--tion, controls, and replication-all can be difficult to satisfy in field studies of environ-mental impact:
Lack of randomization: A fundamental problem with applying traditional experimental designs to studies of avian - wind power interactions is the impossibility of randomizing in most cases. Locations of wind power facilities are fixed by economics and politics, not randomization. It may sometimes be possible to randomize the type of turbine allocated to a particular area of land within a wind farm. However, even that may be problematical if the wind farm with the vari-ous types of turbines is in operation before the experiment begins.
Lack of controls: Because of the large scale of many wind farm devel-op-ments, it is often very difficult to find reasonable control areas for comparison with the treated (wind farm) area. In addition, the cost of monitor-ing the large areas may be prohibitive even if control areas can be found. Also, when designing a study of a wind farm already in oper-ation, it may be difficult or impossible to determine whether, before the wind- plant was constructed, that site was similar to a suggested "control" site.
Lack of replication: It is usually impossible to replicate wind farms. They are unique! Even when two or more wind farms are present in one region, their environmental situations will inevitably differ. It may be possible on rare occasions to replicate control areas but the cost is usually prohibitive. During studies on a smaller scale, where the objective might be to compare types of turbines, replication can be achieved for each treatment.
Hurlbert (1984) has discussed in detail examples of what he calls pseudo replication dur-ing ecological field studies. The widespread practice of using inappropriate or pseudo rep-li-cation is related to the difficulties in obtaining true replicates due to cost and practical-ity.
4. Alternative Study Designs for Environmental Impact Assessment
4.1 Before-After-Control-Impact Type Designs (BACI).-Stewart-Oaten and Murdoch (1986) popularized the Before-After-Control-Impact Design of Green (1979). This design attempts to get around the problems of traditional experimental designs and especial-ly the lack of randomiz-a-tion. The idea is that, by comparing control and impact sides before and after the impact, it should be possible to separate the impact effects from temporal changes. The rationale of BACI designs in the context of avian - wind power interactions was discussed in PNAWPPM (1995, p. 65-69).
Extensions of this concept have been described by Underwood (1994) and others. Underwood (1994) recommends multiple control sites. However, the scale of his examples is much smaller than here. His examples often involve sampling invertebrates in marine intertidal zones affected by sewage projects. There are severe logistical and cost complica-tions in applying these principles in the present context, where both the temporal and the spatial scales are much larger. Eberhardt and Thomas (1991) also discuss the design of environmental field studies. They recommend that a wide variety of approaches be con-sidered for different problems. They also recommend that modeling approaches be con-sidered.
4.2 Modeling Impacts.-Formidable difficulties arise when attempt-ing to apply traditional experimental designs, with their reliance on randomization and replication of different treatment conditions. Logistical and other difficulties also are common when using the BACI-type extensions. I am, therefore, reluctantly drawn to conclude that study design and research on avian - wind power interactions will often have to rely on modeling of the impacts. The scale of the problems encountered neces-si-tates radical methods.
I have been involved in an advisory team of scientists designing a detailed population study of Golden Eagles effected by the Altamont wind energy facilities in central California. The objective of the study is to assess the impact of the wind development on the population of eagles in the area. Is the population likely to decrease because of the wind develop-ment? After considering traditional experimental designs and their BACI modifica-tions, we con-cluded that the only feasible approach to the Alta-mont Golden Eagle study was to use a stage structured population model (Caswell 1989; Shenk et al., these Proceedings, p. ). The inputs to the population model are survival and reproductive rates estimated by sampl-ing the population. We recom-mend-ed use of radio tagging methods to estimate the survival rates and use of nest searches to estimate reproductive rate.
It is important to acknowledge, however, that use of models in environ-mental impact studies is not without serious drawbacks. Inferences made will necessarily be weaker than from traditional experiments and will depend heavily on the validity of model assumptions. Part of the study design should involve validation of model assumptions where possible.
5. Some Possible Avian - Wind Power Interaction Study Designs
In this article I have attempted to show the complexity of studies to assess avian - wind power interactions. Here I present a tentative list of possible studies that might be useful, based on ideas from the NREL Avian Research Brainstorming Group. The studies are very varied and range from small scale to large scale.
5.1 Avian Risk Reduction Studies.-The objective here is to test methods of treating wind turbines to reduce the risk of killing birds. Some possible treatments might be painted blades, tower configuration, perch guards, decoys, and other warning devices. The experi-mental unit would be the area around a turbine or group of turbines in a wind farm. The variables to be measured would be avian utilization and mortality.
As the experimental unit is on a fairly small scale, replication should be possible. Also, randomization of treatments to experimental units should be possible if the study is planned into some new or expanded wind farm developments. Therefore, this study fits into the framework of a traditional experimental design.
5.2 Avian - Wind Farm Interaction Studies (BACI).-The objec-tive here is to measure the effects of wind farms on avian species in the area. For the wind farm area and a reference (or control) area before and after construction, a comparison of important variables (such as utilization, mortality, species composition, etc.) is made
These are very complex and expensive studies to carry out. They can be viewed as being generally of the BACI type. However, it may be difficult to have any replication of control sides, and replication of the wind farm area is usually impossible. Randomization is also not practical.
5.3 Assessing Population Impact by Sampling and Modeling.-Following from Section 5.2, there may be difficulties in implement-ing before and after assessment and also in finding reference sites. There-fore, if one wants to study an important population in detail, there may be a need to combine modeling with sampling, with the latter being used to estimate parameters in the model.
An example of this is already being implemented in the Golden Eagle Population Study in the Altamont Region in central California (see also Section 4.2). We considered a stage structured population model (Caswell 1989). We recommended sampling using radio tagging to estimate survival rates, and using ground nest searches to estimate reproduction rate.
These studies are weaker than traditional experiments. However, they are often the only possible way of assessing population impacts.
5.4 Preliminary Population Modeling Exercises.-In some cases prelim-in-ary population modeling with parameter estimates from the litera-ture may be used to help plan field research. This approach could be very helpful and cost effective, in that field studies would be more likely to be attempted only when they were necessary, and would be more likely to measure the critical parameters.
- Clarify when Standard Experiments, BACI extensions and models are best used in assessments of avian - wind power interactions.
- Develop detailed protocols for different types of study designs.
- Fund and implement at least one study with each type of design so we can learn more about the usefulness of various approaches.
- Assess the results of studies to enable better decisions about the need for and design of further research.
- Use models as planning exercises to help improve the design of future research.
- Continue to include statisticians and modelers in research teams.
I would like to thank all the members of NREL Altamont Golden Eagle Study Review Team and the NREL Avian Research Brainstorming Group for their assistance. Most of the ideas in this paper came from discussions with these individuals. Of course, responsibility for any deficiencies in this paper is entirely my own. I also thank W. John Richardson for editing the manuscript.
Buckland, S.T., D.R. Anderson, K.P. Burnham and J.L. Laake. 1993. Distance sampling. Estimating abundance of biological populations. Chapman and Hall, London, U.K. 446 p.
Bunck, C.M., C.L. Chen and K.H. Pollock. 1995. Robustness of survival estimates from radio-tagging studies to uncertain location. J. Wildl. Manage. (To appear).
Caswell, H. 1989. Matrix population models. Sinauer, Sunderland, MA. 328 p.
Cochran, W.G. 1978. Sampling techniques. Wiley, New York. 428 p.
Eberhardt, L.L. and J.M. Thomas. 1991. Designing environmental field studies. Ecol. Monogr. 61(1):53-73.
Fisher, D.S. 1935. The design of experiments. Oliver and Boyd, Edinburgh, Scotland. 248 p.
Green, R.H. 1979. Sampling design and statistical methods for environmental biologists. Wiley, New York. 257 p.
Heisey, D.M. and T.K. Fuller. 1985. Evaluation of survival and cause-specific mortality rates using telemetry data. J. Wildl. Manage. 49(3):668-674.
Hurlbert, S.H. 1984. Pseudoreplication and the design of ecological field experiments. Ecol. Monogr. 54(2):187-211.
Moore, D.S. and G.P. McCabe. 1993. Introduction to the practice of statistics, 2nd ed. W.H. Free-man, New York. 853 p.
PNAWPPM. 1995. Proceedings of National Avian - Wind Power Plan-ning Meeting, Denver, Colorado, 20-21 July 1994. RESOLVE Inc., Washington, DC, and LGL Ltd., King City, Ont. 145 p.
Pollock, K.H., S.R. Winterstein and M.J. Conroy. 1989a. Estimation and analysis of survival distributions for radio-tagged animals. Biometrics 45(1):99-109.
Pollock, K.H., S.R. Winterstein, C.M. Bunck and P.D. Curtis. 1989b. Survival analysis in telemetry studies: the staggered entry design. J. Wildl. Manage. 53(1):7-15.
Pollock, K.H., C.M. Jones and T.L. Brown. 1994. Angler survey methods and their application in fisheries management. Am. Fish. Soc. Spec. Publ. 24. 371 p.
Pollock, K.H., C.M. Bunck, S.R. Winterstein and C.L. Chen. 1995. A capture-recapture survival analysis model for radio-tagged animals. J. Appl. Stat. (To appear).
Seber, G.A.F. 1982. The estimation of animal abundance and related parameters. Macmil-lan, New York. 654 p.
Steel, R.G.D. and J.H. Torrie (1980). Principles and procedures of statistics: a biometrical approach, 2nd ed. McGraw Hill, New York. 633 p.
Stewart-Oaten, A. and W.M. Murdoch. 1986. Environmental impact assessment: "pseudorep-lication" in time? Ecology 67(4):929-940.
Thompson, S.K. 1992. Sampling. Wiley, New York. 343 p.
Underwood, A.J. 1994. On beyond BACI: sampling designs that might reli-ably detect environmental disturbances. Ecol. Applic. 4(1):3-15.
White, G.C. and R.A. Garrott. 1990. Analysis of wildlife radio-tracking data. Academic Press, San Diego, CA. 383 p.
Sampling Avian Populations.-Several questions and comments were raised con-cern-ing problems associated with repeated count-s of the same birds. Many statistical procedures assume that each observa-tion-al unit is represented in the dataset only once. Violations of this assump-tion are common. Dr. Pollock noted that this was not a serious problem in the Golden Eagle study because it was based on individually tagged animals. Other commenters noted that the repeated-counting issue can be a signif-icant concern in studies of habitat use, where there may be repeated counts in a given area, and that the severity of the problem depends on the parameter being measured and the way in which it is measured.
Traditional Experimental Design.-Questions were raised regard-ing what might be use-ful as a blocking factor. Dr. Pollock noted that habitat or topography could be appropriate for blocking. For example, one block of experi-mental units (e.g. turbine strings) might be on north facing slopes and another block on south facing slopes. A commenter noted that one should not block experimental units based on some aspect of habitat that could change differentially among experimental units during the study. Dr. Pollock indicated that, with blocking, the blocking factor must not interact with the treatment factor(s); if it does, then the blocking factor is really a treatment as well, and should be handled as such.
Analysis of covariance vs. blocking: Dr. Pollock was asked whether analysis of covar-i-ance is useful as an alternative to blocking when homo-gen-eous blocks are not present. He said yes, but cautioned that this method is based on assump-tions about linear models. Dr. Pollock indicated that the clearest distinc-tion between the two approaches is that blocking is appropriate when discrete and homogeneous groups of experimental units can be ident-ified, whereas the covariance approach assumes that there is a regression relationship between a continuously-distributed predictor (auxiliary) variable and the main variable of interest.
Hypothesis testing: A commenter noted that null and alternate hypoth-eses should be defined in advance, not after inspecting the data. Dr. Pollock agreed; traditional experimen-tal design requires predefined hypoth-eses, and traditional statistical tests are not valid if the same data are used to identify hypotheses and then to "test" them.
Another question was whether traditional hypothesis testing can be applied to the following common type of question: whether the impact from Phase I of a development exceeds a threshold level that has been identified as being too severe to allow continuation with Phase II. The answer is yes, based on discussion after the meeting. For example, the null hypothesis could be, "Post-development mortality will not ex-ceed pre-development mor-tal-ity by more than x", where x is some pre-defined "thres-h-old of concern". In most hypoth-esis testing x is 0, but x can be non-zero.
Alternative Study Designs: BACI.-Some commenters noted that prob-lems can arise when the bird populations at nearby treatment and con-trol (or reference) sites are not independent. One participant recommended that sites should be spaced far enough apart so that birds using one study site would be unlikely to also use another study site.
The desirability of more than one reference site, as emphasized by Under-wood, was men-tioned. However, the costs and logistics of multiple reference sites are likely to be dif-fi-cult to accommodate, especially when development of a given wind resource area is just begin-ning. One partic-ipant suggested that government might pay for initial studies, and then charge future developers a portion of the already-incurred cost. Govern-ment would assume a risk; it would recover its initial costs only if the area were later developed.
There was some discussion of the possibility that a site initially selected as a control/reference site might later be selected for development. If there were only one such site, responsibility could fall on the regulatory system to ensure that the control/reference site remains as such. However, if more than one of these sites was selected and studied from before the start of development of a given region, it might be acceptable, within the BACI context, for some (not all) of the initial reference sites to be developed later. However, there was further mention of the logistical and economic constraints on this approach, given the recognized difficulties in obtain-ing an adequately-long set of pre-development data from even one or two sites.
Dr. Pollock noted that Underwood has made strong method-o-log-ical recom-mendations based on invertebrate sampling near sewage outfalls. How-ever, these approaches are very dif-ficult to apply to larger-scale issues such as those associated with avian - wind power interactions.
Alternative Study Designs: Modeling.-One participant noted that the limitations of traditional and BACI designs do not necessarily mean that one should abandon field stud-ies entirely in favor of modeling. He mentioned that, even without randomization or adeq-uate replication, well-designed field efforts (e.g., matched-pair comparisons) can be useful, at least on a weight-of-evidence basis even if not as a basis for formal hypothesis testing.
Dr. Pollock agreed that modeling and fieldwork are comple-mentary and should be done in a coordinated and iterative manner. It is also important to use existing ("old") or additional ("new") data when attempt-ing to validate or confirm a model. It was pointed out that all models are simplifications of reality. For this reason, some consider it more appropriate to refer to a model as "confirmed" than as "validated". However, a model cannot be "confirmed" solely by demonstrating consistency with data used in developing the model.
One participant stated that there may be insufficient incentive to follow through with model validation/confirmation. For the model-er, the benefits accruing when a model is confirmed may be too small to offset the risk of negative consequences should the model be demonstrated to be wrong. However, if the modeler is part of a research team, this should not be a serious concern. Methods should be sought to encourage a productive interplay between modeling, fieldwork, and other forms of model confirmation.
