GUIDELINES FOR EVALUATING FISH PASSAGE TECHNOLOGIES

(Initiative 2: Fish Passage Technologies Research Development Process)

Prepared by the

American Fisheries Society Bioengineering Section

Committee Members:

Ned Taft (Chairman)

Ken Bates

Tim Brush

Joan Harn

Al Solonsky

Marcin Whitman

Ed Zapel

January 2000
TABLE OF CONTENTS

SECTION TITLE

1 INTRODUCTION

2 BACKGROUND INFORMATION

2.1 Technologies Defined

2.2 Controversial Issues

2.3 Existing Guidelines and Recommendations

2.4 Guideline Implementation

3 PHASE 1 - CONCEPTUAL DEVELOPMENT

4 PHASE 2 - “LABORATORY” EVALUATION

5 PHASE 3 - PROTOTYPE EVALUATION

6 PHASE 4 - TECHNOLOGY SELECTION AND APPLICATION

6.1 Site Assessment

6.2 Review of Alternative Fish Protection and Passage Technologies

6.3 Design, Operation, and Post-Installation Evaluation

7 LITERATURE CITED

SECTION 1

INTRODUCTION

Losses of fish at hydroelectric projects and water intakes for steam electric plant cooling, irrigation diversions and other municipal and industrial uses have led to the development of numerous alternative fish protection and passage technologies that mitigate this problem. Only a relatively small number of technologies are currently considered by the industry to be highly effective and/or are acceptable to the various agencies that are charged with protecting the resource. Fishery managers and other industry professionals typically greet new approaches to safe fish passage and diversion at water intakes with caution. There are multiple reasons for this caution, which include:

· The results of evaluations of some technologies have been equivocal, with inconsistencies in biological effectiveness both between different test sites and between test years at individual sites;

· Many of the studies conducted in the past have been reported in client reports and conference proceedings that are considered to represent “gray” literature; many professionals are reluctant to accept test results that are not presented in a peer-reviewed document;

· Inventors, manufacturers and/or sales representatives have a vested interest in the sale or use of their technology and may be considered biased in their claims of product effectiveness;

· Due to increasingly stringent requirements for biological effectiveness that have evolved over the last few decades, especially for listed species, “structural” technologies that physically exclude fish (e.g., diversion screens) are generally favored over behavioral barriers which may not be as effective in protecting a variety of fish under variable conditions.

Equivocal results in past studies have resulted from improper applications of technologies and differences in experimental design employed by different researchers with varying levels of experience in the conduct of fishery investigations. Variations in site conditions and fish species and sizes may also give different results. The reporting of results in gray literature, where original data are often lacking and data analyses are not clearly presented, also has added to the confusion over the biological effectiveness of certain technologies and contributed to the skeptical attitude that study results often are overstated. This is particularly true when results are presented or reported by parties with a vested interest in the success of a technology. However, it should be recognized that peer-reviewed documents are often not feasible due to time constraints. For example, license requirements may dictate a reporting schedule that will not provide adequate time for the peer review process. Therefore, gray literature will continue to be a source of information on which decisions will be based. This guideline attempts to address this issue by providing for a type of peer review throughout the process of developing a technology. In this way, even those with a vested interest in a technology, who deserve support for their inventiveness and enthusiasm, can expect to have their invention or product receive a fair and unbiased evaluation.

There is clearly a need for improving the process of evaluating fish passage and protection technologies such that there is greater consistency in experimental design and results and that the evaluation process is scientific and objective. The process also must provide evaluations that are relevant to regional and/or local fishery management objectives. As stated previously, there are still only a small number of technologies that are in common use and that are not considered experimental, despite decades of research and development efforts with a wide variety of technologies. Given the decline in fish stocks in some rivers and the inability to restore historic runs in others, it is in the interest of all parties involved in fishery management and technology development to develop a process that will lead to improved experimental design, increased communication, and, eventually, general consensus on the biological effectiveness (or ineffectiveness) of a technology. This process should replace the “trial-and-error” approach that often has been employed during past studies. The need for standardized guidelines is supported by a government report on fish passage technologies [Office of Technology Assessment (OTA) 1995] that identified a critical need for accepted scientific methods and independent evaluations for the successful development of new fish passage technologies.

At the 1997 Annual Meeting of the American Fisheries Society in Monterey, California, the Bioengineering Section met to discuss this issue. As a result of that meeting, a committee was formed to develop a guideline for improving the process by which fish passage and protection technologies are evaluated. The Committee comprised resource agency and industry professionals. This guideline is the product of the Committee’s efforts. The guideline document is intended to provide standardized procedures for the development, evaluation, and application of technologies that will facilitate fish passage and/or protection through the development of sound scientific evidence. Subsequently, the guidelines are designed to assist technology developers, researchers and fishery managers and regulators in gaining approval of new technologies by providing general development and evaluation steps that have been peer-reviewed by agency and industry biologists and engineers. The proposed approach provides for an ongoing peer review process during technology development and testing that will permit further development and application of effective new technologies more reliably and consistently. Also, having a panel of experts involved from the beginning of technology development should aid in securing funding sources to support the development and evaluation process.

Naturally, while standardized procedures are desirable, any guideline must have a degree of flexibility that recognizes the diversity and varying complexity of fish passage and protection technologies and the methodologies available for evaluating them. Attempts have been made to build such flexibility into this Guideline. The Guideline is intended to serve as a tool for bringing new technologies into practical application. It is not a specific prescription for how new technologies should be evaluated and does not address the issue of what constitutes an aquatic impact and when fish passage or protection technologies are warranted to alleviate any such impacts.

SECTION 2

BACKGROUND INFORMATION

In developing the guidelines, it was considered important to address the following questions:

Downstream fish protection and passage technologies encompass devices that are designed to reduce entrainment and possible mortality of fish at water intakes. This group of technologies includes devices that are used at hydro projects for downstream passage of fish and devices used at other types of water intakes (e.g., pumped storage, cooling water and irrigation diversions) to minimize entrainment and/or mortality. Upstream fish passage technologies include fish lifts and ladders and associated facilities. Tailrace barriers include devices that are used to improve upstream fish passage efficiency by diverting upstream migrants to passage facilities or bypass reaches, or to block access to tailrace areas (e.g., draft tubes) where fish can be injured or migrations delayed.

2.2 Controversial Issues

Many controversies with the application of fish protection and passage technologies have been associated with systems and devices that are used to repel or divert fish from water intakes or pass fish through turbines. Upstream fish passage technologies are better understood for applications with many species, although considerable developmental work is currently ongoing with additional species (e.g., sturgeon), small fishways and culvert passage; controversial issues generally have been related to site-specific designs. Tailrace barriers, although important at sites where there is a need, are required less often than upstream or downstream facilities, and the question of their need is usually more controversial than the technology selected for application.

Controversy associated with the evaluation and application of fish protection and passage technologies have been related to all aspects of evaluations as presented in Table 2-1. In general, controversies arise when industry, consultant, or vendor representatives conclude that a technology is effective and should be considered for general application when the responsible resource agencies or NGOs have concluded otherwise or do not have sufficient information to draw conclusions. Controversies can be associated with site-specific applications of a technology, or with the general application of a device to any given site. Most disagreements center on the issues listed above.

2.3 Existing Guidelines and Recommendations

There have been no formal guidelines published for evaluating fish passage and protection technologies similar to the guidelines presented in this document. There is literature available that presents general information on, as well as specific design and operating criteria for selected technologies (e.g., angled, fixed fish diversion screens); pertinent publications are presented in the List of References. However, standardized evaluation processes have not been developed to provide investigators and resource agencies with data derived from a rigorous scientific evaluation on which they can base judgements on the biological effectiveness of a technology and its potential for further application. The National Marine Fisheries Service Southwest Region (1994) and Northwest Region (1995) have issued Position Statements on the use of experimental fish guidance devices (refer to List of References). While these Position Statements address these devices relative to regional fishery issues, they also (1) reflect the philosophy of a key resource agency and (2) present guidance that is of general importance. Therefore, these Statements (and any others that might be developed by other agencies in the future) should be reviewed by any individual planning to conduct or sponsor a study of an experimental technology.

2.4 Guideline Implementation

Fish protection and passage technologies need to be evaluated and applied in a step-wise manner that will allow investigators and fishery managers to make application decisions using data and information from rigorous scientific assessments. An outline of an evaluation process that will improve the potential for industry and agency acceptance is presented in Table 2-2. A four-phase process is recommended for the development, evaluation and acceptance of a technology:

Phase 1 - Conceptual Development. Establishment of an Expert Review Panel and development of a study plan that outlines the biological and engineering basis of operation and expected effectiveness and presents an approach to initial evaluation. All alternative study methods that meet the objectives of the evaluation should be reviewed and considered.

Phase 2 - “Laboratory” Evaluation. Initial evaluation of the technology at a reasonably small scale in a location where operational and environmental conditions can be controlled.

Phase 3 - Prototype Evaluation. Large-scale field evaluation where the sometimes subtle, yet critical, implications of real-world operational and environmental conditions can be fully understood.

Phase 4 - Application and Evaluation. The Expert Review Panel verifies, based on Phase 2 and 3, the conclusions of the evaluations relative to the degree or range of effective protection provided by the technology. The Panel should also verify that the stated conditions under which further applications can be considered (e.g., species, life stage, and hydraulic and environmental conditions) are valid and that any limitations of the technology are clearly defined.

Each phase is discussed individually in the following sections. It should be pointed out that this process may be an iterative one in which researchers may have to repeat earlier phases during the development of a technology. For example, problems discovered in a Phase 3 prototype study may best be resolved by returning to the laboratory.

SECTION 3

PHASE 1 – CONCEPTUAL DEVELOPMENT

The first step in the evaluation process involves the development of basic information regarding the intended design, operation, and biological basis of a technology that can be reviewed and commented upon by industry and agency experts. This step will act to ensure that the technology is based on reasonable engineering and biological principles and expectations, thereby improving the potential for acceptance following subsequent laboratory and/or field evaluations, as described in Section 4. The following presents the key elements of the technology development process.

Expert Review Panel. It is recommended that an expert review panel be assembled during the initial stages of a technology’s development. The review panel should consist of a diverse group of professionals (e.g., fishery managers, engineers, research scientists) representing groups directly associated with the development of the technology (funding organizations/companies, consultants, regional resource agencies) as well as groups not directly associated with a technology’s development but knowledgeable in the area of evaluating technologies (research universities, consultants, or resource agencies). The review panel should be consulted throughout the development and evaluation of a technology and be involved in assessing study plans, data analyses, and progress and final reports.

Literature Review. A thorough literature review should be conducted during a technology’s development. Literature to be reviewed should include all publications that provide information on biological, environmental, and site parameters that are important to the design and operation of a technology. To the extent possible, the developer of a technology should provide information on the evolutionary, physiological and/or behavioral basis on which the developer believes that the technology will be effective. The literature review should address:

(1) whether the technology is targeted at certain species of fish,

(2) if its effectiveness is expected to be influenced by the behavior, physiology, swimming abilities, age, lifestage and size of the target species, and

(3) if its effectiveness is expected to be influenced by physical conditions such as water temperature, turbidity, salinity, velocity, etc.

All past evaluations and applications of similar devices, including successes and failures, will be important to presenting the concept of a new or modified technology and for providing justification for testing or application of an existing technology. Past failures or shortcomings of the technology and identifying reasons for these shortcomings should be fully disclosed. Lack of transparency on this issue has often generated controversy in the past.

Design and Operation. It is important that individuals who will be asked to support the use of a technology understand its basic design and operation, particularly as these factors may affect product reliability and maintenance costs. Any experience with operation and maintenance problems should be fully disclosed. An evaluation of the reliability of a technology should be an integral part of the study plan. If the technology is proprietary (e.g., the inventors plan to file for a patent), a confidentiality agreement or other form of legal protection should be prepared to allow the disclosure of the design on a “need to know” basis. Many stakeholders are very skeptical of “black boxes” that are accompanied with unsubstantiated claims of potential effectiveness.

General Plan of Study. A general plan of study should be prepared by the developer of the technology (or a qualified Contractor) and reviewed by the Expert review Panel. The general plan should describes the approach to be taken in the next phase of development, namely laboratory and/or field studies. The plan should include recommendations for test methods, possible test locations, test species and life stages, physical, environmental and hydraulic conditions, and data recording and analysis procedures. In essence, the general plan of study is a proposal for conducting an evaluation of a technology. Who will perform the study and how it will be financed are issues outside the purview of this Guideline Committee.

Depending on the technology, it may be appropriate to use physical or mathematical model studies to develop a concept prior to laboratory and/or field testing with live fish (e.g., a screen model to ensure that the design configuration chosen will meet established hydraulic criteria for safe fish passage). In the past, most modeling involved scaled physical models. Recently, computational fluid dynamic (CFD) techniques have been used in developing fish protection and passage technologies, as well as in addressing site-specific application issues. CFD allows for thorough analyses of flow dynamics using standard hydraulic principles and available flow and design data from a site and for the technology being assessed. CFD analyses can be conducted in lieu of physical model studies, or to provide additional information either prior to or after model studies have been completed.

Independent Review and Comment. All biological and engineering data from the technology development effort described above should be summarized in a comprehensive technology development report. The report should include the general plan of study and should be submitted to the Expert Review Panel for review and input. Input from the Panel could lead to improvements in the technology or allow for potential problems with design and operation to be identified early in the evaluation process.

SECTION 4

PHASE 2 - “LABORATORY” EVALUATION

The next step in the evaluation of a technology should be to conduct the laboratory and/or field studies discussed in the previous section of this Guideline. In the context of this Guideline, the term “laboratory” is not intended to describe a physical research laboratory facility per se. Rather, while the term encompasses such facilities, it also includes small-scale test facilities, such as test cages, land-based tanks and flumes that can be constructed or deployed at or near a potential site of application. The key distinction of “laboratory” studies is that they are conducted under a set of tightly controlled conditions.

Laboratory studies have been successful in the past in the development of various fish diversion screens that are now in full-scale use. For behavioral fish protection systems, laboratory studies allow researchers to determine the basic fish response to a stimulus under controlled conditions without interference from the many uncontrolled variables that occur in nature. On the other hand, laboratory studies are sometimes considered to be too controlled and unrepresentative of real world conditions. Therefore, the various advantages or disadvantages of laboratory versus field studies must be carefully weighed when deciding the location for the first evaluation of a technology. The decision on whether to begin with studies in the laboratory or to proceed directly to the field can be addressed by answering the following questions:

· Is the technology new or is it a variation of an existing technology (i.e., are data available from the existing technology that may be sufficient to obviate the need for laboratory testing)?

· Does the technology have numerous alternative configurations and/or operating conditions which need to be evaluated in order to identify optimum engineering design criteria and hydraulic performance prior to testing with live fish?

· Can the technology be “scaled” to a level where meaningful results can be obtained with live fish of the proper species and life stages in a laboratory test facility? (“scaled” refers to a small version of the technology rather than a true scaled model)?

· Will laboratory experiments serve to isolate the behavioral characteristic (e.g., phototaxis) responsible for the observed fish response to the technology (something that is difficult to isolate in the field)?

· Is the technology of such a design that it can be easily deployed on a small-scale basis at a field site?

· As a corollary, does a field test site exist that can provide (1) appropriate physical and hydraulic conditions, (2) target or representative species in sufficient abundance and duration to provide statistically meaningful results, (3) features that will permit the proper deployment of performance monitoring equipment (e.g., traps, nets, bypasses, hydroacoustics, telemetry), and (4) will allow testing without causing unacceptable impacts due to installation (e.g., riparian/upland destruction) or operation (e.g., entrainment of ESA listed species)?

These questions are addressed in the following discussions of laboratory and field evaluations.

Laboratory studies can provide a vital step in evaluating the effectiveness and future applicability of fish protection and passage technologies by providing a rigid scientific framework within which a technology can be studied under reasonably controlled conditions. Such studies are particularly useful in evaluating technologies that can have wide variation in design and operational parameters. For example, fish diversion screens can incorporate a range of screen angles and flow velocities that influence hydraulic conditions. In a hydraulic model, many variations can be evaluated quickly and inexpensively to determine which combination of parameters yields the optimal hydraulic conditions for effective fish diversion with minimal stress or injury. Similarly, laboratory test flumes are effective in evaluating the effectiveness of diversion devices (e.g., screens and louvers) with multiple species over a range of operating conditions in a short time frame.

The primary goal of laboratory investigations should be to collect data that will support the basic biological and engineering principle governing the potential effectiveness of a technology and provide clear evidence that future testing of a prototype at a field site is warranted. It should be clearly understood by all study participants that the results of the laboratory studies may indicate that a technology does not perform as expected and (1) that future testing is not warranted or (2) that major modifications in design or operation are needed. It is natural to expect that the first evaluation of a new technology may not produce the desired results. In such cases, researchers should review the results, make appropriate changes and re-evaluate the technology in the laboratory. In the past, market forces or the desire to proceed to the next level of testing have resulted in inappropriate applications of new technologies in field applications that have led to equivocal results. This approach has heightened the skepticism of many toward new technologies. The following discussion presents the key factors that need to be addressed in planning and conducting laboratory studies.

Goals and Objectives. It is critical to any research project that reasonable goals and objectives of the project are clearly defined and reviewed a priori by the Expert Review Panel, the researchers performing the study, and the inventor/supplier/manufacturer (vendors) of the technology. Poorly developed or understood study objectives can leave the door open to various and biased interpretations of study results. Properly worded goals and objectives also minimize the potential for false expectations among participants. The goals and objectives should pertain to the laboratory phase only. At this point in the development process, goals and objectives should not be related to site-specific needs that might arise in the future when the technology might be applied to meet a specific fishery management program objective or to conform to a regional biological effectiveness requirement.

Study Participants. The primary participants will be the researchers conducting the study. However, the Expert Review Panel and the vendor of the technology (if any) should be involved in a review capacity, providing input into the Plan of Study, any changes to the proposed testing protocols that may become necessary during the evaluation, the test results and the study report.

Test Facility. If the test facility is intended to develop optimum design and hydraulic performance parameters for a technology, it may be appropriate to use a scaled model for the evaluation. As mentioned, such models have been used successfully for the development of a variety of fish passage facilities. If testing with live fish is intended, the facility should be of suitable size that “natural” behavioral responses can be expected. For example, a fish diversion screen test flume should be wide enough that the test fish are not unnaturally crowded and should include a sufficient length of screen to ensure that fish have actively guided on the screen and have not merely passed directly into the fish bypass. Studies of repelling behavioral devices should be conducted in facilities that have adequate escape routes. Particular care must be taken when studying the effects of sound on fish to ensure realistic propagation of the sound signal without reverberation and large boundary layer effects. It should also be kept in mind that evaluations of some technologies on a “laboratory” scale might not be appropriate under any circumstance, requiring researchers to proceed directly to field studies.

If live fish are being tested, adequate fish holding facilities must be provided. Appropriate methods for handling and holding fish should be used at all times to minimize injury and stress to the test fish. Past studies of technologies have occasionally been negatively impacted by the inability of the researchers to maintain test fish in a reasonably healthy state. Unfortunately, the lack of effectiveness of a device has sometimes been attributed to “the poor condition of the test fish.” Such statements have not helped to quell the skepticism of regulatory agencies asked to review the study data. If the test fish are in poor condition, it is recommended that they not be used. Rather, healthy fish should be used and, when the evaluation of a technology includes latent survival, control groups of fish should be held such that treatment and control survival rates can be calculated. In many cases, if control survival is reasonably high (e.g., greater than 80 percent), treatment survival can be adjusted for control mortality.

Holding facility design requirements and fish handling procedures vary by species and are not within the scope of this guideline. However, such information is widely available and can be obtained in other publications (EPRI 1997).

Quality Assurance Plan. A Quality Assurance Plan should be developed to describe and define objectives, experimental design, methods, personnel training requirements, data quality objectives and acceptability criteria, data reduction and analysis methods, and standard operating procedures for all aspects of the evaluation.

Test Species. Selection of appropriate test species and life stages (and related size) is one of the most critical components of a technology evaluation. If the technology development phase has been performed properly (see Section 3), it should be a straightforward task to select species/life stages on the basis of one or both of the following criteria:

· The species and life stages are of great enough importance at enough sites that might employ the technology (if effective) that they are appropriate for evaluation.

· There is an evolutionary, physiological and/or behavioral basis to expect that the selected species will adequately represent the performance of the technology for another species/life stage of interest.

To the extent possible, the species and life stages should be ones that are in need of protection. While it may be appropriate to use one species as a surrogate for another species, both species should be of importance and the surrogate should reasonably represent an important, known attribute of the other species (e.g., swimming capability, body shape, behavior). Also, it may be appropriate to use surrogate species in preliminary trials; however, detailed evaluation with target species must eventually be completed.

Test Conditions and Procedures. To the extent possible, tests should be conducted under the full range of (1) operating conditions of a technology (e.g., device settings, such as screen angle or sound amplitude) and (2) environmental conditions (e.g., water quality, lighting).

Data Analysis. Appropriate and adequate analyses of data are very important aspects of any scientific evaluation and will be vital in gaining acceptance of study results. Use of inappropriate statistical models can lead to erroneous conclusions. Consideration should be given to involving a professional statistician for assistance in developing the experimental design for laboratory studies, as well as in the analysis of data. It is incumbebt upon reviewers to have an understanding of the analysis techniques or to consult an authority on the specific statistical approach employed. Due to the natural vagaries in biological response, data often can be widely scattered, requiring a large number of replicates to produce statistically reliable results. The inclusion of a statistician on the Expert Review Panel is recommended. A well-defined plan for data collection and analysis can avoid the problem of “false positives” and “false negatives” that have occurred in past studies. A well-defined Quality Control/Quality Assurance plan should also be developed.

Reporting. Laboratory study reports should present all methods, collected data, statistical analysis results, and conclusions in a comprehensive and logical manner. A description of methods should include test facilities, equipment, procedures, and data analysis methods. Data summaries, trends, and statistical results should be presented in tabular and graphical formats in the body of a report and, to the extent possible, all raw data should be included in appendices. A lack of information pertaining to how a study was conducted, how data were analyzed, why some data may have been discarded, and thorough justification of all conclusions and recommendations often leads to controversy. Test data and information included in a report should be adequate to allow reviewers to independently replicate analyses and assess the validity of any conclusions or recommendations.

The report should also include a summary of previous studies (if any) related to the technology and provide a complete bibliography.

Acceptance of Results, Recommendations, and Conclusions. The Expert Review Panel should review laboratory study results, conclusions, and recommendations and verify that the conclusions drawn are supported by the available data. The review panel would be responsible for submitting comments on draft reports and for confirming that the study was conducted according to the Plan of Study developed in Phase 1. Verification of the results is not an endorsement of the technology but rather a statement that (1) the methods used to evaluate the technology were appropriate and (2) the conclusions drawn are consistent with the results obtained. General considerations for accepting results of fish protection and passage technology evaluations are summarized in Table 4-1.

Study Participants. Generally, the participants will be the same as in the laboratory with the possible exception of the researchers. Field studies require different skills and are best performed by experienced field organizations. At this stage, resource agencies may be expected to have a greater role in defining acceptance standards.

Site Selection. Site selection criteria should be developed for identifying an appropriate site for field studies of a technology. The criteria will vary depending on the type of technology being evaluated, but general factors to be considered in the site selection process include the following:

· Species Availability: The species of interest must occur at the site in sufficient numbers and for long enough periods to provide statistically meaningful results. It should be demonstrated that the evaluation of a technology at a given site will not cause unacceptable injury or losses to the fish or other sensitive species involved.

· Site Representativeness: The site should be reasonably representative of other sites of intended future use of the technology relative to fish species and life stage present, site layout and operating conditions.

· Hydraulic Conditions: The existence of appropriate hydraulic conditions is one of the most critical requirements for site evaluations of technologies. Velocities that are appropriate for the species/life stages being evaluated are essential. If the technology’s effectiveness is considered to be sensitive to hydraulic conditions such as non-uniform velocities, turbulence, and effects of debris loads, these factors need to be specifically included or avoided, depending on the objective of the field evaluation.

· Existing Features: Some technologies have specific power requirements, installation specifications and/or operational needs that cannot be met at all sites. If a site has existing design and operating features that can support these needs, considerable cost savings can be realized.

· Past Experience: Sites at which previous studies of fish protection or passage technologies have occurred offer two advantages - (1) many of the “unknowns” of a new site have been previously identified and (2) sampling equipment with proven capabilities might be available for use and may allow for side-by-side comparison.

· Ability to Modify Project Operations: Evaluations of some technologies require periodic modifications to normal operations at a test site (e.g., shutting down hydro units to permit sampling equipment installation or preferential operation of a unit). The need to modify operations should be identified prior to the site selection process and be made known to potential site operators.

· Access and Safety: Reasonable access to test and sampling equipment should be available to permit researchers to conduct the study in a safe manner.

Scale of Prototype Field Facilities. Many of the past studies that have produced equivocal or controversial results suffered from the selection of an inappropriate scale for the first field trial of a new technology. There has been a tendency to evaluate new technologies on too large a scale, which can make monitoring of performance difficult and expensive. During the site selection process, attempts should be made to identify sites, or areas within sites, where the technology can be installed for testing under appropriate physical and hydraulic conditions at a scale that is large enough to produce data that is representative of results that would be expected at larger scales. A common approach to prototype testing is to install a technology on one unit of an operating plant; if the technology is effective, it can be “scaled up” by installing it at the other units.

Test Conditions. Test conditions include the operation of a technology (e.g., device settings, such as screen angle or sound amplitude), operation of site facilities (e.g., hydraulic conditions, turbine operation, diversion intakes), and environmental conditions (e.g., water quality, debris load, lighting conditions). To the extent possible, all important variables and combinations of variables (both controlled and uncontrolled) should be evaluated. A phased approach to testing is recommended in which a wide range of test conditions is sequentially narrowed down to a few optimum performance conditions. Where possible, a bracketing approach to testing is recommended (e.g., starting at extremes in the ranges of particular variables). This approach could substantially reduce testing and analyses costs in some cases.

It should be recognized that natural variables outside of human control can confound test results or, in extreme cases, cause the loss of data. For example, high flow conditions at a hydroelectric project during the fish migration period of interest might result in the planned test fish bypassing the test facility (e.g., by passing through opened spill gates). Although such events are a fact of life, it is encumbent on planners and reviewers to adequately consider the potential for these events. Every effort should be made to minimize the likelihood that such events will occur or to minimize the impacts of the events on the data if they are unavoidable. When data are lost due to them, the loss should be acknowledged and subsequent analyses, if any are possible, should clearly state the limitations of the data and take those limitations into account.

Target Species. The species selection process for field evaluations is similar to that for the laboratory. Target species may include specific species for which a technology is designed, or representative species if a device is designed for application with many different types of fish. Target life stages (i.e., size classes) also will be important to the evaluation of most technologies. In some cases, interactions with predatory species may be important in prototype evaluations.

Data Analysis. Appropriate and adequate analyses of data are very important aspects of any scientific evaluation and will be vital in gaining agreement on conclusions based on field study results. With more uncontrolled variables in the field than in the laboratory, the analytical techniques to be used should be developed a priori by individuals knowledgeable in the design and operation of test site features. As with the laboratory evaluations, consideration should be given to involving a professional statistician for assistance. It is encumbent upon reviewers to have an understanding of the analytical techniques used or to consult an authority on the particular approach employed. The inclusion of a statistician on the Expert Review Panel also is recommended.

Reporting. Study reports should present all methods, collected data, statistical analysis results, and conclusions in a comprehensive and logical manner. A description of methods should include site design, test facilities, equipment, procedures, and data analysis methods. Data summaries, trends, and statistical results should be presented in tabular and graphical formats in the body of a report and, to the extent possible, all raw data should be included in appendices. A lack of information on how a study was conducted, how data were analyzed, and why some data may have been discarded, coupled with an incomplete justification of all conclusions and recommendations, has led to most of the controversies that have been experienced in past evaluations and application of new and experimental technologies. Test data and information included in a report should be adequate to allow reviewers to independently replicate analyses and assess the validity of any conclusions or recommendations.