The primary goal of evaluation during a Firefighter’s probationary period is improved performance; i.e., accurate evaluations should logically lead to strategies for improvement. The following statements emphasize the importance of accurate evaluations for Probationary Firefighters:
The purpose of the probationary period is to provide guidance to new employees and to have company officers work with them to make them into efficient firefighters. For a probation program to be effective, the worker’s immediate supervisor should be required to approve, in writing, the retention or termination of the employee before the probationary period ends.
……………………………………………………………………………………………..An ongoing review process should be used periodically during the probationary period and should include periodic appraisals and counseling. (Source: William S. Johnson in The Fire Chief’s Handbook (5th edition) edited by Joseph R. Bachtler and Thomas F. Brennan)
Perhaps a supervisor’s most challenging job is that of determining the relative efficiency or service value of each employee. The job is difficult because the supervisor must use unbiased judgment in evaluating those personal qualities and characteristics that best represent employee work performance. In essence, the supervisor is taking a photograph of the employee’s overall performance for a specific period of time, and if a proper appraisal is not made, a distorted work picture results and the employee may not receive fair treatment. (Source: Employment Performance Appraisal Booklet, City of Long Beach, Civil Service Commission)
Each Probationary Firefighter shall be evaluated in a number of categories which, when taken together, reflect the totality of the job for which a Probationary Firefighter was hired. When possible, these categories will be rooted in documents such as the Monthly Probationary Evaluation instrument. These documents are used to measure and describe the knowledge, skills and behaviors required for satisfactory job performance.
Once the job related categories have been determined, what is to be evaluated has been identified. How to rate these categories becomes the issue. How is based upon the Probationary Firefighter’s performance as measured against the evaluation factors contained in the Monthly Probationary Evaluation instrument. The Monthly Probationary Evaluation is used to insure that each Captain’s rating of a Probationary Firefighter will be equal and standard throughout the program. The Monthly Probationary Evaluation must be applied equally to all Probationary Firefighters, regardless of their experience, time in the program, or other incidental factors.
The Captain’s role is to examine the Probationary Firefighter’s performance and choose the appropriate description as provided in the relevant Monthly Probationary Evaluation. The Captain selects the description that “fits” the behavior of each dimension of evaluation factor he/she is evaluating (i.e., “above standard,” “at standard,” “below standard,” “significantly below standard”). The most difficult part of the evaluation process for the Captain is to surrender their opinion of what the Probationary Firefighter’s performance should be. The Captain will rate the Probationary Firefighter pursuant to the language in the guidelines, if the Probationary Firefighter’s performance is consistent with the language of that guideline. The Captain shall have no discretion in this matter. It is the only way that objective evaluation will be accomplished. If each evaluator (Captain) uses the same measuring device (i.e., Monthly Probationary Evaluation), you should see the same results and the same scores for similar Probationary Firefighter behavior, as the Probationary Firefighter moves from one Captain to another.
Following your selection of the behavioral description that “fits” the Probationary Firefighter’s performance, write “specific descriptions” to support your rating. Your “specific descriptions” must paint an accurate picture of the Probationary Firefighter’s performance on each evaluation factor or dimension. (A more complete discussion on narrative statements is presented later in this document under the “Evaluation Content” section.)
Evaluation is a process, and like any process, if it is used in a way other than how it was designed to be used, it will not be effective. There are a number of common performance evaluation errors, which if present in the Probationary Firefighter’s training program, may destroy the goals of the program and the morale of the participants.
If the objectivity of the evaluation process is called into question, it is most likely because the rater (Captain) did not follow the guidelines, and one or more of the following “evaluation errors” crept into the evaluation process.
The error of leniency occurs when the rater assigns scores beyond those, which are deserved. This error often occurs because the Captain introduces the variable of “experience” or the amount of time the Probationary Firefighter has spent in the program. In other words, the Captain recognizes the performance as less than adequate for the job of Firefighter, but considers it “OK” given the limited amount of experience the Probationary Firefighter has had. The same performance, seen several weeks later, may result in the awarding of an unacceptable rating. If the performance does not change, the score should remain the same regardless of how long the Probationary Firefighter has been in the program. Standards do not fluctuate…a mile is 5,280 feet long, no matter where in the country you may measure it!
The error of personal bias (also called the “halo effect”) occurs when the rater allows personal feelings about the Probationary Firefighter to affect the ratings. Particular “likes”, or “dislikes”,
limits appraisal objectivity. What is rated is whether or not an individual can do the job as described…period.
The error of central tendency is seen when the rater routinely “bunches” scores toward the center of the rating scale. Central tendency errors also occur when the rater does not pay close attention to performance and to be on the “safe side,” or to avoid any controversy, rates in the middle of the scale (i.e., “slightly below standard” or at “standard”).
The error of related traits happens when the rater gives the same rating to traits that he/she considers related in some way. The value of rating each trait separately is lost and the overall rating loses specificity. As an example, a Captain has a good working relationship with a Probationary Firefighter (“Approach Towards Supervisor”) and he/she confuses this evaluation factor with “Response Towards Other Members.” Thus, he/she rates the “Response Towards Other Members” the same as “Response Towards Supervisors.” In fact, the Probationary Firefighter, may have a below standard relationship with other members.
Event bias comes into play when one or two traits (or a particular behavior) dominate the appraisal. The rater may evaluate all remaining traits based on the dominant trait or performance. An outstanding bit of work, or a severe mistake, not treated as an individual occurrence, may bring about the “halo effect.”
“No rookie ever gets an above standard” is a belief often expressed. The Monthly Probationary Evaluation should be based on real-life experiences and should not reflect artificial standards. While it may be difficult for many Probationary Firefighters to perform at a “above standard” level in a number of categories, that score could be attainable for some. There is no place for unrealistic expectations/goals in a job-related performance evaluation system.
“Not enough” improvement to get a higher score is a judgment that a Captain may make when the Probationary Firefighter’s performance is slightly better than that described as “needs improvement” in performance, in the Monthly Probationary Evaluation.
The “room to grow” theory has been around for a long time—too long as a matter of fact. The evaluator, wanting to “motivate” the trainee to work harder, assigns a rating less than what the employee deserves. When someone fails to get the recognition that they deserve they may experience a loss, rather than a gain, in terms of motivation.
“Averaging scores” has no place in an evaluation system. Captains who assign a rating based on an average of the Probationary Firefighter’s performance for the day have selected a score that is not accurate nor is it indicative of the true nature of performance. As an example, a Probationary Firefighter usually activates his PAD device when he dawns his SCBA. The Captain gives him an acceptable rating because “on average” the Firefighter activates his PAD. Because this is a safety issue, it is a condition requiring 100% compliance. Thus, the Probationary Firefighter must be rate below standard, and written documentation should be made. Inconsistency results when the rater assigns an acceptable score, but documents unacceptable performance.
Captains are often uncomfortable about giving an “unacceptable” rating when a Probationary Firefighter has performed well in an area throughout the day with one or two exceptions. Objective evaluation requires that the rater acknowledge the mistake(s) by assigning a rating less than “acceptable.” The Captain does not want to appear “picky” to the trainee, particularly with one who is doing very well, is in the latter stages of the program, and will likely be retained. The Captain must give the trainee an “unacceptable” rating in an area regardless of how minor or infrequent the mistake when weighed against the Probationary Firefighter’s otherwise good performance.
Finally, there are other errors that Captains must guard against. These are biases that have a tendency to influence us when rating the performance of another. Taking into account such factors as a person’s standing in the Academy class; his or her relationship to another member of the Department; the presence or absence of educational achievement; his or her age, gender, race or sexual orientation; physical appearance must not dilute objectivity. Performance-related evaluations tend to be more objective and center on what the individual does rather than who the individual is.
The only measure that a Captain should use when evaluating the behavior and performance of a Probationary Firefighter is the Probationary Monthly Evaluation. The rater must avoid all other subjective influences.
To make the most effective use of the narrative portions of written evaluations, it is important for the Captain to remember four “goals” of documentation. To meet these goals, the documentation should be:
1. Clear
2. Concise
3. Complete
4. Correct
The following suggestions will aid the Captain, Battalion Chief, and Training Staff in accomplishing these goals.
Set the stage. Provide a description of the situation or conditions that are present when the Probationary Firefighter performs. This will allow the reader to understand more fully why the Captain is satisfied or not satisfied with the behavior.
Example (“Response Towards Training and Emergency Activities”): Firefighter Jones, when asked by me to present a table discussion on how to approach elderly Cambodian-American patients, met with several members of the United Cambodian Community, Inc. to understand culturally related communications. He then developed three medical aid scenarios in
cooperation with Robert Suon, MD, Mary Suon, RN, and David Say, DDS. The seven other member of our station all commented that they learned a great deal from his scenarios.
Use verbatim quotes. It is sometimes clearer to report what was said rather than attempt to describe the effect of the words.
Example (“Personal Traits—Verbal Communications”): Firefighter Inzunza walked up to the patient in the alley behind the convenience store at 1234 Abbey Lane and said loud enough so that patient could hear, “¡No este cavrón otra vez!” [“Not this asshole again!”] Immediately, the patient spit in Firefighter Inzunza’s face.
Report the facts; avoid conclusions. Let the facts speak for themselves. Do not form conclusions, unless they are clear from the facts.
Example (“Fire Prevention/Public Relations—Notice Preparations”): The Firefighter lacks motivation or confidence. Despite training in exit lighting requirements, the Firefighter, although admitting that he saw the violations, had to be told to write these Fire Prevention notices on three separate occasions. (Stating “lack of motivation or confidence” is a conclusion statement!)
Remember your audience. Who is going to read this report? Your Battalion Chief and the Director of Training will. How familiar with the Probationary Firefighter’s behavior are these Chief Officers? Write as though someone other than the Probationary Firefighter will read your documentation.
Watch your grammar, spelling, and legibility. Avoid slang, jargon, and swearing. Your credibility and professionalism are at stake. Be a good role model.
Speak to performance, not personality. Criticize the act, not the person. Criticizing the person brings about defensiveness. While more difficult to do in written versus oral form, the “impersonal” style of documentation relieves some of the stress.
Example: Rather than write, “You did a poor job of handling the last call…” try “Firefighter Jones did not handle the call according to DHS and LBFD standard.”
Use lists, if appropriate. The use of a “list” will sometimes save time and space.
Example (“Emergency Duties and Training—Tools & Equipment”): Firefighter Jones, when presenting a station drill on the nomenclature and use of the chain saw, failed to note correctly the following: (a) criteria for changing the chain, (b) the proper fuel mixture, and (c) how to trouble-shoot the automatic oiler.
What has been tried? How did it work? What will you try next? Do you have a plan, or idea? Document the results.
Use quantification whenever possible. Quantification or the documentation of a standard that is familiar to every reader adds clarity to the documentation.
Example (“Emergency Duties and Training—Safety Consciousness”): It took Firefighter Jarrett, during a station drill, three tries to dawn successfully her SCBA in accordance with the Department’s grading sheet.
Do not predict. Avoid statements such as “I am sure that Firefighter Jones will, with a little more effort, be able to master the chainsaw” or “Firefighter Jones’ skills will no doubt improve as the weeks go by.” Rather than make statements of this nature, the Captain should write what the behavior should produce; i.e., “When Firefighter Jones can complete reports of this nature within 30 minutes or less, he will be performing at an acceptable level.” Predictions set up false and inaccurate expectations and goals.
Writing narratives should be no mystery to any Captain or Battalion Chief. If an officer can write an acceptable report, he/she can write an evaluation narrative. A way to keep documentation of this type in perspective is to write as though you are telling a story to a close friend or co-worker who was not present when the behavior was observed. Would you include all the details, or talk in generalities? When in doubt, read again what you have written and then ask yourself if you would really know what happened from what you have written. Another excellent approach is to have another Captain read your narrative. Do they have any questions? If so, your documentation may need some additional work.
The Captain and Probationary Firefighter’s discussion of the evaluation is an extremely important aspect of the Probationary Firefighter’s career development. Merely completing the evaluation and having the Probationary Firefighter sign it will not achieve the objectives of a proper evaluation. Failing to hold a discussion serves to destroy the practical aspect of the evaluation system.
To preserve this practical aspect and put it to use, the performance evaluation must be understood and accepted by the Probationary Firefighter. This does not mean the Probationary Firefighter has to be in agreement with the entire evaluation. It must be the basis for plans to help the Probationary Firefighter improve performance as needed. Give the Probationary Firefighter recognition for strong points and also call attention to weak areas. The result is a better understanding between the Captain and the Probationary Firefighter. The Captain should allow ample time to discuss the evaluation with the Probationary Firefighter. Discussions should be held where privacy can be maintained with little or no interruptions. These discussions
should be a “two-way conversation.” Probationary Firefighters should be encouraged to express how he/she feels.
The Captain should listen to what the Probationary Firefighter has to say and not show disapproval when he/she does respond to the evaluation. The Captain should get across the idea that performance is being discussed and not a defense of the evaluation. The Probationary Firefighter should be encouraged to analyze his/her own deficiencies and motivations. The Probationary Firefighter should be allowed written comments or a rebuttal.
Once a discussion has been completed, the Captain should ensure that the Probationary Firefighter signs the evaluation.