On Monday, November 12, 2018, the New Orleans Civil Service Department sent test results from the recent Lieutenants Exam to the test takers. 74 NOPD Police Sergeants took the exam. Out of those 74, 51 passed the exam (69%) and 23 failed (31%). The highest score 94 and the lowest score was 20. The average score was 57 and 53 was the cut-off between pass and fail (the lowest passing score). The Lieutenants List can be found here.

Starting November 14, 2018, the Civil Service Department will begin having candidate feedback sessions with Ms. Bharati Belwalkar. Ms. Belwalkar is the Civil Service Department’s psychometrician who is dedicated to NOPD testing. The intent of this article is to try to give candidates a realistic idea of what to expect in that candidate feedback session.

First, the candidate feedback session is NOT an appeal of your score or a protest of any questions. Your score on the exam will NOT change as a result of the candidate feedback session. The purpose of the candidate feedback sessions is to help the candidate understand their areas of strength and improvement, and to better prepare for the next opportunity to take the exam.

You will NOT see the test questions, your answers, or how they were scored. You will be given information intended to help you focus your studies next time. If you know where your weakest areas were, then you will be able to prepare better next time.

The meeting, which will probably be recorded, will start off by reviewing the 3 main components of the exam: Part 1: In-Basket, Part 2: Structured Interview, and Part 3: Oral Presentation. All three of these components were equally weighted in calculating the candidate’s final score. You will see a graph that looks like this:

The graph shows each component of the exam, the highest score of each component, and the candidate’s score for each component. On the example above (not real test results), the candidate scored 67% on Part 1, 28% on Part 2, and 54% on Part 3. Clearly, the candidate did the best on Part 1 and the worst on Part 2. Ms. Belwalkar will discuss each component in terms of the candidate’s performance on the types of questions covered in it. If any of the raters had specific comments about the candidate’s answers, that information will be shared with the candidate.

Next, Ms. Belwalkar will go through the six competencies tested by the exam. Those competencies followed by the weight assigned to that competency are:

1. Demonstrating Department’s Values (12%)
2. Leadership and Supervisory Responsibility (26%)
3. Operational Effectiveness (25%)
4. Critical Thinking and Strategic Planning/Problem Solving and Decision Making (18%)
5. Communicating Orally and/or in Writing (9%)
6. Partnering with the Community (10%)

You will see another graph that looks like this:

Like the first graph, this graph shows the total percentage weight of each competency and the candidate’s percentage score for each competency. The example candidate profile indicates that the candidate demonstrated about 50% proficiency in every category. Because the competencies are weighted differently, it may be more important to strengthen up those areas, but it appears this candidate needs an equal amount of work across the board. Ms. Belwalkar will work with the candidate to make the information the most useful.

Finally, you will have an opportunity to ask any questions you may have. Remember, you will not have the opportunity to review your test answers, the scoring rubric, or the questions. I guess you can ask, but I would expect the answer to be that it can’t be shared. The reason for this is test security. Sometimes questions are re-used or are changed a little and then re-used. If the questions or answers to the questions were passed around, they would not be able to use any form of those questions again or risk the validity of the exam.

Each candidate’s answer was scored by 3 different raters who used a common rubric to score the exam. The 3 raters then discussed their ratings in order to eliminate error and reach a consensus score.

If you want to schedule a candidate review session, you can sign up for it here. If you have issues accessing this link, contact Ms. Belwalkar at 504-658-3508 or bbbelwalkar@nola.gov. Ms. Belwalkar is New Orleans Civil Service Personnel Administrator for the Test Development & Validation Division, working in the capacity of Senior Psychometrician.

We know that there are going to be at least 23 people who are not pleased with their test scores. Unfortunately, the New Orleans Civil Service Rules do not contain any process for appealing a score or how your exam was graded. When the exam contains a multiple-choice section, the Civil Service Department allows for protests of questions. In those circumstances, the protest is made before the exam is scored and the answer can be validated specifically by reviewing the appropriate text. In the event that a protest reveals a problem, the answer key can be changed or double-keyed to fix the problem. However, there was no multiple-choice section on this exam. The answers, and the scoring rubric, were developed in consultation with subject matter experts (SME’s) from the NOPD. The overall examination and its scoring rubric were reviewed by another group of NOPD SME’s to ensure accuracy and thoroughness. Since the types of questions and answers do not lend themselves to protest, none were allowed. Each exam was graded the same way by 3 different raters. The Civil Service Department has done there absolute best to make sure that everyone had the best chance to succeed.

I also want to take this opportunity to thank Commander Louis Dabdoub for donating his time to help FOP help NOPD officers prepare for the exam. I also want to thank Travers Mackel of WDSU for helping candidates prepare for the exam. As usual, the FOP is dedicated to providing whatever assistance it can to help FOP members succeed.