Marine Services Competency Assessment Toolkit

Steps in planning and conducting the competency assessment planning process

As described in the Aviation Competency Assessment Toolkit, the planning and design phase for creating a customized competency assessment plan should include the following steps:

1. Determine who will manage the planning and implementation, and who will be part of the planning team.
   
   
2. Identify the competency framework performance criteria that fit the current job/position in your country/organization. Strongly consider making changes to the job/position description to account for the international competency standards. Add any additional criteria, if necessary, based on local procedures and conditions. The result of this step is the first step in an Assessment Matrix as displayed in the first column of Toolkit 1 of this assessment toolkit website.

3. Determine the tools that will be used to assess each of the performance criteria. An example of this is provided in the excerpt from WMO-No. 1205 below.
   

4. Provide guidance on how the tools will be applied, including a checklist of evidence of competency. This is particularly important for an assessment tools new to the organisation.

5. Determine how you will manage the process:

   • Who will be the assessor(s)? Will they require training? (See #6)
     

   • Will the assessment have regional or station groupings?

   • Who will review/moderate the assessor reports? Will results be reported by each individual assessed or grouped by station or region?
   • Determine the competency improvement methods you will use to help those assessed as "not yet competent."
     

6. Seek sources of competency assessment training if you decide this is required to better prepare assessors. This toolkit is a starting point, but also see WMO-No. 1205. WMO or your national organization may be able to recommend other sources.
   

7. Plan a schedule for when the assessment will be conducted and how how much time will be allotted. Determine the number of sites where assessment should take place and the number of persons that must be assessed.
   

8. Communicate this plan with upper management or government administration as appropriate.

9. Decide how records of results will be kept.
   
   
10. Conduct the assessment.
    
    
11. Collect and report results.
    
    
12. Implement competency improvement methods for those staff members that require it.
    

WMO-No. 1205 offers much more guidance on the assessment process in section 2.2. It is available in English, French, Spanish, Russian, Chinese, and Arabic.

Assessment Matrix example for aviation forecaster competencies from WMO-No. 1205. A descriptive text excerpt is provided below the table.

Text Excerpt:

"Once the assessment tools have been established, it is good practice to map them back to the performance criteria. Table 6 provides an example of assessment tools used in competency assessments of Aeronautical Meteorological Forecasters (AMF) in Australia, and how they map back to the elements of competency.

Ideally, each performance criterion should be assessed at least twice to demonstrate that the forecaster can apply the skills and knowledge in a variety of contexts. More than three assessment tasks for one performance criterion may be considered redundant. In a small organization, using two assessment tools might be difficult unless one of them requires less effort, such as existing or extended supervisor’s reports.

Evidence is defined as direct (D), indirect (I), or third-party (T); x denotes the main evidence gathering activity, while x_s denotes a supplementary activity that may be required as a backup if sufficient evidence is not collected."

- - - - -

Some additional details on assessment planning from WMO-No. 1205 and from the Aviation Competency Assessment Toolkit are provided below:

• Determine who must be assessed. Previously assessed staff members deemed competent may need assessment only after several years, depending on your assessment plan.
  

• Determine which competencies and performance criteria will be assessed.

• Determine how long the competency assessment will remain valid. This may differ depending on organisational services and responsibilities and changes that occur to these.
  

• Address logistical considerations (such as rostering or relief of duties).

• Determine how and what you will communicate about the assessment within the organisation.
  

• Create clear instructions for the assessment process, both assessors and those being assessed.

• Decide what reasonable adjustments may be applied in special situations (disabilities, work location, operational tools available to the person assessed, travel required and stresses and costs this entails, language and culture of the person assessed). See WMO-No. 1205, section 2.2.5 for more details.
  

• Determine the feedback and reporting mechanisms to staff and management.

• Determine what actions should be taken in the event of a "not yet competent" result.

• Decide what remedial actions should be taken and the reassessment arrangements for a "not yet competent" staff member.

Recall that a competency framework outlines the minimum skills, knowledge and behaviours personnel need to demonstrate to perform the job function. Once competency has been demonstrated, it needs to be maintained over time and, ideally, organizations should implement strategies to enhance performance once the minimum requirement is met.

The process of maintaining and developing competency should be documented. For example, how often competency reassessment will occur, what assessment tools will be used, how they will vary from the initial assessment, how and when the organization will provide training to develop skills and knowledge and how this will be monitored.