Cluster (Multi-Capability) Assessment

Cluster Assessment is about assessing multiple IT-CMF critical capabilities in a single assessment. Depending upon the assessment objectives, any combination of critical capabilities can be included in an assessment. Participants will be asked to assess each critical capability included in the scope. Each critical capability in the scope is evaluated using its capability maturity model.

In CAMP, critical capabilities are organized into clusters and domains. The clusters map to macro capabilities in IT-CMF. The cluster assessment is NOT the same as selecting cluster (i.e. macro capability) as a scope for the assessment. The CLUSTER in cluster assessment is in fact a collection of discrete critical capabilities selected for assessment.

By default, the people with the assigned role of system administrator or survey administrator can create and manage assessments. If you are allowed assessment administration, you should see Assessment Administration as one of the choices under Assessment menu.

If you don't see the above menu option, ask your administrator to add survey administrator role to your account.

Assessment administration process is fairly the same for all types of assessments. The scope makes one assessment different from another. As you walk through the process, anything unique to a particular assessment is highlighted.

All routine assessment administration activities can be performed from one place the application i.e. Assessment Administration page. Additional analytical and heat mapping functionality is available from Insight pages. To review the capability model of the items included in the scope, see capability model section.

Major steps in an assessment process:

1. Creating an Assessment, including defining scope, time periods, survey style etc.
2. Inviting participants, including setting up email notification templates
3. Tracking progress, including sending reminders
4. Rolling up and consolidating individual assessments
5. Adding an assessor assessment, if desired
6. Analyzing and creating charts and reports
7. Exporting data for additional analysis, charting and reporting

To create a new assessment, click on the “Add a New Assessment”. This will take you through couple of initial steps in creating a new assessment and setting up the scope. In the first dialog box, enter the name of the assessment, start and end dates for the assessment. If needed, you will be able to change the name and the dates once the assessment has been created. You may ignore the template option at this time.

After saving the assessment, a second dialog box will appear, asking you to select the scope of the assessment.

Navigate through the IT-CMF tree in the Scope Filter section and select the Critical Capability you would like to include in the assessment.

Select one or more Calendar entries in the Valuation Scope. Based on the selection, the participants will asked to enter the maturity score for each of the calendar entries selected. You may select current year to represent current state and another year in the future to represent target/desired state. You will be able to change the valuation scope i.e. pick other calendar entries at a later time.

Further down in the dialog box, select “Question-based Assessment” in the Assessment Level. The number of questions vary by the critical capability selected. The question-level assessment provides most depth in assessing a critical capability. If you want to assess at the capability building block level, in other words, one summary question per capability building block, select capability-level assessment from the drop-down box.

Note: IVI recommends question-level survey for critical capability assessments.

Finally, click “Save & Close” and now assessment with the scope of cluster assessment has been created.

Once the assessment has been created using the dialog box, it will appear under the Assessment List.

Select the assessment from the assessment to revise or to further configure the assessment.

If you don't see the assessment in the list, it means either you are not the original owner/administrator of the assessment or the status of the assessment is other than default “setup”. To see other assessments, use the filter to select assessments with different status than “setup” or show all.

Select BASIC from the information panel. It will display the core information about the assessment. The description in the “Name” field is what participants will see as the name of the assessment.

Start and/or End dates: Change the start and/or end date anytime to set the assessment period. Participants won't be able to enter or update scores after the end date.

Status: Set the status of the assessment. While the status is “setup” or “Dry Run”, you can change the scope. “Open” status indicates it is an active assessment and you are not allowed to change the scope in the middle of an active assessment. If you do need to change the scope, change the status to setup or dry run.

Survey Style: CAMP offers two styles for presenting questions and entering data for participants.

Choose between the two styles

Mode: Either leave it blank or select Maturity-based mode from the drop-down list. Multiple-choice option does not apply to assessments based on capability maturity models.

Keep the Active? and Visible? flags checked. There is no need to check other boxed on the page.

Select SCOPE from the information panel to add or change the scope of the assessment.

If you didn't add the scope during creation of the assessment, click NEW in the top of grid header. To edit a scope already included in the assessment, click EDIT for the scope you want to modify.

There are four components of the overall scope of the assessment.

Capability Scope For IT-CMF Cluster Capability Assessment, make sure you are using IVI in collection field. Select IT-CMF in the domain field. Pick the macro capability from the cluster drop-down list. It will update the list of critical capabilities in the capability set drop-down. Select the critical capability. For other capability assessments, navigate through appropriate collection, domain, cluster or directly pick a capability set from the dropdown list.

You will not be able to change the capability scope. To change, you would need to delete the scope and create a new entry.

Valuation Scope To add or update valuation scope, select one or more Calendar entries from the Snapshot dropdown field.

Other Fields Pick Question-level assessment in the Assessment Level field. If you want to ask participants about the relative importance of various questions or capabilities, mark “Include Importance” checkbox.

Repeat the above steps to add another capability to the scope. You should see multiple entries in the scope, as shown in a example.

Select DOCUMENTS from the information panel to upload new documents or review available documents.

When adding a new document, select a category and assessment scope from the drop-down lists. Either upload the document or enter the link to the document available at some other site.

CAMP includes number of standard templates for communication with participants. You can create assessment-specific templates. Similar to mail-merge in Word documents, these templates can include “placeholders” for personalized addressing.

Select NOTIFICATIONS from the information panel to see the list of available templates and to create/edit assessment-specific templates.

Even if you are planning to use the standard, system-defined templates, you must review and save them for the assessment.

When you try to edit any of these templates, for the first time, it will pre-populate them with the system-defined template. Make any changes and save the template.

The template editor is a rich-text WYSIWYG HTML editor. You can visual design the content and layout using the formatting buttons. You can review the HTML code and extend it with any valid HTML statements and properties.

To include placeholders for user name, organization name and many assessment related data fields, click on the field chooser (last icon in the menu header of the editor) and select the field and it will be placed in the template at the point of the cursor.

Select PARTICIPANTS from the information panel. Next panel shows the list of the participants with their status.

Click Enter Email Addresses to Invite to add participants using their email addresses. In the open text field, enter email addresses separated by coma (,) or by semi-colon(;).

System does not send an email invitation as soon as you add a participants.

If you no longer want the participant's response included in the analysis, you can edit the participant's record by setting Active? to false.

As an administrator, you are always an Assessor. You may also assign someone else as an Assessor.

Select SEND NOTIFICATION EMAIL from the information panel. It will list all participants in the next panel. Check the people you want to send the invitation and then click “Send Invitation Email” and emails using the invitation template will be sent immediately.

You can always re-send an invitation, if necessary.

As participants start and complete surveys, the status is updated. To know who has started, completed or in progress, select PARTICIPANTS from the information panel. The next panels shows the list of participants with their status. Start Date is the date the participant started. Completion Date is the date when the survey was completed. You can also see % completed at any given time.

If you want to remind participants who have not started or not yet completed, select SEND NOTIFICATION EMAIL from the information panel to send reminder email to specific participants.

Beyond tracking individual progress, you can also check statistical analysis i.e. min/max, standard deviation across the responses. Select ASSESSMENT SUMMARY from the information to see the statistical summary for maturity score or importance.

Anytime during the process or at the end of the assessments, you can consolidate and summarize individual assessments. A consolidated scores is a weighted average of all individual scores. The maturity scores for the questions are rolled up to Capability Building Block (CBB) and Capability category and then eventually to a Critical Capability.

Rollup is required to compute the consolidated maturity score based on all individual assessments.

Select ROLLUP from the information panel to start the consolidation and roll up. The rollup score for each critical capability in the scope are displayed in the next panel.

To check out individual and consolidated scores, select ASSESSMENT DETAILS from the information panel. To see the details of a specific critical capability in the scope, select the critical capability from the drop down list. The consolidating scoring for each question is listed as Consolidated Assessment.

Use the Field Chooser to see additional information. Simply drag and drop items from field chooser list into the header row of the table.

This is an optional step. You may want to add an assessor assessment if you or assigned assessor would like to add a perspective based on the interviews or additional information collected during the process.

There are two ways to add an assessor assessment.

1. Using one of the assessment as a start
   1. From Assessment Administration, select ASSESSMENT DETAILS. Select the critical capability from the drop-down list to see individual and consolidated assessments. Select an assessment by checking the radio button next to individual or consolidated assessments and then use Promote as an Assessor Assessment
   2. Promotion creates assessor records which can be updated. It redirects to MY ASSESSMENT page with the newly created assessor assessment. Edit the scores and add comments. In the end, calculate maturity.
   3. If you are an administrator but not an assigned assessor, promoting an existing assessment is the only way to create and edit assessor assessment.
2. Entering an assessment as an assessor
   1. From main Assessment menu, select MY ASSESSMENT, select one of the critical capability in a assessment from the list available for Assessor Assessments
   2. In the spreadsheet-style layout, enter the maturity scores in the box, and comments by clicking the comment bubble. In the end, calculate maturity.
   3. This option is available to anyone with an assigned assessor role.

Only one assessor record is allowed. If an assessor assessment already exists, the system will ask if you would like to replace the existing record.

For a cluster assessment, you can analyze and develop detailed charts for each critical capability in the cluster. You can also analyze maturity scores across the critical capabilities included in the assessment.

CAMP offers three places to review assessment data.

Select ANALYTICS DASHBOARD from the information panel. Next panel includes the options to select the charts and layout.

Click the SELECT ITEM to open up the list of charts available. Select any number of charts. These charts will be displayed in the panel.

If a chart displays “Data is not available”, check filter settings.

Each chart has filter, cloning and image copy icons when you mouse over or touch the chart header area. Click on the filter to open up the filter options in a dialog box.

In most of the charts, in addition to chart formatting options, you can select

• Assessment Scope - Critical capabilities included in the assessment
• Valuation Scope - Maturity snapshots included in the assessment
• Assessment Type - Consolidated Assessment or Assessor Assessment
• Chart Detail Level - Capability Category, CBB or Question-level.

Assessment Maturity Distribution chart plots the consolidated scores and range across the individual assessments. Assessment Maturity Roadmap charts plots the current and target maturity levels across CBBs.

Select ANALYTICS from INSIGHT menu. The charts under Capability Model Analysis allow you to visualize and analyze the latest maturity data for a capability. The charts under Assessment Insight allow you to analyze the data for a particular critical capability in an assessment.

After an Assessment, you may be interested in the following charts for the analysis.

• Capability Maturity Comparison (by Snapshot) under Capability Model Analysis
  • It displays maturity data at CBB level
  • Select IT-CMF or critical capabilities in the scope filter, one or more snapshots in the valuation scope, polar or other chart types
• Various charts under Assessment Insight
  • Select the assessment and select relevant filter settings to display the chart and data

If you want to export the chart image, select the file type and choose the option to either save the file or open in a new window.

The charts available in the Management Dashboard are similar to charts in Analytics and Assessment Administration. Select the charts, select and save the filter settings for dashboard you want to see on the home page or management dashboard.

To export assessment data, select EXPORT ASSESSMENT DETAILS from the information panel of the Assessment Administration page.

The exported file (Excel format) include both summary and detailed assessment data. Assessment User Details and Assessment Rollup Summary worksheets can be used for further charting and analysis.

To generate a report, select REPORT from the information panel. Use the filter to select the kind of data to include in the report. The report can be saved or viewed on-line in various formats.

• Assessment Mode - CAMP provides two type of assessments. Assessments using a capability maturity model or a simple multiple-choice questions. IT-CMF assessments are maturity-based assessments.
• Survey Style - Preference for layout of assessment questions.
• Assessment Scope - The critical capabilities included in an assessment are viewed as assessment scope. A cluster assessment includes more than one critical capability in the scope.
• Valuation Scope - The capabilities are assessed as of a given timeframe. Snapshots (time periods) included in the assessment provide a mechanism to evaluate capabilities across one or more time periods.
• Roll up Assessments - The question-level maturity scores are “rolled up” to higher levels e.g. capability building blocks, category. The individual assessments are also consolidated as average scores.