Audit of the availability of learning services in English and in French at the Canada School of Public Service - Final report

General finding

• School employees have a positive attitude toward official languages; however, at the time of the audit, the School’s specific obligations under section 37 of the Act were not always well understood and were not incorporated into its governance. Official languages must be taken into account in all School activities. They are a shared responsibility and do not fall under the sole responsibility of a specific group.
  • To improve its official languages compliance over the long term, the School must put processes in place that take federal employees’ official language needs into account when learning events and activities are designed, planned, promoted and delivered, both in person and on-line, and when quality assurance of those events and activities is performed.

Governance structure

• Several key official languages governance tools, such as policies, directives and quality assurance mechanisms, did not exist at the time of the audit. Furthermore, the Official Languages Action Plan 2014–2016 did not specifically take into account the School’s obligations under section 37 of the Act.
  • It is important that the School revise existing framework documents and develop new ones so that its obligations under section 37 of the Act are taken into account. It must then communicate them to all staff promptly and effectively.

Designing learning activities

• The School’s practices for consulting other federal institutions and measuring demand did not take official languages into account. Two elements emerged, in particular:
  1. Some regions did not measure demand for in-person courses or events.
  2. It was not possible to demonstrate through the audit that learning needs in English and in French were systematically included in formal discussions with other federal institutions in the regions or in the National Capital Region.
     • It is important for the School to implement processes and fully understand the needs of federal employees throughout Canada with regard to learning in the official language of their choice, to adapt its courses accordingly and to be proactive in adapting its courses.
• Since 2015–2016, the School has required pilot courses to be in both official languages.
  • To ensure long-term success, this requirement must be set out in the School’s governance.
• In the design stage, validating the content to ensure equal quality in both official languages was done during the editing and translation process and during a high-level committee review. However, validation was primarily the designer’s responsibility, and there was not enough support for these designers.
  • The School must ensure that all of its designers are using the same tools consistently: for example, by making its guide for instructional designers mandatory.
• On-line courses purchased from third parties following a call for tenders were withdrawn in 2016 because they were not of equal quality in both official languages. Until then, the School had assumed that the bidders guaranteed quality. The School acknowledged this shortcoming but at the time of the audit had yet to put any formal mechanisms in place to evaluate the quality of its products.
  • The School must put formal mechanisms in place to validate language quality and ensure equivalence in both official languages of all on-line content before making it available to learners.

Promotion and availability of learning activities

• The audit found only minor concerns regarding external promotion and communication in both official languages. In terms of service in the official language of the federal employee’s choice, each region had positions that were designated as bilingual; however, training of School employees in this area was recommended rather than required.
  • School employees must be trained systematically on serving federal employees in the official language of their choice.
• In-person courses are now posted for longer periods. Whether they are given depends on how many people register. However, at the time of the audit, courses were not systematically scheduled in both official languages in the regions, despite the direction given by senior management. There were no learning activities in the official language of the linguistic minority on the schedule in Toronto or in Québec City. Moreover, even though course cancellation thresholds do not necessarily need to be identical in all circumstances, they were not applied systematically across the country, and the School rarely offered alternatives that considered the travel costs that must be paid by the learners and their institutions.
  • The School must put mechanisms in place to ensure that scheduling and availability of learning activities are based on clearly defined criteria so that they are delivered consistently across Canada. The School would also do well to be proactive in offering alternative solutions to learners.
• Events have become an increasingly important delivery method for the School. All national events, which are webcast, have included simultaneous interpretation since 2016. However, this was not specified in any governance instruments at the time of the audit.
  • It is important that the School’s governance instruments include official languages rules and standards for its events across the country.

Quality assurance and evaluation

• The School did not have a way to ensure or evaluate the quality of both official languages in its learning products after they are delivered.
  • To ensure full compliance and improve flexibility in its service offering, it is important for the School to set up an independent function for ensuring and evaluating equal quality, to survey learners on questions of quality and timely availability in the official language of their choice, and to develop an internal process for handling complaints.
• Executive performance agreements contained a statement on language of work, but executives were not evaluated on meeting the obligations under section 37 of the Act.
  • Including official languages in all employees’ performance evaluations—in a way that is appropriate to the nature and scope of their duties—is a key ingredient of long-term official languages success.

Recommendations

The Interim Commissioner of Official Languages has made five recommendations to help the School address the shortcomings listed above. The recommendations are listed in Appendix B.

The Interim Commissioner is satisfied with the measures and timelines proposed in the School’s action plan regarding the implementation of the five recommendations. Appendix B also contains the School’s comments and action plan for each recommendation, as well as the Interim Commissioner’s comments. Within 18 to 24 months after the publication of the final audit report, the Office of the Commissioner of Official Languages will follow up on whether the School has implemented the recommendations.

Objectives and criteria

The audit’s objectives and criteria (Appendix A) are reflected in the main sections of this report: governance structure; understanding learning needs in English and French and designing learning activities; promotion and availability of learning activities of equal quality in English and French; and quality assurance and evaluation.

This audit did not examine any measures implemented by the School to promote a bilingual work environment for its own employees^{Footnote 4}, any actions taken by the School to promote the development and vitality of official language minority communities, or any services offered by the School that are related to learning a second language. Although these issues are of great importance, they were beyond the scope of this audit.

Audit process

An initial meeting was held on April 18, 2016, to present the audit objectives and criteria to members of the School’s senior management. The School provided written documentation to show how it was meeting the objectives and criteria. This information was analyzed between May and August 2016. A total of 45 interviews were then conducted between August and December 2016 that brought together instructional designers, customer service staff, resource people in corporate roles, learning managers in the regions and many executives from the National Capital Region and other parts of Canada.

Audit team

• Marcel Fallu, Senior Auditor
• Nicole Harb, Auditor
• Johanne Morin, Assistant Director, Performance Measurement
• Jean Marleau, Director, Performance Measurement

Measuring demand

At the time of the audit, the School was in the process of changing the way it measures demand and determines required resources. The School stated that after the transition to the new funding model, it would develop tools to evaluate demand for learning services in English and French. The School explained that so far, it has had few reliable sources of demographic data in order to determine the breakdown of federal employees by official language preference, region and occupational category. The audit found that the School has yet to conduct an analysis that focuses on the availability of courses in English and French in order to determine, for planning purposes, whether there are any disparities between functional communities in terms of effective access to learning activities in English or French. The audit also found that the School understood the importance of taking official languages into account when measuring demand; however, there were some difficulties in terms of available data. The School needs to take the initiative and strive to use all demographic data related to official language preference: from statistics to information from central agencies. For example, an in-depth analysis of data from the federal Public Service Employee Survey on training in the official language of the employee’s choice could stimulate discussions with institutions about their employees’ learning needs.

Courses in the official language of the majority

At the time of the audit, it seemed that the current perception among interviewees is that federal employees tend to take training courses in the majority language rather than their preferred official language. According to the interviewees who shared this point of view, two of the factors involved in this choice were the work environment and the terminology used in the work environment.

The audit found that the School had not examined this issue to gain a better understanding of it, even though it has a major impact on the learning activities the School offers in both official languages. Choosing to participate in a learning activity in one’s second official language is a valid choice and can improve the learner’s knowledge of technical and specialized vocabulary in both official languages. It can also help the learner retain his or her second language skills. This kind of individual and informed choice is protected by the Act. However, in practice, it is not always possible to exercise that choice. Data obtained from the School revealed that in certain regions designated as bilingual for language-of-work purposes, as well as in other regions, there were very few or no learning activities in the official language of the linguistic minority. A learner’s decision to participate in learning activities in the official language of the majority may be the result of dissatisfaction, inaccessibility of some courses, or even a lack of awareness of his or her rights in this regard. The School needs to use an established methodology to conduct an in-depth study of why federal employees participate in learning activities in their second official language so that it can make a distinction between situations where it is an informed choice (protected by the Act) and those where the choice is due to limitations in the availability of services.

Designers and the translation process

The audit found that, in general, validating the content of learning activities for equal quality in both official languages during the design stage depends primarily on the designer and on the translation process.

The School has teams of instructional designers who create a significant part of the learning activities for federal employees. At the time of the audit, many designer positions were bilingual with high linguistic profiles (CBC/CBC or CCC/CCC). The courses are created in the language of the designers’ choice. According to several interviewees, translation is done at the end of the design process because of tight deadlines and because it is more efficient.

Copy editing and translation by language professionals are strategically important to ensure that the School meets its official languages obligations. Despite a relatively high level of satisfaction with these services among the interviewees, some thought that there had been a decline in resources, that there had been quality issues because of a shortage of translators familiar with the terminology, or that there had been too much learning material to be processed within tight deadlines during the transformation. The Translation Bureau assigns a few editors and translators to the School as part of an agreement. The School would do well to expand the scope of this agreement so that it could benefit from more resources specialized in the terminology used for learning activities.

Once the translation is finished, the designers play a key role in validating any specialized terminology in the other official language. The audit found that the designers are ultimately responsible for a detailed review of the content to ensure equal quality in both official languages. However, differences were observed in the way various teams work on this upstream quality assurance. In some teams, the designer who created the course examines the content in the other official language, and in other teams, a different designer examines the content of his or her colleague. According to the interviewees, there were similarities in several stages of the design of on-line courses and classroom courses (designed in one language, then edited and translated). The interviews also revealed that the process for creating on-line courses seemed to be more consistent than the one for classroom courses. It should be mentioned that at the time of the audit, one of the teams specialized in designing on-line courses and provided support to the other design teams.

Several instructional designers on various teams were former language teachers, which makes them more sensitive to the importance of language quality. However, because of the normal course of organizational renewal, there is no guarantee that this level of expertise will be maintained in future. It is therefore important to formalize the quality assurance process to ensure equal quality of content in both official languages during the design stage.

Considering the needs of both official language groups

The audit found that the needs of both official language groups had been taken into account during the design stage and that content and expertise from both groups had been used. However, it was difficult to determine whether all designers had received support regarding these issues. Some were satisfied with the fact that the documents are edited and translated and that the content is reviewed by SCIC (see section on SCIC review, below). It is important that the quality assurance mechanisms and procedures take the needs and expertise of both official language groups into full consideration. The School would also benefit from developing training for its new designers on equal quality of official languages and on appropriate information sources in both official languages.

Guide for instructional designers

In 2015–2016, the Official Languages Committee developed a best practices guide for its staff called the Guide to Official Languages Best Practices for Instructional Designers. An official languages job aid was also made available to School staff in the fall of 2016. The Interim Commissioner of Official Languages commends the School on its initiative in creating the guide, which provides clear descriptions of official languages requirements in the context of the work of the designers, whose work is a crucial part of the School’s capacity to meet its language obligations. She encourages the School to share the guide with other federal institutions that provide learning activities to their staff so that they can benefit from these official languages best practices.

Unfortunately, most of the School’s instructional designers were not aware of the Committee’s guide at the time of the audit. With few exceptions, the only interviewees who mentioned the guide were the ones who worked on it. The guide had been distributed by e-mail to employees and made available on-line along with other tools for instructional designers. The School needs to take more active measures (mention the guide during key meetings or hand out hard copies) in addition to the passive ones (e-mail and intranet) in order to raise awareness about this high-quality tool that has been developed in-house. This issue is similar to the one related to communication mentioned in the previous section on Governance Structure. That said, the guide and the official languages job aid are not considered to be official procedures, directives or instructions.

External learning content, references and copyright

The audit found that many interviewees had problems using external content and references in learning activities. At the time of the audit, the School did not have a commonly accepted definition for “equivalent content” in English and French for this type of external content, such as e-books made available to learners or videos used to supplement the course material. These resources are rarely available in both official languages, and those that are available are rarely of equal quality in English and French. For on-line courses, some units seem to have taken a “one for one” approach as a precautionary measure. With regard to references, this means that each reference must be available in both official languages; otherwise it is not used. Other interviewees noted that traditional library services do not necessarily work this way and do not think that this rule is necessary. Many resources are available only in English and others only in French.

In essence, it is acceptable to require that content from external sources be equivalent, but the School needs to define the meaning of “equivalent” and give its employees the tools they need to understand the parameters involved. Unless the references are identical in English and French, equal quality must be ensured by having references that enable learners to achieve the same educational goals in both official languages.

The question of copyright is also at issue. At the time of the audit, the School did not have any exemptions to certain provisions of copyright law, which are usually made available to educational institutions and research facilities. Therefore, designers or instructors who wish to use external content in learning products, including any translations thereof, must obtain permission from the copyright owner. The School is encouraged to conduct an in-depth review of the copyright issue in order to determine the impact of the current situation on the School’s ability to use learning content of equal quality in both official languages.

Pilot courses

During the process of designing learning activities (in class or on-line), design teams test the content on a focus group of experts. The objective of this exercise is to ensure that learning objectives are met and to make any adjustments before the course is launched. After the Office of the Commissioner of Official Languages investigated a complaint in 2015, the School changed its position— which was to deliver the pilot courses in the designer's preferred official language—and now requires all pilot courses to be given in both official languages. The School’s senior management sent an e-mail to staff in February 2016 to inform them of the decision.

At the time of the audit, only some of the teams had had the opportunity to deliver pilot courses in both official languages, but most of the staff were aware of the School’s new position. However, discussions with interviewees during the audit showed that people’s perceptions of the new position, whether positive or negative, varied from one individual to the next, which could affect consistent application. For example, some interviewees said that a pilot course could be given in French at a different stage of the design process than the pilot course in English, even though the School is required to communicate equally with both official language groups. The School therefore needs to provide a clear explanation of its new position.

Some interviewees also said that the School’s new position had made tight design deadlines more problematic. Major changes are often made at the end of a pilot course, which then results in additional editing and translation.

The School has an institutional responsibility to make resources available so that teams can design and deliver pilot courses in both official languages. The School’s new position on pilot courses must be included in governance instruments so that it continues to be applied in the future. At the time of the audit, the requirement for pilot courses to be in both official languages was mentioned only in the official languages job aid. As stated above, this is not considered to be a formal procedure.

Having pilot courses in both official languages and using focus groups that reflect regional, ethnocultural and gender diversity, as well as a variety of expertise from both language communities can only enrich the School’s design process, even if it inevitably requires more resources than previously, when pilot courses were given in only one language.

SCIC review

SCIC plays a key role in validating learning activities at the design stage. An analysis of the School’s documentation clearly showed that the committee was taking official languages into account at the time of the audit. The Interim Commissioner of Official Languages would like to highlight the fact that SCIC’s high-level review included questions on official languages, which is a best practice.

Launching learning activities

The launch of a learning activity is the finish line in the design process. Interviewees who shared their opinions about the availability of on-line or classroom courses in both official languages all said that a course is not launched if the material is not finalized and available in both official languages. This requirement was not found in the governance instruments, however. It is therefore suggested that the School address this situation.

Customer service

In 2016, the School contracted Service Canada to handle initial telephone contact with its learning services clients. School employees are still responsible for responding to more complex questions. Because this was a completely new practice at the time of the audit and because this service was contracted to a federal institution subject to the Act, the audit did not examine it for compliance with the Act.

With respect to School employees, at the time of the audit there was a team responsible for liaising between federal institutions’ points of contact and the required training coordinators. Regional staff played a complementary role in this regard. Interviewees stated that they knew the rules for communication in the preferred official language of employees of other federal institutions. However, these rules were not set out in any of the School’s governance instruments.

Courses on active offer, on providing bilingual services and on language-of-work rights were included in the learning path the School recommended to its employees. However, during the audit it was not possible to establish whether the School had ensured that training on service to other federal employees in their preferred official language had been provided to each staff member in each region. Moreover, the audit did not turn up any documents that provided School staff with a clear explanation of their obligation to serve clients from federal institutions in the official language of their choice across the country. The School did produce a self-evaluation tool for language of work that was intended for all federal institutions; however, at the time of the audit, the tool did not differentiate between the obligations under section 37 of the Act and obligations for institutions that are not subject to section 37 requirements. The School needs to adapt this tool for its own staff and for employees of other federal institutions subject to section 37.

Regarding the capacity to serve learners in their preferred official language, the School had bilingual positions with BBB/BBB or CBC/CBC linguistic profiles in all regions. In addition, all regions outside of Quebec and Nunavut had at least one bilingual position with a CCC/CCC linguistic profile. However, it should be noted that at the time of the audit, several positions were vacant and some incumbents were on secondment; therefore, there was no clear indication that bilingual capacity was maintained at all times. Some interviewees stated that they were aware of situations in the regions where the current language skills of some School employees no longer corresponded to the results they had obtained during their second language evaluation several years earlier. These interviewees would like the School to be more proactive in helping its employees to maintain their second official language skills, especially in regions where the demand for services in French is lower.

Furthermore, because the Québec City office–which, at the time of the audit, had positions identified as “French essential”–also serves the Nunavut region, the School is encouraged to ensure that the linguistic profiles of these positions correspond to the actual duties of providing customer service to other federal employees in the region.

Availability of classroom courses

At the time of the audit, there were some key differences between the availability of classroom courses in the National Capital Region and in other regions. Differences were also noted between key individual courses and transition programs involving cohorts.

During the audit, interviewees in charge of individual courses said that in the National Capital Region, the current practice was to list the courses in both official languages on the GCcampus schedule for longer periods of time than was the case previously. At least one offering of any course was always posted on the schedule in both official languages. Actual availability depends on enrolment. In addition, interviewees said that because courses are free of charge to the learners under the new business model, the School is less dependent on the budget cycle of federal institutions.

The School said that it had added a learning request function to GCcampus that allows learners to enter details on where, when and in what official language they wish to take a given learning activity. According to the School, this function allows it to better plan for demand in both official languages when classroom courses are not available at a particular time.

At the time of the audit, accountability for planning courses in the regions—both individual and for cohorts—was left to the regional directors. Their teams were supported by the Capacity Planning Team at the national level, but they also used their own knowledge of regional realities to create course and event offerings. At the time of the interviews, the practices related to offering learning activities in both official languages varied widely from region to region and depended on the personal commitment of whoever was responsible for the activity.

Generally speaking, at the time of the audit, the School was not systematically scheduling every course in English and French in every region, which seemed to contradict the general guidelines provided by senior management. Some regions—specifically, Toronto and Québec City—did not offer any learning activities in the official language of the minority, despite the presence of a significant minority population. There was no availability or measurement of demand for learning activities in the minority language in either region (French in Toronto and English in Québec City). Interviewees stated that sometimes learning activities offered in the minority language were geared to the demographic weight of the linguistic minority in that region. This is a misinterpretation of the Act because section 37 guarantees an individual right that goes beyond regions designated as bilingual for language-of-work purposes. Leadership program courses in French were concentrated in Montréal and Québec City, with a few others in Moncton.

The audit found that there were no defined criteria across the country for cancelling courses with insufficient enrolment. The School needs to adopt cancellation standards that systematically take into account the specific circumstances of the various regions and language communities. The interviews revealed that, in cases where specific learning activities were unavailable in the official language of the minority, the School rarely offered alternative solutions that took into account the travel costs that had to be paid by the learners and their federal institutions.

Availability of local and national events in English and French

Events are an increasingly popular method of delivering learning activities. The School now organizes hundreds of them every year. National events, which are webcast, have been delivered with simultaneous interpretation since 2016, regardless of where the event takes place. The School also presents hundreds of local learning events on specific topics across the country.

At the time of the audit, the School had no formal written procedures that defined the circumstances or conditions under which events must have simultaneous interpretation, those under which events must have bilingual masters of ceremonies or speakers, and those under which events can be presented in only one official language. In short, there is no formal procedure for holding regional events in the official language of the minority. The School must therefore formalize its recent decision regarding simultaneous interpretation for webcast events, an improvement that needs to be established as a standard.

On-line courses and videos

Nearly all of the interviewees said that every non-interactive on-line course was available in English and French. According to a number of interviewees, changing several classroom courses to self-paced on-line courses (including several required training courses) has had a very positive effect for federal employees from the official language minority because it resolves the issue of accessibility. Interviewees also said that the videos developed for learning activities were always available in English and French. Several interviewees expressed the concern that it is important to ensure not only that videos are bilingual, but also that they reflect both language communities (e.g., by producing videos using English- and French-speaking specialists and by avoiding the use of subtitles). Discussions with interviewees involved in creating videos revealed that one of the School's priorities is making sure that learning products are accessible for people with functional limitations or disabilities. These aspects of the audit did not raise any concerns.

New approaches to participatory learning

The audit found that the School was just starting to introduce new approaches to on-line participation in its learning activities and that it had not yet made the switch to participatory production, which was thought to be inevitable by several interviewees. However, participatory approaches to on-line learning were being used more and more in the School's learning activities. According to the interviewees, on-line participatory activities took place in the language of the learning activity (e.g., a discussion forum related to a given course). When asked about learning activities using WebEX videoconferencing software, interviewees said that the practice was for discussions to take place in only one official language because testing the bilingual format of WebEX did not produce conclusive results. The School had no formal directives on this issue.

Several interviewees mentioned challenges related to discussion forums and blogs. During the audit, it was noted that there were no requirements or guidelines to ensure that on-line discussions were equivalent in both official languages. Interviewees had varying opinions on how to address this issue. Some felt that it did not make sense to separate English- and French-speaking learners into different threads, while others felt that separate environments would make learners more comfortable in taking full part in their preferred official language. Therefore, at the time of the audit, the School was still exploring official languages in the context of new collaborative methods. Some interviewees said that the School would very likely follow the current practices for GCconnex and GCpedia, which are platforms managed by the Treasury Board of Canada Secretariat. The School needs to take the initiative, however, and give its employees the tools they need to ensure that the spirit of the Act (i.e., substantive equality between both language communities) is respected when the letter of the law is not clear on the issue.

If learning activities are of equal quality in both official languages from the outset and course offerings are the same, then they can take place entirely in one language rather than switching between English and French. It is difficult to manage discussions in order to ensure that they are identical in both languages in real time. Equal quality does not necessary mean identical. In all cases, activities in which participants can use English and French at the same time appear to be problematic: it is difficult to set parameters for them, and the balance between English and French can easily shift in favour of one or the other. The practices that the School has put in place seem to reflect this and should therefore be maintained,with the exception of certain learning activities for senior management that pose a compliance issue, as mentioned above.

Complaints and feedback about official languages

At the time of the audit, federal employees had several channels­ in addition to course evaluations-through which they could provide feedback on service in their preferred official language: the School's website, its regional staff and its Client Contact Centre, as well as their institution's learning coordinator and the Office of the Commissioner (by filing a complaint). Depending on the channel, feedback was processed in different ways and through different officials with no overall coordination. For example, if a learner filed a complaint through the Office of the Commissioner, it was handled by the School's human resources team. If the learner raised the same concern with the School's regional staff, it was handled by the branch responsible for the regions. If the learner went through the School's website or Client Contact Centre, the feedback was processed in another way. Establishing a transparent and integrated procedure to address any official languages compliance issues, regardless of how they were communicated, would help provide an overall view to senior management, in particular.

Evaluation, audit and accountability

Because it is a small federal institution, the School does not conduct its own internal audits, which is in accordance with the Treasury Board’s internal audit policy instruments. However, the present audit did not find any evidence of how the School’s internal audits were coordinated to ensure that their objectives and criteria included or reflected official languages. At the time of the audit, the School was conducting only one program evaluation, but it was critically important because it was about the School’s strategic directions. The formative evaluation had been completed at the time of the audit, and the summative evaluation was in progress. Official languages were not taken into account in the evaluation’s objectives or in the issues that were evaluated, even when it talked about equitable access to learning. It was not possible during the audit to show that the evaluation team reported to senior management specifically on official languages.

Nevertheless, the School’s Integrated Plan took official languages into account in its objectives and deliverables, which is a good practice. However, during the audit it was not possible to prove, in concrete terms, that official languages were an integral part of accountability (e.g., reporting on the percentage of activities that were available and of equal quality in both official languages).

Interim Commissioner’s comments:

I am satisfied with the measures the School has proposed to address Recommendation 2. Although the wording of the first measure may seem vague, the relevance of all three measures proposed in response to this recommendation was demonstrated both in the information collection phase of the audit and in discussions with School officials following the submission of the action plan.

Interim Commissioner’s comments:

I am satisfied with the measures the School has proposed to address Recommendation 3.