Observations 2018–2019
The Office of the Commissioner of Official Languages conducts observations on services to the public in order to evaluate the strengths and weaknesses of federal institutions in terms of their various obligations under the Official Languages Act (Part IV of the Act).
Results of observations of service to the public
Results of observations of service to the public
| Federal institutions | In person: Visual active offer (%) |
In person: Active offer (%) |
In person: Availability of service (%) |
By telephone: Active offer (%) |
By telephone: Availability of service (%) |
By email and e-form: Availability of serviceFootnote 1 (%) |
By email and e-form: Response timeFootnote 2 (%) |
Content published on the Internet: Availability of informationFootnote 3 (%) |
|---|---|---|---|---|---|---|---|---|
| Canadian Food Inspection Agency | - | - | - | 100 | 100 | 100 | 71 | Footnote 4 |
| Canadian Institutes of Health Research | - | - | - | 100 | 98 | 100 | 84 | 67 |
| Health Canada | - | - | - | 100 | 95 | 99 | 93 | 89 |
| National Defence | 98 | 40 | 80 | 100 | 93 | 93 | 20 | 61 |
| National Research Council Canada | 100 | 36 | 88 | 100 | 95 | 93 | 64 | Footnote 4 |
| Natural Sciences and Engineering Research Council of Canada | 80 | 46 | 91 | 100 | 98 | 95 | 72 | 50 |
| Public Health Agency of Canada | - | - | - | 95 | 95 | 68 | 74 | 88 |
| Social Sciences and Humanities Research Council of Canada | - | - | - | 98 | 100 | 100 | 63 | 67 |
| Statistics Canada | - | - | - | 100 | 100 | 100 | 69 | 86 |
Observations of service in person and by telephone
Observations of service in person and by telephone
Background
The observations of service to the public conducted by the Office of the Commissioner of Official Languages (Office of the Commissioner) are one of many tools used to measure the performance of federal institutions with respect to Part IV of the Official Languages Act, which deals with service to the public. The Office of the Commissioner observed three types of service provided by the institutions: service in-person, by telephone and over the Internet. This tab describes the methodology used for in-person and telephone observations.
The Office of the Commissioner is supported by Statistics Canada in its methodological approach, particularly with respect to sampling, calculations and validation of results. It is important to note that the results provide an indication of an institution’s performance at the specific time the observations take place. They do not represent the probability of obtaining service in the official language of your choice.
Definitions
In-person observations
Observations of service in person involve making anonymous visits to a federal institution’s bilingual points of service to assess its capability to serve the public in the official language of the linguistic minority. This includes observations of service in English in Quebec and service in French outside of Quebec.
The evaluation is based on the following three criteria:
Visual active offer
The observer indicates (“yes” or “no”) whether bilingual services are offered at the point of service. This offer is provided through the following visual elements:
- bilingual signage outside
- bilingual signage inside
- presence of “English/Français” pictogram
- display of pamphlets, forms or documents in both official languages
The observer may indicate “yes” even if not all of the elements are present. For example, if the “English/Français” pictogram is not visible, but most of the documents and signs at the point of service are in both official languages, the observer will indicate that there is a bilingual visual active offer.
Active offer by staff
The observer indicates (“yes” or “no”) whether initial contact with an employee at the point of service is in both official languages, through the use of the “Hello, bonjour” greeting, a phrase such as “Next, suivant” or a similar phrase.
Availability of service in the official language of the linguistic minority
The observer indicates (“yes” or “no”) whether service is received in the official language of the linguistic minority at the point of service.
Telephone observations
Observations of service by telephone involve making calls to contact numbers that the institution provides to the public. When the telephone number is for a specific physical office, the same approach is used as for in-person observations with respect to the official language of the linguistic minority. If there is only one telephone number for the entire country, the same number of observations is made in English and French.
The evaluation is based on the following two criteria:
Active offer by telephone
Several factors are taken into account when evaluating active offer by telephone. For example, if the institution has a separate telephone number for English and French, the active offer is implicit. If the voicemail system’s main menu lets callers choose the official language in which they wish to proceed, the active offer is explicit. If there is no choice offered, the active offer is evaluated based on the first verbal contact with an agent.
In the latter case, the observer indicates (“yes” or “no”) whether the institution’s first point of contact answers the telephone call in both official languages by using a bilingual greeting, such as “Hello, bonjour,” or by announcing the name of the institution in both languages, for example, “Canada Revenue Agency, Agence du revenu du Canada.” Greetings like this make it clear to callers that service is available in the official language of their choice.
Availability of service by telephone in the official language of the linguistic minority
The observer indicates (“yes” or “no”) whether service is received in the official language of the linguistic minority.
Methodology
The methodology is the same for observations in person and by telephone. It involves making a number of anonymous observations at a representative sample of all the bilingual points of service of the institution being assessed. At the beginning of observation cycle, the Commissioner asks the institutions that will be observed to provide a list of all of their bilingual points of service that are open to the public without an appointment. The list is sent to Statistics Canada to establish a sample. One or more observations of the points of service in the sample are made over a defined period of time. The results provide an indication of the availability of service in the official language of the linguistic minority.
Results
The main objective of in-person and telephone observations is to obtain statistically valid overall results for each of the observation criteria. Unless otherwise stated, the observations do not generate statistically valid results by point of service.
Results quality indicator
When Statistics Canada calculates the observation results, it assigns a quality indicator to each one. The indicator establishes the quality of the sample that was subject to observations.
- standard deviation below 4% (margin of error of less than 8%, 19 times out of 20)
- standard deviation between 4% and 8% (margin of error between 8% and 16%, 19 times out of 20)
- standard deviation between 8% and 12% (margin of error between 16% and 24%, 19 times out of 20)
Statistics Canada considers the quality indicators A, B and C to be appropriate, given the objective of the Office of the Commissioner’s observations. This statistical survey is not an opinion survey, nor is it intended to predict future results.
Comparison of results
Observation results are snapshots of service availability at various points of service at a specific time. Unless otherwise stated, they cannot be compared from year to year or serve to determine progress over time. The ensuing results would not be reliable, as the margins of error increase when results from separate samples are compared. However, it is reasonable to conclude that an institution must make improvements at its bilingual points of service if it obtains poor results in every observation exercise.
Observations of service over the Internet
Observations of service over the Internet
Background
The observations of service to the public conducted by the Office of the Commissioner of Official Languages (Office of the Commissioner) are one of many tools used to measure the performance of federal institutions with respect to Part IV of the Official Languages Act, which deals with service to the public. The Office of the Commissioner observed three types of service provided by the institutions: service in-person, by telephone and over the Internet. This tab describes the methodology used for the observations of service over the Internet.
For the second year in a row, the Office of the Commissioner gathered data using a variety of methods for communicating with federal institutions over the Internet. In keeping with the digital shift, observations were conducted on various platforms, including computers and mobile devices.
Federal institutions use a number of on-line media. The Office of the Commissioner therefore grouped them into two categories of services that have similar functions.
The first category includes interactions with federal employees through e-mail or on-line forms. Using the observations conducted in this category, the Office of the Commissioner sought to compare response rates and times in English and French. The second category of services includes content posted on-line by federal institutions for public viewing. Using samples of content published on federal institutions’ websites, social media accounts, applications and on-line accounts, the Office of the Commissioner sought to determine whether on-line content was equivalent in both official languages.
Methodology
The methodology was established in cooperation with Statistics Canada, which also participated in the interpretation and validation of the results.
Unlike the results of the observations conducted in person and by telephone, which indicate the availability of service in the official language of the linguistic minority, the results of the observations of interactive services over the Internet compare response rates, response times and the availability of information in both official languages.
For the observations of interactive services over the Internet, equivalent messages in English and French were sent to each institution to compare response rates and times. The observations sought to simulate the experience of a member of the public who contacts the institution to obtain information.
Through the interactive on-line observations, it was possible to compare the number of responses and the time it took to receive a response in each official language.
The observations of interactive services over the Internet were conducted between February 21 and March 27, 2019; therefore, the results are representative of that time period.
Observations of on-line content were conducted on a variety of on-line sites and media provided in advance by federal institutions. These on-line observations sought to simulate the experience of a member of the public who browses on-line to obtain information or learn about a topic. Observers evaluated whether the content was equivalent in both official languages by examining the availability in English and French of the same sample content and by looking at the targets of the links provided. A summary review of social media posts was also conducted to compare the quality of the posts in each official language.
The observations of on-line content were conducted from January 31 to February 26, 2019; therefore, the results are representative of that time period.
Appendix A: Comparable response rates and times
1) Comparable response rates
The response rates in both official languages make it possible to determine whether federal institutions provide comparable service in English and French, without taking established service standards into account.
i. Calculating response rates in English and French
-
- Response rate in English: (Number of English responses received ÷ Number of English e-mails and forms sent) × 100 = x%
- Response rate in French: (Number of French responses received ÷ Number of French e-mails and forms sent) × 100 = x%
ii. Determining the score
-
- Comparable response rate score: 100% minus the difference between the response rates in both official languages.
- For example, if Institution A provided a response in English or French for each e-mail and e-form during the observation period, based on the calculation formula, there is no difference (0%) between the two response rates, resulting in a score of 100% for the comparable response rate (see Appendix B).
- As another example, Institution B’s response rate was 90% in English and 40% in French. Since the difference between the two response rates is 50%, the score is calculated as 100 − 50 = 50% for the comparable response rate (see Appendix B).
2) Comparable response times
The average response times for e-mails and e-forms in English and French make it possible to compare response times in each official language. To do this, a score is assigned that represents the proportionality or equivalency of the average response times in both official languages. The closer the score is to 100%, the closer the response times were in Engilsh and French.
i. Calculating average response times
- In order to reduce the effect of excessive response times on the average, the Winsorization estimation method is used. This method involves determining a limitFootnote 5 e.g., 200 hours for a given institution) based on the assumption that a response time exceeding that limit is the result of something other than a question of language. Therefore, any response time exceeding the limit is rounded off to that number for the purposes of calculating the average response times.
- Calculating average response times for English e-mails and e-forms and French e-mails and e-forms
- Average response times (hours) = Response times total (hours) ÷ Number of responses received
- Calculating average response times for English e-mails and e-forms and French e-mails and e-forms
ii. Determining the score
-
- Comparable response time score: 100 × Shortest average response time ÷ Longest average response time.
- For example, if Institution A has an average response time of 75.2 hours for English e-mails and e-forms and 163.9 hours for French e-mails and e-forms, its score is 46% (100 x 75.2 ÷ 163.9 = 46).
- As another example, Institution B’s average response time is 83.3 hours for English e-mails and e-forms and 62.7 hours for French e-mails and e-forms. This results in a score of 75% (100 x 62.7 ÷ 83.3 = 75).
Appendix B: Examples of observations of service over the Internet
| Institution | Response rate in English | Response rate in French | Service availability score | Average response time in English | Average response time in French | Difference between the average response times (hours) | Difference between the average response times (%) | Response time score |
|---|---|---|---|---|---|---|---|---|
| A | 100% | 100% | 100% | 75.2 hours | 163.9 hours | 88.7 hours | 54% | 46% |
| B | 90% | 40% | 50% | 83.3 hours | 62.7 hours | 20.6 hours | 25% | 75% |