A Comparison of 1994 /1995 International Survey of Economic Attitudes Data with Censuses

Introduction

To assess the representativeness of Australia’s, Finland’s and Poland’s editions of the International Survey of Economic Attitudes Round II conducted in the mid-1990s (Kelley and Zagórski 1993), this article compares frequencies on basic demographic and social structural variables from these surveys with data from their respective Censuses. Close resemblance between sample-survey-based estimates and Census-based estimates supports the hypothesis that the sample is representative: that the close resemblance holds not only for the few variables available in the Census for checking, but also on the many variables for which there are no population benchmarks.

Here the 1995 wave of the Australian ISEA (Kelley and M.D.R. Evans 1995) and the 1994 waves of the Finnish (Kangas, Ervasti, Zagórski, and Kelley 1995) and Polish ISEAs (Zagórski, Kolarska-Bobinska, and Kelley 1994)  are compared to data from national Censuses.

Description of surveys

Australia The Australian data was collected in 1995 (Kelley and Evans 1995). It was based on simple random samples of Australian citizens drawn by the Electoral Commission from the compulsory electoral roll. It was conducted by mail using a modification of Dillman's Total Response Method with up to four follow-up mailings, two with fresh copies of the questionnaire, over a six-month period. Completion rates are over 65%. There are 2438 cases.

Finland The Finnish edition of the International Survey of Economic Attitudes was conducted by Olli Kangas and Heikki Ervasti at Turku University in the autumn of 1994 (Kangas, Ervasti, Zagórski and Kelley 1995). It was based on a simple random sample of Finns aged between 18-74. Citizens who speak Swedish as their native language were not included in the sample. Address information was obtained from the Population Register Centre. Originally 3,119 questionnaires were sent of which 1,737 were returned. There were four follow-ups, one with a fresh copy of the questionnaire. Response rate was 56%, which is comparable with similar studies.

Poland The Polish survey was conducted as a II round of International Survey of Economic Attitudes by the Centre for Social Opinion Research, Warsaw, Poland (Zagórski, Kolarska-Bobinska, and Kelley 1994). It was based on an area probability sample of the Polish population, aged 18 and over. It was conducted by face-to-face interviews. Completion rates were over 90% and there are 2127 cases.

I compare these three editions of ISEA to population estimates published in the UN Demographic Yearbook (UN 1997; UN 1996) and national censuses data. The UN data are based on the latest available censuses for a given country. The data for these three countries are regarded as highly reliable. In addition, I refer directly to 1991 Census of Population and Housing, Australian Bureau of Statistics (ABS 1991); Statistical Yearbook for Finland (Tilastokeskus 1995, Tilastokeskus 1997); and 1996 or 1997 Statistical Yearbook for Poland (GUS 1996, GUS 1997).

Comparability of the data

Censuses offer many cases but few variables. Of these, several are useful in establishing representativeness because they are available in closely comparable form in the surveys. Here sex, age and place of residence are compared with the census-based population estimates given in the UN Demographic Yearbook. Education and occupation data tend to be less comprehensive in the UN publications, so I used other sources.

To facilitate the comparison, I recoded survey data to match the (typically coarser) Census categories as closely as possible. Percentages are based on numbers of people who provided data on a given variable, i.e. missing data were omitted from the percentage calculations.

The surveys included respondents aged 18 and over. Most census data are reported for population aged 15 and over with categories broken at 5-year intervals. For maximum comparability I used (where possible) census data for people aged 20 and over, and compared to the whole survey sample, in which the proportion of 18 and 19 year-olds is minimal (on average about 2.5 %). In fact, after rounding, the survey distributions are the same, regardless of whether 18 and 19 year-olds are included in the calculations or not. The Finnish and Polish census data are reported for population de jure and include people temporarily outside the country (who could not be reached by surveys). These differences involve such small numbers of people that they are unlikely to distort the comparison.

Sex

All the three survey samples are representative by sex. The proportions of men and women aged 20 and over are exactly the same for the survey and the Census in Australia and almost identical in Poland. Finnish men are somewhat over-represented, although this is probably within the bounds of normal survey error, which occurs because of sampling variability (Bean 1991) .

All three surveys appear to be representative of place of residence. The census and ISEA figures for Australia are quite close. There are a few more urban respondents in the survey, but the difference is very small amounting to just 2%. Finns from urban areas are perhaps slightly over-represented in the survey (5%). The proportions of Polish urban and rural residents are also very similar between the survey and Census with rural residents very slightly over-represented (by just 2%).

On average the Australian survey were 3.5 years older than the census-based estimates (because the under-representation of the youngest groups is greater than the under-representation of the oldest groups). The Finnish respondents were on average about 3.3 years younger then the census and the Polish respondents were only about a year older than the population. On the whole, the shape of the age distribution in each survey strongly resembles the age distribution in the corresponding census.

Education

Prior research suggests that the better educated tend to be better represented in surveys as they are more literate and more confident to express their views on different issues (Bean 1991). This is true in all three nations compared here. The survey data are recoded to match the most detailed categories available in national censuses. This preserves the detail but makes comparisons country specific. Alternatively one could collapse statistical yearbook figures into cross-country comparable categories, but this would lose information to little purpose, since my goal here is to assess representativeness of the survey within each nation. The results are presented in three separate tables (Tables: 4, 5, and 6).

In Australia the education variable is "age first left school" (Table 4). The data in the census publication are for people aged 15 and over and it was not possible to eliminate the under 18 year-olds to match more closely the survey age groups. In an effort to enhance comparability, I omitted the still-at- school census category from the percentage calculation. Treating that category as missing data leaves a considerable residual downward bias in the Census data, because the only representatives of the younger cohorts to report on their education are those who exited the educational system early (and hence are not "still at school"). Yet, the distortion is probably less than leaving them in. The survey did not include 15-17 year-olds, so still-at-school groups consist of different age cohorts and could not be compared.

* Still-at-school category is omitted from census and survey data

Taking advantage of a more detailed Finnish census data (Tilastokeskus 1997) I show the percentages of population with basic, upper-secondary or tertiary qualification in different age groups (Table 5). Here, people aged 19 or younger were left out both from the census and survey data calculation, as well as the eldest category of people aged 65 or more. The eldest category is smaller than in the census, so the inclusion would distort percentages. In the census the eldest group consists of 79% people with only basic education, 15% with upper secondary qualification and 6% with tertiary degrees. In the survey the corresponding proportions are: 86% of respondents aged 65 or more with only basic education, 10% with upper secondary and 5% with tertiary qualification. Even though   fewer over 64 year-olds responded to the survey, they are well represented with regard to education (Table 5).

*Note: levels of completed education unless indicated otherwise

**To make categories of survey comparable to the census, I included everyone from the cohort aged 20-35 years, in the 'basic education' group if they had 11 or less years of education and did not report having college (opistoasteen koulutus) or tertiary (korkeakoulututkinto) qualification. In the older cohorts people with up to 9 years of education were included into 'basic education' group, unless they reported having college or tertiary qualification. This differentiation was necessary to account for the education system reform in the early seventies in Finland.

Categories of education levels in Finnish Census and the Finnish edition of ISEA match well in across different age cohorts. Minor discrepancies are probably the effects of sampling variability. The Finnish survey included slightly fewer very young people with only basic education. The proportion of young people with upper secondary qualification is somewhat higher in the survey, while the groups with tertiary education make up exactly 15% in the Census and the ISEA. Polish census gives data for people aged 18 and over (Table 6), which matches exactly the age of the ISEA respondents. The two most educated categories are minimally over-represented, but overall the distributions are almost identical.

* Note: levels of completed education unless indicated otherwise

The Australian Bureau of Statistics codes occupation into the Australian Standard Classification of Occupations, which has the 8 major categories used in Table 7. The distributions of the survey and census data are quite close. Yet, as expected, the professionals are the most over-represented group in the survey. On the other hand tradesmen and laborers are several percent less common in ISEA than in the census distribution.

The occupational composition of the Finnish sample is fairly close to the distribution in the Finnish census 1990 data (Table 8). Two top categories are minimally over-represented. The survey included 4% less farmers and farm related workers then the census. Production related workers are underrepresented by just 3%.