


	History Doc Evatt Our Purpose Who's Who FAQs

	Evatt News Evatt Events

	evatt e-news info

	Books Papers Order Form Letters Archives

	Join Online Request Form Download Form Donate Online

	Send Feedback Contact Us Web Survey Change Details

The State of the States 2006: Performance methodology

The performance assessment method upholds an ideal.

The method

As in past years, each of Australia�s states has been assessed by integrating a range of discrete social, environmental and economic indicators to produce a single � but very broadly based�measure of its performance. The methodology produces a single figure assessment of performance by following three steps.

Selecting the yardsticks

First, 15 statistical measures are selected as performance yardsticks. An obvious but unavoidable shortcoming is that the choice of statistics is constrained by the limited range of useful, consistent, accurate and comparable data collected across all of the states. The paucity of meaningful, disinterested, accurate and comparable data collected across jurisdictions at once justifies and hinders The State of the States series. The scarcity highlights the need for investment in performance monitoring and accounting if citizens are to participate effectively in public affairs and have confidence in their governments, but the same paucity also hinders independent attempts to remedy this failure.

Within the limitations of the available data, The State of the States method involves selecting five statistical measures of the relative performance of the states in each of the areas of social, environmental and economic policy. Each of the sets of five indicators is selected according to how broadly the individual indicator appears to measure performance within each of the three still broader performance areas. Of the 15 indicators, nine are derived from the 2006 update of general grant relativities by the Commonwealth Grants Commission (CGC), five are drawn from Australian Bureau of Statistics (ABS) sources, and one is from the Australian Greenhouse Office within the Comonwealth Department of Environment and Heritage. All the indicators for this year�s League Table measure the financial year ended on 30 June 2005. The measures are set out in Table 7, and full references are in Appendix A.

Ensuring quality measurements

Secondly, the performance of each state is calculated by comparing it with the average performance of all the states. For CGC data, actual per capita levels of spending are divided by the CGC�s assessments of what each state should spend in order to provide an average national standard of service to citizens within their jurisdiction. This procedure assumes an average level of productivity and is consistent with the way these services are calculated to measure Australia�s Gross Domestic Product. More importantly, the great virtue of the CGC statistics is that they enable us to compare like with like by taking into account the specific costs due to influences beyond the policy control of the states (costs that are known by the CGC as �disabilities�).

The role of the CGC and the unique quality of its statistics warrant further elaboration, since the work is not only too little understood and appreciated, the institution is regularly criticised by the treasurers of the three wealthiest states, New South Wales, Victoria and Western Australia. The Commission�s assessments are undertaken to facilitate �horizontal fiscal equalisation� between jurisdictions in the annual distribution of the Commonwealth�s funding grants to the states, including the funds raised by the GST (goods and services tax). �Horizontal equalisation� is a feature of almost all the world�s major federations and is also found in many unitary countries. The concept is, as the CGC has described it, �part of the fabric of nationhood � that better endowed states or regions contribute to the capacity of those whose resource bases are not as abundant or whose needs are greater, for reasons beyond their own control�. The emphasis on horizontal equity is an expression of what it means to be an Australian, as distinct from, say, an American. To be a US citizen also means living in a country where per capita spending on public schools, for example, can vary by up to 200 per cent between states. This would be practically impossible under Australia�s system of horizontal equalisation. The concept and practice of horizontal equity articulates the idea of a nation choosing to invest in perpetuating the ideal of an Australian community. Countries as diverse as China and South Africa have set up similar equalisation systems by drawing directly on Australia�s experience.

As a by-product of the equalisation process, the CGC generates the best statistics for comparing the expenditure levels of the different states in particular service areas. The Commission pools the expenditure of all the states and developes an average Australian per capita state budget. Next it identifies and measures the differences between the states in what it costs to provide the Australian average level of services to their populations, adjusting for �disabilities�. The disabilities that the CGC takes into account include socio-demographic composition, which takes into account differences in the characteristics of state populations; the demand for and unit costs of state services; administrative scale; input costs; urban influences; economic environment; dispersion; isolation; service delivery scale; and physical environment.

For example, hospital services are used more intensively by some age groups and by Indigenous people; some welfare services are used more intensively by people on low incomes; some services are used more by people living in remote areas. States are assessed as having a disability if the groups that make most use of a service comprise a larger proportion of their population. Likewise, cost disabilities arise because the prices of inputs vary for reasons that are beyond the contol of individual states. Wage rates and office accomodation costs, for example, differ across states. The CGC makes discrete allowances for state disabilities to enable state expenditure and revenue levels to be fairly compared against what the state should be taxing and spending to have the capacity to provide the same average level of services as the other states. The average that The State of the States series uses to compare the performance of the states is, therefore, a �CGC�adjusted� national average. While the results that come out of the CGC are of course only as good as the data that go into its calculations, they remain extraordinarily sophisticated and easily the best available state comparative measures.

The ABS statistics are more simply calculated by dividing each jurisdiction�s actual per capita result by the unadjusted average per capita result for all six states. Implicitly, comparisons based on ABS data are cruder than the CGC figures, since they assume there is a �level national playing field�, that Commonwealth policies have consistent national effects, and that Australians share a common global economic environment. Nonetheless, the ABS statistics are of a high level of integrity, and they are primarily used in the League Table to measure outcomes. To the extent that the available levels of public services are equalised through the CGC system, these outcomes are exceptionally good quality performance indicators.

The final results

Thirdly, a final single figure measurement of each state�s performance is calculated by averaging the three sub-totals. No weighting is used for any of the three major performance areas or the separate categories and sub-categories. Social, environmental and economic policy performances are ranked equally, reflecting the philosophy underlying the methodology; namely, governments should be assessed according to their effort in a broad range of policy areas that are significant in determining the quality of life enjoyed by Australia�s citizens.

Other explanatory notes

The social and economic policy performance indicators are unchanged from previous years. This year�s environmental indicators are, however, different from last year. Instead of measuring the availability of environmentally friendly products and the prevalance of water conservation practices, the statistics for neither being published by the ABS this year, the table has employed an ABS survey measuring household awareness of greenpower energy schemes and, for the first time, a comparison of greenhouse gas emissions. The changes are simply due to the unavailability of the other measures this year. This means that a good deal of care should be made in reading changes in rankings from year to year. More detail is provided in Table 10 and Appendix C.

As with past practice, the territories have been excluded from the assessments, mainly because the small size of their populations relative to their high level of (heavily subsidised) services distorts the basis of comparison too unfairly, but also because these jurisdictions are structured differently in relation to local governments and are in any event ultimately creatures of the Commonwealth government. The territories do, however, contribute to the calculation of the CGC-adjusted average measure, since it would be too complex to net them out for the purpose of the League Table.

The full picture?

It is important to emphasise that the Evatt Foundation does not hold that the assessment method used in The State of the States series is perfect. No measurement system can pretend to be able to reduce the complexity of government, social and individual reality to unambiguous numbers. For every performance answer provided by numbers that have been torn from their contexts, more questions will always go begging. These qualifications do not diminish the virtues of the method used in The State of the States, of which there are at least four.

First, the method is a commonsensical one that is easily understood by the intelligent and educated citizen who is curious about or wishes to get an idea or form an impression of how the different Australian states are performing. Secondly, The State of the States gives wider circulation to the research undertaken by the Commonwealth Grants Commission than would otherwise be the case. This unique, high quality research is precious in the Australian context, but is unfortunately little known outside the world of a few specialist finance journalists and treasury officials. As well as spreading the information around, the series aims to help build an appreciation for the institution of the Commonwealth Grants Commission itself and to increase understanding of the important role it performs. Thirdly, unusually, the method used by The State of States series yields broad comparisons between the states based on their social, environmental and economic performance. Finally, uniquely, the method provides a readily accessible, broad, overall government performance comparison. Which state is closest to that formula for perfection: first class social services, uncompromising environmental protection, and thriving, job-generating economic activity?

That said, the chief virtue in the approach of The State of the States does not reside in the actual results it produces. Rather, a higher value is embodied in the way that it upholds as an ideal principle the belief that governments should be assessed on a broad basis according to the effort they put into serving Australia�s citizens. The scarcity of assessments along these lines means that the most enduring significance of the League Table results lies in the way that they are used as an instrument to stimulate questions and debate about the performance of the states and the quality of different assessment methodologies. As such, the method runs against the grain of the continuing emphasis among Australia�s public commentators on the need for governments to primarily be accountable to finance markets. It also tilts against many of the popular methods of assessing governments, such as the charisma of the leaders, and most of the other less well known and less accessible performance methodologies used in Australia and overseas. Perhaps most importantly of all, The State of the States represents one of the few regular public non-government reports that aim to draw national attention to policy and performance at the sub-national level of government.

All the same, no statistical series, no matter how broadly based or sophisticated in its adjustments, can pretend to capture the full complexity of government performance. The method used in The State of the States series is distinguished by virtue of its generally straightforward empirical procedures and the broad range of measures it draws upon to formulate the League Table. Yet, if the method is less partial than most other performance assessments, it still remains only a partial representation of how well our governments are doing. In particular, it should be recognised that The State of the States League Table ranks the governments in relation to each other. It does not provide an absolute measure of performance. Irrespective of the different positions on the Table, it is entirely possible that all the states have performed better than last year. Equally, it is possible that the standard may have dropped.

This is Chapter Four "The Method" from The State of the States 2006.

Buy The State of the States 2006 online now:

The State of the States 2006 costs $24.95 plus postage. You can purchase The State of the States 2006 securely online. Click here for more information.