Open
Society Institute: Justice Initiative
Managing
for results
A
Report
prepared by Paul Hubbard
13 December 2006
Executive Summary
l
The Justice
Initiative is an operational program of the Open Society Institute.
l
Its mission is
expressed in terms of activities, such as pursuing law reform and contributing to legal capacity. Similarly its objectives and program materials are expressed
in terms of activities.
l
Where goals have
been expressed, the majority have not been met. For example, from a sample of nine goals in 2003, only three
have been fully realized as at November 2006.
l
The
organization's reporting focuses exclusively on activities it undertakes. This distracts the organization from
achieving its strategic goals.
l
In the absence of
measurable goals or objectives, it is difficult to evaluate what the
Justice Initiative's activities have in fact accomplished.
l
The inability of
the organization to demonstrate its effectiveness in these terms is reflected
in a budget decrease from $15.2m in 2003 to $11.2m in 2005.
l
The
Justice Initiative should adopt results based management. By focusing on results rather than
activities, the Organization will be able both to justify its resource
allocation, and also achieve its medium and long term strategic goals.
Overview
of Open Society Justice Initiative
In 1993, investor
George Soros founded a private non‑government organization, the Open
Society Institute, to promote democratization in the former communist eastern
bloc. The Open Society Justice
Initiative is a program of the Open Society Institute. It was established in 2002 to replace
the Constitutional and Legal Policy Institute.[1] The Constitutional and Legal Policy
Institute had focused regionally on Central and Eastern Europe, Central Asia
and Mongolia.[2] The Justice Initiative was intended to
replace this program as a worldwide initiative.[3]
Stated Mission and Objectives
The Initiative's
mission is expressed in terms of activities. It 'pursues law reform activities grounded in the protection of human rights, and contributes
to the development of legal
capacity' (emphasis added).[4] The mission expands these activities
into litigation, legal advocacy, technical assistance and knowledge
dissemination. Furthermore, it
gives five 'priority areas', which are national criminal justice, international
justice, freedom of information and expression, and equality and
citizenship.
While it is possible
to judge whether these activities have taken place, the mission provides no
clear targets against which to determine whether the activities have been
effective. It is therefore
difficult to establish that the activities have been effective and if they have
efficiently used of resources.
The Initiative's
objectives are also stated as activities.[5] They are to 'Give local meaning to
global norms', 'Universalize local norms', 'Foster a legal culture for an open
society', 'Catalyze a process of change' and 'Produce and spread
knowledge'. In order to determine
whether these objectives have been met, it is necessary to recast them as
end-states the Institute would like to see, rather than activities that it has
undertaken.
Justice Initiative goals
In
August 2003 the Executive Director of the Initiative published an introduction
to the Initiative's activities stating some of its goals, and time frames in
which they would be achieved.[6] This provides some evidence of
strategic planning within the organization. These goals can be expressed as end states, from which it is
possible to determine whether or not they have been achieved.
This
list is not exhaustive, but does provide provide a sample of short, medium and
long term end‑states for evaluation. The short term goals are most precise, expressed as end
states. The medium term goals are
more generally stated. This makes
it harder to form an objective judgment of whether the goals have been met
satisfactorily. For example, goal
five neither states the number of Mexican government officials we would expect
to see trained by August 2006 nor what we can expect them to have
learned.
The
long term goals are most problematic.
They are either difficult to measure or dependent upon results being
achieved by other parties. For
example, the abolition of criminal sanctions for defamation depend on the
change of national laws over which the Justice Initiative may have little
influence. Conversely, a country
might remove criminal sanctions for defamation without any input from the
Justice Initiative. In each case
the stated outcome is not directly related to the performance of the Justice
Initiative's actions. Even if the
outcome is achieved, we can not confidently infer that that the program itself
was effective or efficient. In any
event, there is no date at which the goals are to be systematically
evaluated.
Reporting
The
Justice Initiative does not publish documents tracking its goals or the steps
taken to achieve them. In the Open
Society Institute's Annual Reports only the major activities in which the
Justice Initiative are reported.
Since February 2004, the Justice Initiative has published nine
'Activities Roundups' reporting what the program has done in the preceding two
to four months.
In
the period from Feburary 2004 to October 2006, these included 212
reported activities. The Institute
separately reports published eight books, 29 legal briefs, reports or papers,
and 17 conference reports. As with
the Justice Initiative's stated Mission and Objectives, the focus of
management reporting is on the activities undertaken by the organization,
rather than the realization of specific end states.
The
reporting does not detail resources allocated to the individual activities that
took place, nor a qualitative evaluation of the activity's contribution to the
overall objectives. It would
therefore be difficult for management determine which activities are an
efficient and effective use of the Initiative's budget.
Evaluation of goals
From
the reported activities, it is nevertheless possible to determine whether or
not the goals specified in August 2003 have been met. Of the nine goals, only three have been
fully realized (1, 5 and 7). This
is where one of the reported activities of the Justice Initiative demonstrates
that a clinical legal education conference was held,[7]
that freedom of information training in Mexico did take place[8]
and that a set of fundamental rights for non‑citizens have been
internationally recognized.[9]
Although it met one of the Justice Initative's stated goals, the adoption of general recommendations on the rights of non‑citizens by the United Nations' Committee on the Elimination of Racial Discrimination was reported neither in the Justice Initiative's periodic reports nor the Open Society Institute's Annual Reports. This demonstrates the failure of the Justice Initiative's management approach in focusing on the activities of the organization. Because the activity of adopting the recommendations was done by the UN Committee rather than the Justice Initiative, the Justice Initiative did not claim any credit for it.[John Math1]
The
Justice Initiative could be contrasted to a strategy of the successful non‑government
organization Greenpeace, which
claimed some credit for the entry into force of the Kyoto protocol.[10]
[John Math2] By focusing on activities rather than
end-states, the Justice Initiative missed the opportunity to publicize a
success. However, it should be
noted that the achievement of this goal, while a success for the
Justice Initiative, is not evidence that its program or activity in
question is managed efficiently.
The other six 2003
goals of the Justice Initiative have only been partially met. While the organization has reported
activity in most of the areas surrounding the goals, the goals themselves have
not been met. In cases where the
reform process has not been in the control of the Justice Initiative, for
example the civilian oversight of police in Nigeria, or the abolition of
criminal sanctions for defamation, the Justice Initiative has been able to
assist various official processes.
However, the necessity to react to the external processes mean that the
Justice Initiative has not been able to maintain control of their
direction in which case the activities are related, but not directed to the
goal.
One goal that is
particularly difficult to evaluate is the reduction of corruption surround
electoral campaign finance (number 8).
The Justice Initiative does not report any metric by which this
corruption is to be measured, in which case it is not possible to determine if
it has been reduced. The Justice Initiative which will
undoubtedly contribute to this goal, such as the publication of a practical
handbook for non‑government organizations.[11] However, without being able to
demonstrate empirically that the goal has been met, it is difficult to show
that the goal has been met, or what further activity is required.
One unequivocal
success of the Justice Initiative is that it has met its founding objective to
service regions beyond Eastern Europe and Central Asia and Mongolia. The fact that activities have been
reported in Argentina, Mexico, Peru, Cambodia and throughout Africa demonstrate
that the Justice Initiative does operate on a world-wide basis in
accordance with the original goal.
It is not clear whether the Justice Initiative intends to expand or
consolidate its international programs.
An evaluation of its programs by country may aid the decision of where
its resources are most effectively allocated.
Budget and finance
The Justice Initiative does not publish
program budgets, only the expenditure for the overall program is reported as
part of the Open Society Institute's Annual Reports. However the limited financial information is sufficient to
show a real decline in program funding between 2003 ($15.2m) and
2005 ($11.2m), at a rate only marginally higher than the regionally focused
Constitutional and Legal Policy Institute received in 2001 ($10.1m).
This significant decline in funding supports
the conclusion that the Justice Initiative has not demonstrated to its
funding body, the Open Society Institute, that its allocated resources have
been effectively spent. In order
to demonstrate to the Open Society Institute that resources allocated
to the Justice Initiative are well spent, the Justice Initiative
should report annually on program expenditure and the outcomes that have been met
with that expenditure.
Conclusion and Recommendation
Since its
foundation, the Open Society Justice Initiative has undertaken many activities
in the areas of law reform, human rights and legal capacity. However, the organization's planning
and reporting focus only on the activities that the Institute performs. This makes it difficult for the
Justice Initiative to plan over the medium to long term. It also makes it difficult for the
Justice Initiative to demonstrate that its resources are used effectively. Despite involvement in hundreds of
activities, as well as the publication of several high quality resources, the
Justice Initiative's funding has declined significantly. Based on this evidence, there are
major problems with the strategic management of the Justice Initiative.
The focus of
management needs to shift from being activity focused to being results
focused. The organization's
mission and objectives should be recast in terms of achievable goals, rather
than activities. These goals should
be expressed in terms of what the Justice Initiative would be able to
observe if the goals have been met.
This provides objective criteria by which to evaluate the effectiveness
of individual programs. Budgeting
should be undertaken on the basis of achieving these goals, rather than simply
supporting activities. Similarly,
the Justice Initiatives periodic 'Activities Roundup' should be renamed
the 'Results Roundup' in order to report the results that the
Justice Initiative has achieved.
Without observable
indicators, it is difficult to evaluate whether activities are an effective and
efficient way to meet the goals of the Justice Initiative or the
Open Society Institute.
No further funding should be advanced to the Justice Initiative
until each of its program goals is explicitly defined in terms of the observable results that management
expects it to generate.