IEEE Technology and Society Magazine - December 2021 - 46

Soft Law Governance of Artificial Intelligence
market forces, labeling, professional journals, FTC
enforcement, and liability.
For indirect enforcement, the efforts most likely
to succeed are those focused on lawyers and other
regulated legal-services professionals, given their
existing professional obligations, including under
rules of professional conduct. The possibility of liability
for harm caused by AI is also likely to drive
compliance by lawyers, other legal-services professionals,
and AI developers. Organizations that
employ lawyers, other legal-services professionals,
and developers would likely enact internal measures
to comply with soft law AI guidelines, including
because they may themselves be subject to
discipline (e.g., for failure to supervise others developing
AI) and liability.
Below, I discuss the mechanisms that hold the
greatest promise for soft law programs to result in
the development of and adherence to standards
for evaluating the effectiveness of AI for legalservices
delivery.
NGO monitors and academic researchers
Nongovernmental organizations and others
could monitor and report publicly on legal industry
compliance with AI guidelines. It is unlikely that
lawyers would consent to direct monitoring, given
their obligations to clients, including confidentiality.
Nevertheless, there are many other ways to monitor
compliance, from evaluating specific AI tools to
monitoring organizations' adoption of and adherence
to AI guidelines.
Academic researchers could fill important
roles. Examples include Buolamwini and Gebru's
[3] " Gender Shades " research about facial analysis
technology effectiveness and bias and Grossman
and Cormack's [40], [41] research evaluating
the effectiveness of AI for electronic discovery. In
addition to research focused on evaluating specific
AI tools, academic research could contribute
to developing specific guidelines and methods
for evaluating AI effectiveness. Academics could
do some of this in collaboration with professional
societies and trade organizations. They could
develop and market guidelines that lawyers and
other professionals, by their association with lawyers,
would feel compelled to adhere to given
existing obligations under the rules of professional
conduct, fiduciary duties to clients, and the risk of
malpractice liability.
46
Academics and organizations could develop and
market:
* checklists, toolkits, and other resources to foster
compliance by those subject to such guidelines;
* checklists, toolkits, and other resources to make
it easy for customers, including corporate legal
departments, to require that
their
suppliers
(including lawyers) adhere to principles and
guidelines and monitor and audit compliance;
* standardized materials for teaching lawyers,
other legal-services professionals, and developers
about evaluating AI effectiveness;
* conferences and workshops on AI principles and
guidelines, both to foster research and to train
individuals;
* online training, including continuing legal education
(CLE) for lawyers; and
* programs to certify individuals, organizations, and
specific AI tools.
Academics could also study compliance with
AI principles and guidelines by vendors, law firms,
legal aid organizations, and other organizations.
This could begin with reviewing public-facing materials
on websites and evaluating each organization's
transparency, including the availability of information
that permits others to determine each organization's
compliance. Follow-up requests could be
made to determine what additional information
the organizations would disclose. The assembled
information could be made available in an open
database. Transparency could help develop a
market for this information, creating incentives for
organizations to demonstrate compliance with AI
principles.
Fostering public release of information about AI
effectiveness may itself lead to improved effectiveness.
For example, although there has not been a
definitive finding, healthcare quality experts believe
that the act of releasing information about hospital
performance itself leads to improvements in
quality [50].
Supply chain
require law firms,
Corporate legal departments and courts could
legal technology companies,
and other legal-services providers to comply with
specific AI principles. Law firms and other services
providers could likewise require compliance
by their
suppliers. Customers purchasing legal
IEEE Technology and Society Magazine
IEEE Technology and Society Magazine - December 2021

Table of Contents for the Digital Edition of IEEE Technology and Society Magazine - December 2021
