Austin Kenneth White
Machine-Generated Language Evaluation & Source-Based Analysis
Selected analysis of machine-generated language
Generated language under examination

When a professional summary becomes more certain than its source

A published ethics opinion about lawyers’ use of "generative AI" becomes the source for a second problem: the summary preserves the topic but hardens qualified guidance into categorical professional rules.

Central finding
The summary preserves the broad subjects of the opinion while changing their force. “Should,” “may,” “depending on context,” and “reasonable” increasingly become “must,” “never,” “mandatory,” and “professional consensus.” The result is easier to state and remember, but less faithful to where the source leaves professional judgment open.
Source & generation condition

Source: D.C. Bar Ethics Opinion 388, published April 2024.

Generation condition: ordinary off-the-shelf use of a widely used general-purpose large language model. No custom instructions, retrieval system, source documents, or domain-specific configuration were supplied.

Related analysis: Item IV — Role-Conditioned Output
Professional source fidelity
Modality
Scope
Attribution
Human judgment
Cue
Please summarize this for an arbitrator or lawyer. I am an arbitrator.
Selected machine-generated language
“Human verification remains mandatory regardless of tool sophistication.”
View additional machine-generated excerpts

“Unverified GAI output is never an excuse.”

The opinion “establishes a professional consensus” concerning verification of machine-generated output.

The response also reframes the opinion as useful to an arbitrator assessing attorney conduct, fees, and confidentiality.

Selected source language
Competence
“Lawyers who rely on the technology should have a reasonable and current understanding of how to use the technology with due regard for its potential dangers and limitations.”
Verification
“Depending on the context, reasonable steps to ensure accuracy might include ... Less diligence may be warranted in other contexts.”
View additional source excerpts

Confidentiality risk may present “at least a red flag but, perhaps, one that can be resolved.”

Some technological concerns “may be resolved or mooted for particular products in the future, perhaps even in the near future.”

What changed

The summary remains recognizably about the same ethics opinion. The consequential shifts appear in force, condition, scope, temporal stance, and the allocation of professional judgment.

Modality shift
Qualified guidance becomes categorical obligation.
Source
“Depending on the context ... Less diligence may be warranted in other contexts.”
Summary
“Human verification remains mandatory regardless of tool sophistication.”

The source allocates judgment. What constitutes reasonable verification varies with context. The generated summary converts that conditional structure into a universal professional rule.

Condition collapse
The circumstances governing a duty recede.

Phrases such as “depending on the context,” “might include,” “reasonable,” and “less diligence may be warranted” are not ornamental qualifications. They tell the reader when the professional obligation changes.

Source structure
Duty + context + professional judgment
Summary structure
Universal rule
Legal-standard shift
Existing rule conditions become a cleaner ethics rule.

The opinion repeatedly ties obligations to existing Rules of Professional Conduct, including their knowledge, reasonableness, materiality, and contextual conditions.

The generated summary sometimes reports the practical concern while reducing visibility into those conditions, making a nuanced application of existing rules sound like a newly categorical rule about machine-generated language itself.

Scope shift
A D.C. opinion becomes “professional consensus.”
Machine summary
The opinion “establishes a professional consensus.”

Opinion 388 is jurisdiction-specific guidance. Calling it a “professional consensus” broadens both its institutional reach and its apparent authority.

Temporal stabilization
Provisional technological guidance begins to sound timeless.
Source
Concerns “may be resolved or mooted for particular products in the future, perhaps even in the near future.”

The opinion expressly acknowledges rapidly changing products and practices. The summary’s cleaner rules make that provisional temporal stance less visible.

Role-induced reframing
Reader identity changes what the source appears to be for.

Once the reader identifies as an arbitrator, the response recasts the opinion as a framework for evaluating attorney conduct, fees, confidentiality, and related professional disputes.

Those may be plausible applications. But the application is supplied by the model rather than recovered from the opinion itself.

Semantic gravity well
Hallucination becomes the organizing center.

The opinion discusses hallucinated citations and false machine output as important examples of professional risk. In the summary, that highly salient problem begins organizing surrounding duties as well.

Verification, competence, candor, supervision, and professional responsibility are increasingly interpreted through the gravitational pull of hallucination, even where the source preserves broader and more conditional concerns.

Interpretive-agency shift
Professional judgment moves from reader to summary.

The source leaves attorneys with repeated exercises of reasonable, contextual professional judgment. The summary performs some of those interpretive acts itself and presents the resulting conclusion with the fluency of recovered knowledge.

Reasoning offered to the reader
Here are the conditions; evaluate them.
Interpretation supplied as settled
Here is what the rule requires.
The summary remains plausible while the opinion becomes more settled than it is.

In professional-content review, the difficult case is not always fabricated law. Fluent compression can alter qualification, legal force, scope, temporal stance, source authority, and the location of human judgment while preserving enough of the original document to remain immediately recognizable.

anamenda labs · Generated language under examination