Rytr and Pieces Settlements Signal New Age of AI Enforcement

Since the U.S. Federal Trade Commission has been at the forefront of combating perceived misuses of AI, the AI policy community was not surprised by the FTC’s announcement on September 25 that it had taken action against alleged AI-powered deception by five companies. At the same time, one of these actions – against the generative AI company Rytr – has novel elements, as discussed below.

More surprising than the FTC’s actions was the September 18 announcement by Texas Attorney General Ken Paxton that he had reached a “first of its kind” settlement with Pieces Technologies, a provider of generative AI for healthcare, described further below.

 FTC’s ‘Operation AI Comply’ and the Rytr Complaint

On September 25, the U.S. Federal Trade Commission announced actions against five companies for allegedly using AI to “supercharge deceptive or unfair conduct that harms customers.” Two of the companies – DoNotPay and Rytr – reached settlements with the FTC, while the other three companies are subject to court orders and awaiting trails. 

The most controversial of these actions was directed against Rytr, a company that enables the generation of customer reviews, testimonials, blog posts, marketing materials, e-mails and other text content in specific tones (such as ‘serious’ or ‘humorous’). Two of the five FTC Commissioners voted against the FTC’s issuance of the complaint. Several FTC-watchers and commentators described the FTC’s action as unprecedented.

The FTC’s complaint focused on consumer reviews and testimonials, which the FTC suggested often contained “specific, often material details that had no relation to the user’s input” and were therefore “false and deceptive.” Since most people would not be able to distinguish between AI-generated customer reviews and testimonials and legitimate ones, Rytr’s services could “pollute the marketplace with a glut of fake reviews” and harm consumers, according to the FTC. Further, by providing this ability to customers, Rytr furnished customers with “means and instrumentalities for the commission of deceptive acts and practices.”

The two dissenting Commissioners argued that the FTC’s complaint did not show evidence of any false and deceptive reviews actually being posted online, that it constituted a “dramatic extension” of means-and-instrumentalities liability, that it did not appropriately consider benefits to consumers during the drafting process and that it was part of a premature and aggressive effort that could potentially “bend the law to get at AI” and “chill innovation.”

Texas and Healthcare AI Company Settle Deceptive Claims Matter

In a surprising development, Texas Attorney General Ken Paxton announced a settlement with Pieces Technologies on September 18. Pieces is a health care company that uses generative AI to produce medical documentation, often based on patient data. At least four Texas hospitals use services from Pieces. Attorney General Paxton’s office suggested that Pieces made “deceptive claims” about the accuracy of its documentation. Though Pieces denied wrongdoing and maintained that its accuracy claims were correct, it agreed to provide further information about the accuracy of its products and to make sure that hospital staff using its services understand the extent to which they may or may not be able to rely on them. The episode signals that state authorities are increasingly willing to act against allegedly deceptive AI claims.

Enzai is here to help

Enzai’s AI GRC platform can help your company deploy AI in accordance with best practices and emerging regulations, standards and frameworks, such as EU AI Act, the Colorado AI Act, the NIST AI RMF and ISO/IEC 42001. To learn more, get in touch here.