Writing

AI time savings are being booked as permanent. The law says otherwise.

On January 1, 2027, two state rules give people a way out of automated decisions. California’s new CCPA regulations require a business that uses automated decision-making technology (ADMT) for a significant decision about a consumer, such as lending, housing, employment, education, or healthcare, to offer either an opt-out or an appeal to a human reviewer with the authority to overturn the result [1]. Colorado’s rewritten AI law, SB 26-189, takes effect the same day and gives consumers the right to request meaningful human review after an adverse automated decision [2].

Both rules assume a person is available to do the work the automation was built to remove. In tech, operations plans increasingly rely on agents to do the work of frontline associates, and the time and output those agents free up is already counted as savings. Teams are converting AI time savings into headcount and capacity assumptions as if the automation were permanent, while the law increasingly treats automation as something a customer can step out of. The dissonance between those two assumptions is an operational liability, and it is widest among the tools nobody has registered.

The savings are already in the headcount plan

The push to automate is rational. Operations leaders are being asked to do more with less, AI agents that summarize tickets, triage queues, and apply deterministic policies do remove real manual work, and the pressure to implement quickly is real. In a Gartner survey of 321 customer service and support leaders, conducted in September and October 2025, 80% reported pressure to make workforce changes because of AI, 63% said they are reducing frontline headcount gradually through attrition, and 31% said they have implemented or plan layoffs in response to AI through the first quarter of 2027 [3]. Gartner reads these numbers as workforce redesign rather than elimination [3], but redesign still runs on attrition that is not backfilled.

That is how time savings become a plan. Attrition, whether voluntary or forced, is not backfilled, targets per person rise, and the hours an agent removed are written into next year’s capacity model. What the models rarely account for is the set of conditions under which those hours come back. The same Gartner research found that 85% of leaders are adding responsibilities to frontline agent roles, and in a separate survey of 5,801 US customers, 54% said they trust human agents more than AI for product or service recommendations [3]. The human’s role is getting wider while the headcount behind it shrinks.

The law already requires a human path

The common framing of regulatory risk is a future event: a new law arrives and forces a company to switch a model off. That risk is real, but it misses the obligation that already exists. Several rules, in force now or taking effect within the next year, require a human alternative to an automated decision, which means the capacity the automation replaced has to be available on demand.

California requires at least two ways to submit an ADMT opt-out, and at least one of them must reflect the primary way the business interacts with the consumer. A business can skip the opt-out only by offering an appeal to a designated human reviewer who knows how to interpret the system’s output and has the authority to change the decision, and the remaining exceptions for hiring, admissions, and work allocation are narrow [1]. Appeals must be acknowledged within 10 business days and answered within 45 calendar days [1]. Either route ends with a trained person working to a deadline.

Colorado requires a plain-language explanation within 30 days of an adverse outcome and a right to human review and reconsideration [2]. In the United Kingdom, the automated decision-making provisions of the Data (Use and Access) Act 2025 came into force on February 5, 2026, and permit significant automated decisions only with safeguards that include human intervention and a way to contest the result [4][5]. California SB 947, which would bar employers from relying solely on an automated system to discipline or terminate a worker from July 1, 2027, was awaiting the Governor’s decision as of September 28, 2026 [6].

The European Union adds the scenario the common framing describes. Two new prohibitions under the AI Act apply from December 2, 2026, and remove affected systems from the market entirely. Under the amended transition rules, a high-risk system already in use becomes subject to the high-risk obligations if its design changes significantly after the application date [7]. A routine model update can move an agent into a different legal category. Model cards and an agent registry are how a team notices when that has happened.

None of these rules says how many customers will opt out or ask for a human. That number is a variable the operator does not control, and a capacity plan that assumes zero has chosen a value without data. The useful question for each automated decision that touches a customer is concrete: if a meaningful share of people ask for a human, how many hours return, and whose queue do they land in?

The agents nobody registered

Much of the automation in operations does not come from engineering roadmaps. Frontline staff are encouraged to find their own ways to optimize their work, often with tools running on their own machines and built for one person’s workflow rather than for scale. The survey data on unsanctioned AI use points in the same direction. In research conducted by Sapio Research for BlackFog in November 2025, covering 2,000 employees at US and UK organizations with more than 500 staff, 49% said they use AI tools their employer has not sanctioned, and 51% said they have connected AI tools to other work systems without IT approval. A third said they had shared research or datasets with such tools, 27% had shared employee data, and 23% had shared financial statements or sales data. Sixty percent agreed that using unsanctioned tools is worth the security risk if it helps them work faster or meet deadlines [8].

That survey measures tool use, not agent building, and I do not want to conflate the two. The pattern I have seen is the next step: the same people who adopt tools informally start chaining them into small automations that read customer data or make routine calls about customer accounts. Each one is reasonable in isolation. Together they create two blind spots. A company cannot budget fallback capacity for an agent it does not know exists, and it cannot honor an opt-out or answer an access request for a decision it does not know is automated. California’s access right requires a business to explain the specific purpose of the ADMT, the logic behind it, and how its output was used in the decision [1]. That explanation is difficult to produce for a script on someone’s laptop.

The builders are ahead of the training

The people building these tools are acting in good faith. They are solving the problem in front of them with the tools available. What they usually lack is specialized knowledge of privacy, AI governance, or the regulatory requirements their tool now touches, and that is not a personal failing. Corporate privacy and security training tends to cover phishing, data handling, and acceptable use. In my experience it rarely covers what to do when an employee builds an agent.

Governance functions are catching up from a small base. In the survey behind the IAPP and Credo AI’s AI Governance Profession Report, which drew more than 670 respondents in spring 2024, 77% of organizations said they were working on AI governance, only 1.5% said they would need no additional AI governance staff in the following 12 months, and 23.5% said finding qualified AI professionals was part of the challenge of delivering AI [9]. Many AI councils are still defining their own standards.

What good looks like

None of this requires slowing adoption. It requires applying governance standards to every agent, spreading that knowledge across every level of the business, and planning for a forced pause before a regulation or a customer imposes one. Here is what I think a working AI governance program does.

  • Register every agent, including the ones built by individuals, through an intake that takes minutes rather than a committee cycle.
  • For each agent that touches customer decisions or sensitive data, record the hours it replaced and the team that would absorb them if it stopped. That is the fallback line the capacity plan is missing.
  • Build and test the human review and opt-out path before January 1, 2027, and measure its volume from the first day.
  • Add a governance checkpoint to change management, because a design change can move a system into a different legal category.
  • Give builders a one-page standard: what data an agent may touch, which decisions it may not make alone, and where it must be logged.

A company that does this keeps its savings and can show exactly where the work would go if the rules or the customers change their minds. A company that does not is exposed every time the rules change, and they are still changing.

Risk is a dial, not a switch. Every agent needs a way to turn it back toward a person.

References

  1. California Privacy Protection Agency. “Text of Regulations: CCPA Updates, Cybersecurity Audits, Risk Assessments, Automated Decisionmaking Technology, and Insurance Regulations.” Approved September 22, 2025. Cal. Code Regs. tit. 11, secs. 7021, 7220 to 7222. cppa.ca.gov
  2. Colorado General Assembly. “SB26-189 Automated Decision-Making Technology.” Signed May 14, 2026. leg.colorado.gov
  3. Gartner. “Gartner Survey Finds 85% of Service and Support Leaders Are Expanding Human Agent Responsibilities Despite Expectations of Mass AI Layoffs.” Press release, April 28, 2026. gartner.com
  4. UK Government. “The Data (Use and Access) Act 2025 (Commencement No. 6 and Transitional and Saving Provisions) Regulations 2026.” SI 2026/82. legislation.gov.uk
  5. UK Government. “Data (Use and Access) Act 2025,” section 80 (automated decision-making). legislation.gov.uk
  6. California Legislature. “SB-947 Employment: Automated Decision Systems.” Bill status, 2025-2026 session. leginfo.legislature.ca.gov
  7. European Union. “Regulation (EU) 2026/1744 of the European Parliament and of the Council of 8 July 2026 Amending Regulation (EU) 2024/1689 (Digital Omnibus on AI).” Official Journal of the European Union, L, July 24, 2026, amended Articles 5, 111, and 113. eur-lex.europa.eu
  8. BlackFog. “Shadow AI Threat Grows Inside Enterprises as BlackFog Research Finds 60% of Employees Would Take Risks to Meet Deadlines.” January 27, 2026. blackfog.com
  9. IAPP and Credo AI. “AI Governance Profession Report 2025.” April 16, 2025. iapp.org
Topics: AI governance operating privacy