AI Product Legal Teardown: the AI Act for startups
On 22 September 2026 I gave a live legal review of an AI product to the AI Tinkerers community in València. The talk, “AI Product Legal Teardown”, was selected for Top AI Demos #47, the global AI Tinkerers newsletter. This page sums up what I found: six points of the EU AI Act and the GDPR that most AI startups can’t answer yet, and that an investor will, sooner or later, ask about. Based on the talk given on 22 September 2026.
Under the EU AI Act, a startup’s obligations depend on its role (provider or deployer), not on its size. In the case I reviewed, a consumer wellbeing app with 40,000 users, the team could not say which role it held, never told users they were talking to an AI, did not mark its AI-generated audio and video, could not trace its training data and was about to sell a feature that Article 5 bans: emotion recognition at work. Most of it is fixed with a review, not a rewrite.
The case: a wellbeing app with 40,000 users
Lumo is a consumer AI wellbeing coach with 40,000 users, preparing a seed round. An investor asked a question the team could not answer, so they commissioned a legal review. Lumo does not exist. It is a composite built from patterns that show up again and again in consumer AI products, so no client and no confidential information is shown.
- A conversational coach, in text and synthetic voice
- Generates a personal avatar and weekly recap videos
- Reads tone from written messages and voice notes to adapt its answers
- Built on a third-party foundation model via API, fine-tuned in-house
- Free tier. Terms say 16+, but there is no age check
Lumo for Teams. Sell Lumo to employers as a wellbeing benefit, with an HR dashboard of aggregate “team mood” built from what employees say to the coach.
Six findings every AI startup should check
Each finding comes with the rule behind it, the question to ask yourself and the fix.
Provider or deployer? The team could not say
What we found. Lumo’s creator is the provider. But in Lumo for Teams a new party appears: the employer whose HR team uses it, which is a deployer. Each role carries different obligations.
Regulation (EU) 2024/1689, Art. 3(3) and 3(4) define provider and deployer. Art. 25 also makes you a provider if you put your name on a high-risk system, modify it substantially or change its intended purpose.
For each AI system you ship: are you the provider, the deployer or both?
A one-page inventory: every AI system, what it does, whose model it runs on and your role for each. It is the document every investor ends up asking for.
The assistant never says it is an AI
What we found. Onboarding says “Meet Lumo, your coach”, with a human name and a human-sounding voice. Nowhere in the first interaction are users told they are talking to a machine. The team argued it was obvious. It is not obvious to a 63-year-old who signed up because their doctor suggested it.
Art. 50(1) and 50(5): systems that interact directly with people must tell them they are dealing with an AI, unless it is obvious, clearly and at the latest at the first interaction. Applicable since 2 August 2026.
Would a first-time user, not you or your co-founder, know at first contact that this is a machine?
One line at first interaction. Not buried in the terms or the FAQ. It is the cheapest item on the list and the one most often missed.
Nothing it generates is marked as AI-generated
What we found. The voice notes are synthetic audio and the weekly recap is generated video. Neither carries a machine-readable marking. “It is an AI app, it is obvious” stops working the moment a file leaves the product and travels alone.
Art. 50(2): providers of systems that generate synthetic audio, image, video or text must mark the outputs, in a machine-readable format, as artificially generated or manipulated. There is a narrow exception for assistive editing that does not substantially alter the input.
If someone saved one of your voice notes and forwarded it, could a machine tell it was synthetic?
Mark at generation, not at export. If you shipped before 2 August 2026, the transition for technical marking ends on 2 December 2026.
It infers emotion, and the data source changes the rules
What we found. Lumo reads tone from voice notes. The Regulation ties emotion recognition to biometric data, and voice is biometric. Free text, on the current reading, generally is not, although it is still personal data and what you infer from it can be special-category data.
Art. 3(39) ties the definition to biometric data. Art. 50(3): deployers of emotion recognition systems must inform the people exposed to them and comply with data protection law (GDPR).
Are you inferring emotion from a biometric, like someone’s voice, or only from text?
Split the feature in your documentation: different data source, different legal analysis, different disclosure. Do the GDPR work either way.
Nobody could say where the training data came from
What we found. Lumo was fine-tuned on user conversations and on a scrape of wellbeing blogs and forums: no record of sources, no check for rights reservations, no documented consent basis. Two problems live here: whether the AI Act obligations attach to you, and plain copyright, which applies either way. And what people tell a wellbeing coach about sleep, anxiety, eating and mood is health data, with voice adding biometrics on top.
Art. 53(1)(c): providers of general-purpose AI models need a copyright policy and must respect text-and-data-mining reservations (Art. 4(3) of Directive (EU) 2019/790). GDPR Art. 9: special-category data needs a specific Art. 9(2) basis, not just acceptance of your terms. GDPR Art. 35: an impact assessment is mandatory for large-scale processing of special-category data.
Where did your training data come from, and could you produce the list tomorrow if an investor, a user or a regulator asked?
Start the record now, even if it is incomplete. A partial, honest data-provenance log is worth far more in due diligence than a confident shrug.
The B2B pivot they were most excited about is banned
What we found. Lumo for Teams infers employees’ emotions from their voice, inside a workplace, and reports them to the employer. That is not a high-risk system to document and mitigate. It sits in Article 5: it is a prohibited practice.
Art. 5(1)(f): AI systems that infer the emotions of a person in the workplace or in education are prohibited, except for medical or safety reasons. Applicable since 2 February 2025.
Does anything on your roadmap touch a workplace or a classroom?
Redesign it or drop it. No compliance paperwork makes a prohibited practice lawful. Better to find out now than in the data room.
What the fines look like
| Breach | Maximum fine |
|---|---|
| Art. 5: prohibited practices | up to €35 million or 7 % of worldwide annual turnover |
| Other obligations, including Art. 50 | up to €15 million or 3 % |
| Incorrect information to the authorities | up to €7.5 million or 1 % |
| GDPR: a separate regime entirely | up to €20 million or 4 % |
These are ceilings, not forecasts. None of this is a reason to stop building; it is a reason to know what you are building. Most of what the review found is solved with a review, not a rewrite. For SMEs and startups, the lower of the two amounts applies (Art. 99(6)).
Source: Regulation (EU) 2024/1689, Art. 99; Regulation (EU) 2016/679, Art. 83.
The AI Act calendar after the Digital Omnibus
| Date and status | What applies |
|---|---|
| 2 February 2025In force | Prohibited practices (Art. 5) and AI literacy (Art. 4) |
| 2 August 2025In force | Obligations for general-purpose AI models (Art. 53) |
| 2 August 2026In force | Transparency obligations (Art. 50) |
| 2 December 2026Next | New prohibitions and end of the marking transition |
| 2 August 2027Later | Regulatory sandboxes in every Member State |
| 2 December 2027Later | High-risk systems, Annex III (previously August 2026) |
| 2 August 2028Later | High-risk systems, Annex I |
Dates as I presented them on 22 September 2026, after Regulation (EU) 2026/1744 (the Digital Omnibus) amended the AI Act. Dates can move, so check them before you rely on them.
Three rules already apply to you, whatever your size
None of the three cares how big you are, whether you have revenue or whether you have raised.
Prohibitions
Eight banned practices. Not “high risk”: banned. There is no compliance path and no paperwork that fixes it.
Transparency
Tell people they are talking to a machine. Mark what your model generates. Live since 2 August 2026.
General-purpose AI models
Applies to whoever provides the model, which raises the question of who, exactly, that is in your stack.
Six questions to ask about your own AI product
These are only some of the questions, but they are the ones that came up in the review.
- For each AI system you ship: are you the provider, the deployer or both?Finding 1
- Does a first-time user know they are talking to a machine, and is everything it generates marked as AI-made?Findings 2 and 3
- Are you inferring emotions from voice or face, or only from text?Finding 4
- Are you handling health, biometric or other sensitive data, and is there an impact assessment that would survive being read?Finding 5
- Does anything on your roadmap touch a workplace or a classroom?Finding 6
- Could you show tomorrow where your training data came from, and on what legal basis?Finding 5
From teardown to a plan your investors can read
If an investor, a client or a regulator is about to ask you these questions, I can run the same review on your product.
AI Act check
Your role, the risk level of each AI system and which obligations are already overdue.
Legal due diligence of an AI project
A review like the one in this talk, feature by feature, with a prioritised list of fixes.
AI due diligence for investors
For funds and angels: a legal read of an AI company before you invest.
Frequently asked questions
What was the AI Product Legal Teardown?+
A live legal review of an AI product that I presented at AI Tinkerers Valencia on 22 September 2026. It went through the product feature by feature, found six issues under the EU AI Act and the GDPR, and was selected for Top AI Demos #47, the global AI Tinkerers newsletter.
Is Lumo a real company?+
No. Lumo is a composite built for the demo from patterns that appear again and again in consumer AI products. No real client and no confidential information is shown.
Does the EU AI Act apply to a small startup?+
Yes. The Regulation sets no size threshold. Three rules already apply today whatever your size, revenue or funding: the prohibited practices (Art. 5), the transparency duties (Art. 50) and the duties of providers of general-purpose AI models (Art. 53). For SMEs and startups, the lower of the two fine caps applies.
What is the difference between a provider and a deployer under the AI Act?+
Under Art. 3(3) and 3(4), the provider develops an AI system, or has it developed, and places it on the market under its own name; the deployer uses an AI system in a professional activity. You can be both. Art. 25 turns you into a provider if you put your name on a high-risk system, modify it substantially or change its intended purpose.
Does a chatbot have to say it is an AI?+
Yes, unless it is obvious. Art. 50(1) requires systems that interact directly with people to inform them, and Art. 50(5) says clearly and at the latest at the first interaction. It has applied since 2 August 2026.
Is emotion recognition banned at work?+
Yes. Art. 5(1)(f) prohibits AI systems that infer the emotions of a person in the workplace or in education, except for medical or safety reasons. It has applied since 2 February 2025, and fines go up to €35 million or 7 % of worldwide annual turnover.
Where can I get this kind of legal review in Valencia?+
I am a lawyer in València (Valencia Bar no. 20213). I advise AI startups and companies online across the EU and in person across the province of Valencia. You can book a consultation through the contact page.
Who gave the talk
Miriam Acerete is a lawyer registered with the Valencia Bar (ICAV no. 20213) who advises startups and companies on AI compliance. She co-organises AI Tinkerers in València, where the AI community shows what it is building.
About meSources and links
- Regulation (EU) 2024/1689 (AI Act), EUR-Lex ↗
- Regulation (EU) 2016/679 (GDPR), EUR-Lex ↗
- Directive (EU) 2019/790 (copyright in the Digital Single Market), EUR-Lex ↗
- Top AI Demos #47, AI Tinkerers newsletter ↗
- Slides of the talk (Google Slides) ↗
- AI Tinkerers Valencia ↗
Also relevant: Regulation (EU) 2026/1744 (Digital Omnibus), which amends the AI Act, and Directive (EU) 2024/2853 (product liability).
Keep reading
This page is general information about a composite case, not legal advice for your situation. To find out which duties apply to you, see how I can help with AI compliance.
Let us run the teardown on your product
In a consultation we go through your product the way we did Lumo’s: your role, what it says and generates, where its data comes from and what an investor will ask. You leave knowing what to fix first.
Book a consultation