For founders
Case study: oversight of employee AI use
Shadow AI is the new shadow IT. A company's staff used dozens of AI tools and nobody knew what data went into them. I designed a governance console that shows which tools are in use, what leaves the building, and one place to set policy. The hard part was making oversight feel like help, not surveillance.
Prashant Kumar Singh · · 4 min read
In short
- The product shows three things: which AI tools staff use, what data goes into them, and one place to set and enforce policy.
- The central design decision was to make policy feel like help: default-allow with visibility, clear reasons for every block, and a request path for staff.
- Twenty screens on a full design system, built around a dense inventory table and a single policy editor.
A decade ago IT departments discovered that staff were running their work through unapproved cloud tools, and "shadow IT" became a category. The same thing is happening with AI, faster. People paste customer records into chatbots, upload contracts to summarisers, and connect browser extensions to their email, because the tools are useful and the company has not said what is allowed.
The client for this project, who is not named, had exactly that situation: dozens of AI tools in use across the company, no inventory, and a security team that could only guess at what data had left. I designed the console that gave them an answer. This is the design, with the decisions behind it.
The problem, precisely
Three questions nobody could answer. Which AI tools are staff using? What data is going into them? What should we do about it? The first two are discovery; the third is policy. Most products in this space do one of them well and bolt the others on. The brief was a console where the three were the same screen.
Decision 1: an inventory, not an alert feed
The obvious design is a security-style alert stream: every risky paste becomes an event. We rejected it. An alert feed makes the security team the bottleneck and makes staff into suspects. Instead the home screen is an inventory: every AI tool detected, who uses it, how often, what categories of data have gone to it, and its policy status. Dense table, fixed columns, sortable, with one accent colour for tools that need a decision. Discovery first, then judgement.
Decision 2: policy as help, not surveillance
This was the hard part. A console that blocks tools silently teaches staff to hide them. So three rules shaped the policy editor. Every block comes with a reason the employee can read and an approved alternative where one exists. There is a request path: a staff member can ask for a tool to be allowed, and the request lands in the same inventory with context. And the default for a newly detected tool is visible-but-allowed, with the data categories flagged, rather than blocked, so the company sees what people need before deciding.
Decision 3: one place to set policy
Policies are written once, per data category rather than per tool: customer records may not leave the company; internal documents may go to approved tools; public information may go anywhere. Tools inherit the policy from the data they are seen receiving. That keeps the policy page to a single screen and means a new tool is governed the moment it is detected, without anyone writing a new rule.
| Screen group | What it does |
|---|---|
| Inventory | Every tool, users, frequency, data categories, status, one accent for "needs decision" |
| Tool detail | Who, when, what data, request history, the policy that applies and why |
| Policy editor | Rules by data category; tools inherit; plain-language reasons shown to staff |
| Requests | Staff requests with context, approve or decline with a reason |
| Reports | What left the building, by category and month; what changed after a policy |
What was delivered
Twenty screens, a design system covering the dense table, status chips, policy cards and the request flow, in light and dark themes, with the data model documented so engineering could build the inventory from the detection pipeline directly. The product is in build and the client is not named. The scenarios in the designs are realistic categories of tools and data, not the client's real inventory.
Questions people ask
- What is shadow AI?
- The use of AI tools by employees without the company's knowledge or approval, in the same way shadow IT described unapproved cloud tools. The risk is confidential data entering tools nobody has vetted.
- What does an AI governance console do?
- Shows which AI tools staff are using, what categories of data go into them, and provides one place to set policy by data category, with reasons and a request path visible to staff.
- How do you design AI oversight that employees accept?
- Start with an inventory rather than an alert feed, give every block a reason and an approved alternative, default new tools to visible-but-allowed, and provide a request path so staff can ask instead of hide.
If you're a founder
If your product sits anywhere near monitoring, compliance or governance, the design question is the same: how does it feel to the person being governed? Send it for the free teardown.
If you run an AI lab or data team
The inventory-first principle applies to task programmes: before writing rules about what authors may do, see what they are doing and why. The guideline that comes out is shorter and followed.
Sources