Review After Using Meeting Minutes Automation for 3 Weeks (Long Post)
On my team, we tried automating everything from meeting recording -> summarization -> action item extraction. To get straight to the conclusion: full automation is still too much to ask; semi-automation is realistic.
1. The Setup
- Meeting recording: the built-in recording feature of our video conferencing tool
- STT: one that does speaker diarization (this is the most important part)
- Summarization: extracting structured JSON with an LLM
- Storage: automatically uploading to a Notion database
2. Results After Actually Using It for 3 Weeks
The biggest takeaway was that STT quality determines overall quality. If proper nouns or internal terms are transcribed incorrectly, that hallucination carries straight into the summarization stage. In the end, I had to build a separate internal glossary and preprocess with it.
Fixing the schema made the summarization prompt far more stable. When I used something like "summarize this well," the format changed every time. If you specify fields (agenda / decisions / owner / deadline), it becomes at least usable.
Action item extraction requires human review. Anything where the owner is mentioned ambiguously is almost always wrong. So now we only extract them as "candidates" and have the team lead verify them.
3. What We Gained and What We Lost
What we gained: post-meeting cleanup time dropped from 30 minutes to 5. What we lost: we spent almost 2 weeks on initial setup.
And honestly, the moment people know they're being recorded, they become a bit more guarded in what they say. I still don't know whether that's a pro or a con.
Next, I plan to work on improving speaker diarization accuracy. If anyone has built something similar, please share your preprocessing tips.
9 answers
If speaker diarization doesn’t work, it’s just tedious grunt work, fr
Once that works, it suddenly gets way easier.
Which STT did you use? The speaker diarization quality varies so much depending on whether it's a conference room mic or a headset..
Preprocessing your internal glossary is honestly the key here. We just ran it as-is at first too, and things like 'he/she' and 'Project X' all got summarized incorrectly, which turned into a total mess.
In the end, we made a glossary CSV, registered it in the STT dictionary, and added mappings to the summarization prompt like 'write the following terms this way,' which greatly reduced hallucinations. On the other hand, we still needed someone to keep managing the glossary. It feels less like automation and more like transferring the work to someone else lol
Well, after trying it for 3 weeks, concluding that "semi-automated is realistic" seems a bit premature. Once you factor in review time, many teams actually end up with a net loss.
Can you show the time saved in numbers? Like how many minutes before and after per one hour of meetings. Without that, it just reads as a personal anecdote.
When uploading to Notion, matching action item assignee names against your company's internal account directory makes a huge difference in perceived quality. After all, 'Manager Kim,' 'Kim Minsu,' and 'Minsu' all get entered as different people. I got stuck on this for quite a while.
In the end, it's automation that requires human review lol That's honestly the most realistic approach.
But honestly, this is more of a team alignment problem than a technical one. Getting consent for recordings is part of it, and if we don't decide up front who's going to own review, nobody's going to look at it two months from now. Seems like nailing down the rules matters more than picking up the tool.
I really relate to this—vaguely mentioning who owns an action item is a mistake… The scariest part is being left in a “Who was supposed to do it?” state.
You did a really great job locking down the summary schema. I use it with a few things added on top:
- Low temperature (0–0.2)
- For each field, specify 'if absent, null; no guessing'
- Lock in 2–3 examples as few-shot
- If the meeting transcript is long, split it by agenda item before calling the model
Especially after adding 'no guessing,' the frequency of making up decisions that weren't there dropped sharply. To put it politely, I think reducing hallucinations matters more than increasing the automation rate. If you lose trust, nobody will use it.