How We're Using AI in Our Work These Days (Practical Notes)
We've been integrating AI tools at the team level since last year, and I've gathered only the ones that actually worked. About half of what we adopted just because it was trendy has been scrapped.
1. Code review assistance
Run it yourself before opening a PR and preemptively fix the issues that would get flagged. You can really feel how much reviewer time it saves.
2. Document drafts
Design docs or meeting summaries. But you can't just use them as-is; terminology consistency has to be done by hand.
3. Log analysis
Paste in error logs and get a "list of likely causes in order of probability." I make the final call.
4. Generating test cases
For getting ideas for edge cases.
5. Internal search chatbot (RAG)
The more documentation a team has, the bigger the effect. But if you don't set a cadence for ingesting the latest docs, it can actually lose people's trust.
Overall: It's not a silver bullet; it's strong for repetitive, draft-type work. If you use it as-is where judgment is required, it causes problems.
6 answers
Agreed. Especially #1, the code review assistant really makes a noticeable difference. The number of times I get nitpicked by reviewers has dropped significantly.
Well, I'm a bit skeptical about log analysis. It's nice that it lists candidate causes, but when plausible-sounding wrong ones get mixed in, I've wasted time digging in the wrong place early on more times than I can count. Ultimately, it's not much different from the skill of the person reading the logs.
In the draft for Document 2, the part about “unifying terminology manually” is important, but people don’t really stick to it lol. A lot of people just pull the output, copy-paste it, and call it done. I put the whole team glossary into the prompt and run it, so it’s somewhat better, but final review still absolutely has to be done by a human. Since you end up spending on review the time you saved on drafting, the total benefit might not be as big as you’d think.
I agree with the overall take. I really agree that if you just use it as-is in places that require judgment, you're asking for trouble. But these days, everyone habitually slaps it on even where judgment is needed, so I can't help thinking that after about half a year we'll end up in a state where 'we can't work without AI, but we don't know why it got this way.' Productivity metrics are going up, but it feels like the team's fundamentals are eroding along with them.
I hadn't thought of extracting test cases—I should try that.
RAG chatbots can really be poison if you don't set a document refresh cadence. Our team also integrated one last year and almost scrapped it within three months, and that was exactly the reason. When you ask it about an old policy doc, it confidently gives the wrong answer, and a new hire believed it and relayed it to a customer, which caused a mess. Now, only after we agreed to assign document owners and run indexing once a week did it become barely usable. It really hits home that this isn't a tooling problem but an operations problem.