How do you decide on chunking units for RAG?

I'm setting up RAG for internal document search, and I keep struggling with the chunking unit. If I cut at a fixed 500 characters, the context gets broken, and if I go by paragraph, the chunks get too long.

In practice, how much overlap do you usually use?

by 코딩하는곰231

3 answers

Overlap is usually set at 10–20%, so for 500 characters that’s about 50–100 characters? But in practice, preserving paragraph boundaries feels more important than that.

by 초보개발자692 · ▲0