At work, there were so many internal documents that we ended up developing a search engine. We considered Elasticsearch, but since we needed semantic search, I tried the OpenAI Embeddings API.
First, building the data pipeline took the longest. I split documents by chapter and converted each text into embedding vectors, but hit token limits and struggled with adjusting chunk sizes.
When I actually applied it, the results were clearly different from simple keyword matching. For example, searching for "cost reduction measures" would also match documents about "budget optimization."
As for cost, with so many documents, optimization was necessary, and I'm currently handling it by running batch jobs and introducing caching. Personally, this was a satisfying project.