Szymon Nastaly

When working with libraries that are either new or not widely used, LLMs don't really know about their APIs. Especially when starting to use some API and trying to get a general understanding of the APIs capabilities it would be so helpful to have some kind of LLM support or semantic search.

Update 2025-04-15: In the most recent episode of the How about Tomorrow? podcast, Dax and Adam talk about exactly this topic (timestamp: 05:22). Dax is thinking about fine-tuning an LLM on recent versions of open source software (such as SST v3 or Tailwind V4), because he tries to fix the bad LLM-experience of new users trying to use SST v3 (which is barely represented in the LLM dataset). Adam proposes to create a MCP documentation server for LLMs to interact with open source documentation. But according to Dax, this would probably only help slightly (based on some manual experiments of his).

Some relevant topics: RAG, Anthropic's Contextual Retrieval (September 2024), and more recently DeepSearch/DeepResearch (February 2025)