AI Tools
I gotta say that Simon Willison’s blog and videos are simply amazing. He gives such great explanations for what he does and makes things reproducible so you can follow along.
I came across this :livestream he did with Val Town’s founder, Steve.
Plethora of topics covered. Among them:
Shot Scraper
A command line tool that lets you do screen grabs from the command line; you can also screen scrape pages using javascript and then feed the results as inputs into his llm tool. He does this by loading up the site in a headless browser and then running javascript snippets to retrieve whatever you need from the page.
As an example, he scrapes Google results using his tool, then feeds it into an LLM to do summarization of the results.
RAG Explanation
Simon explains 2 ways that people do RAG.
- Using vector search engines (normal way)
- Use it with a full text search engine: you take the user’s question, use an LLM to turn it into a search query, run the search, give the LLM the results and then assemble an answer based on that.
For RAG, you don’t need a fancy model, just one that’s good at summarization.
He explains why RAG can be so difficult. It’s because it’s impossible to predict the questions that users will ask. So if for example, if a user asks something that’s based on a date, and we haven’t coded our app to take into account this kind of query, (which should do a date-based search), we might fall back to a full-text search and not give good results.
The way to build good RAG is to come up with hundreds of potential questions and then build automated evaluations that you can run to ensure that your system is retrieving the correct content.