Trying Google's NotebookLM, Part 2

Our most popular newsletter here at Machines on Paper was my quick and dirty review of NotebookLM, Google’s new AI product that promises to let you upload a series of documents and query them. 

When I wrote that piece in late 2023, the app was very much in an Alpha state, and many of the features didn’t work too well, if at all. Even so, I clocked it as a potential breakthrough AI application and encouraged our dear readers to give it a spin. 

Nearly a year later, the perception of NotebookLM has gone in a direction I... uh, did not quite expect. When people mention it online, it’s mostly to rave, gawk, panic or despise a feature called Audio Overview that turns your documents into a super-realistic sounding podcast.

To me, this feels like a classic Google product thing: No one knows where to put the cool new thing, so they shoehorn it onto some other thing, and over time, normal people don’t have a clear picture about how the product is actually supposed to be used. But I have no evidence of that beyond the last 15 years or so of Google. On the other hand, at least people are talking about NotebookLM! [If you have time, do check out the link above. Simon Willison’s take on this is typically charming] 

Digressions aside, let’s get to my new test of Notebook LM. First, I’ll remind you of the rules:

1. I’m going to write this out live in the style of an ongoing, well, notebook. AI tools should be fast and efficient if normal people are going to take the time to upend their proven routines, and I’m going to write this as fast as I can to give you a sense of how NotebookLM does in that regard. 
2. What I Have: A set of detailed notes in fairly random order. These notes include research links, summaries of my main points, and freehand analyses that need to be re-written into legible sentences and paragraphs. I also have a few MoP posts of mine that qualify as think-pieces. 

My Needs: I want to write a blog post about highly-studied periods in the history of journalism, like the Progressive Era. I’m hoping to build out my research notes before publishing something to the public. In addition, I want to turn this material into a podcast on AI in Media for some reason.

First, I’m prompted to upload my sources. This interface has been cleaned up since last year. 

However, I’ve run into a small snag. As a weirdo LARPing as a normal human, I definitely assumed NotebookLM would accept .docx files. That’s a shame. I’ll just paste my work into a Google Doc though.

Now that I’ve uploaded my notes, I get this overview page that summarizes my doc and asks what I need help with.

This is crazy good, especially the suggested questions part. I’m used to seeing AI crank out a decent summary, but these suggestions questions lap the field – In particular LinkedIn’s anemic attempts to use AI to prompt conversations on public posts. 

So, I’m working on turning this into a presentation, and I suppose “briefing doc” under the Help Me Create header is the closest fit. 

Selecting that option surprisingly took me away from that page, and, after about 40 seconds or so, generated this briefing: [I later found that “* Notebook guide” on the bottom right of the screen above takes me back to the overview. Not very intuitive.] 

The headers offered by the Notebook LM document analysis are: Main themes, Key Ideas and Facts, Quotes, Key Questions, and Call to Action.

Thinking about the UX prompts that led me here, it’s clear that Notebook LM is more targeted at studying and preparing research artifacts for my bosses, or for school. But maybe not so much for turning thoughtful, non-presentation-ready documentation into a killer pitch. And that’s okay! Every product should have a clearly targeted audience, after all. 

At the bottom of the page, we have a nifty chat to quiz an AI against my document. This is the pure RAG element that should tie AI responses directly to the documents I provide.

Essentially, what you’ll get in response are topical remixes of the documents in question. This will be super useful if, say, your grad school professor or boss has overloaded you with reading and you just need the gist of the docs. 

The real trick to using NotebookLM well is knowing how to ask the right questions. My advice: Ask NotebookLM how a really hard or impossible problem might be solved, even if you already know it has no proper solutions. There, Google’s LLM tech will really shine, and neatly gather a host of pointed evidence in its attempt to lead you to a true answer. If you want to use this for work or school, that’s the kind of thing that will give you practically useful material for a high performer, and not just the stock responses for someone who has kind of scanned the documentation. 

For example, I asked NotebookLM: How can journalists make sure their reporting is always presented in the proper context? 

It’s not all flowers and gumdrops (whatever those are) for NotebookLM, however. Adding a second document leads to a bit of an adventure: Does the Notebook Guide now describe both documents or just the first one? Totally unclear.

Turns out you need to choose to create another “Help Me Create” option (FAQ, Study Guide, etc.) before the guide updates, even though the text field tells me there are already two sources and the app hasn’t told me otherwise.

Okay, okay, let’s get to the main event and make a weird creepy AI podcast. I’ll now add URLs from a bunch of our “Paper” content, and see what we get. 

I’ve chosen a bunch of newsletters about the automation of sports writing, including AI content scandals at Sports Illustrated, along with thoughtful pieces on preserving some standard of quality written material on the internet in an age where the economics of AI seems to discourage putting unique effort into editorial presentation. 

Note that you can’t give Audio Overview any specific instructions about how the generated podcast should deal with the material. You get what you get.

Point taken, I’ll be back after a late lunch… 

Notebook AI's Podcast On My Research

0:00

/1026.384

1×

I’m pretty proud of this sandwich I made, but now I feel like my entire being has been run through a banality machine? This podcast has given me an existential crisis from which I may never recover. You know what, let’s just stop the post now, I’m going to go play video games and eat Haribo Star Mix.