INDEXIA BLOG
How to index your book using AI
My co-developers and I have been thinking about how to use AI to index books.
The Challenge of Book Indexing
This was inspired by my own experience as a first-time academic author: six months ago, I had to prepare an index for an academic book that I coauthored, and was surprised both by how expensive it would be to hire someone to do the index and by the dearth of tools that could automatically generate one. Most of the indexing software that I saw available either used dated technology or used AI badly, generating outputs that were not sufficiently comprehensive and that required a huge amount of time to manually review.
Our Solution: Smarter AI Indexing
We've aimed to solve these challenges at Indexia. Rather than feeding an entire book into an LLM, we use thousands of focused LLM calls to read context from a book and in turn to extract its core indexable terms. We then enrich this information with summaries of how these terms are used to allow further AI systems to identify relationships between these terms (i.e., duplicative terms, cross-references, and subentries).
The Power of Context-Aware AI
This is a powerful tool. Automatically, our system can identify whether "Beagle" stands for the "HMS Beagle" or the dog, and when terms like "Pope" should be subentries of "Catholic Church."
Built for Verifiability
Our software is also highly verifiable. A core problem with AI systems is hallucination—we've solved for this in two ways:
-
Page-Level Verification: We ensure that every term our AI system extracts is verified by our software in the individual pages on which it is said to occur.
-
Direct Source Access: We allow you to instantly open up the source text to let you make a determination on whether the term or subentry was properly included.
Powerful Features That Save Time
We've added a few other features to Indexia to save you time:
Automatic Subentry Generation
Our system can automatically generate and assign subentry labels. We've been impressed by how helpful and robust those can be, and it can save authors much time in categorizing terms that appear in their work frequently.
Find Similar Terms
Our find similar terms feature can automatically pull up the terms that are most similar to the one that you're looking at, allowing you to draw connections between them far faster than you would be normally. We can automatically create groups of terms for you based on these similarities to make your categorizations go faster.
The Result: Quality + Speed
All of this has the potential to:
- (a) Generate a detailed, high-quality index on a first pass
- (b) Allow you to quickly make refinements to this first pass as needed to finalize your index
You can export those results to Word or .ixml at your convenience.
See It In Action
Want to see some examples of how it works?
Darwin's Origin of Species
Below is an index of On the Origin of Species, which has received zero human editing (you can compare ours to Darwin's own index if you'd like):
Large-Scale Document Indexing
Indexia likewise has the capacity to index much larger documents: earlier this week, we used a simplified version of our software (no generation of subheaders) to index the House Oversight Committee's 11/12 release of documents from the Epstein Estate. That project went mildly viral:
In that vein, you have the option to make public any index you create on the site—that is, to create an uneditable version of the index with a stable URL that you can share with friends, colleagues, or the public at large.
Happy Indexing!
Feel free to get in touch: admin@indexia.tech