Agent Vs. AI (Storywise)

Recently I encountered this new agent, coming from publishing, who is known to depend heavily on AI for her acquisitions analysis. Storywise is what she promotes, with a large online presence devoted to this topic.

(I’m writing this post myself but used some staccato cadence to the beats to convey surprise.)

Naturally, I was curious to compare Storywise with her treatment of my book! I paid for one month to get the full critique. Here I will share this bizarre experience with you, because it revealed a lot of interesting things and layers of themes intertwined. There is a LOT to unpack.

First, get this. Storywise engaged with the complex structure of my writing much better than the human agents.

That was NOT what I expected!!!

AI actually saw the machinery of the book where humans just experience confusion. The humans in the industry FREAK OUT that the writing and structure are more complex, and view them as “wrong” and needing to be fixed. Make it more simple to make us more comfortable, so we know what we’re doing. (Is definitely a THEM problem, not a ME problem.)

I. Strengths: (from Storywise)

“The manuscript’s strongest asset is its ambition. It attempts a full imperial system rather than a decorative fantasy backdrop, and that work shows in the palace hierarchy, the ritual life, the military structure, the prefectural politics, and the marriage system. The fusion of dynastic governance, sacred ceremony, and military strategy gives the book a distinct identity within epic fantasy.

“The best material comes when private family strain and state danger collide. Asher’s jealousy becoming a security concern, the courtship turning into a spy hunt, and the assault by the Dark Continent envoy all exploit the novel’s core strength: this ruler’s household is also the machinery of empire. The manuscript also understands spectacle. The Sacred Meadow, Emine’s workshop, the wedding rites, and the combat tournament are visually memorable.

“Several characters carry real narrative weight. Isarina is consistently presented as a ruler who thinks in systems, Asher has a usable insecurity-to-support arc, and Emine’s progression from guarded uncertainty to visible political and intellectual value is one of the cleaner growth lines in the book. The manuscript also has a strong instinct for institutional tension; councils, advisers, envoys, teachers, and generals all create pressure from different directions.”

HOLY SHIT!

Humans can express, “I’ve never read a book like this before,” but the AI just articulated WHY this book is unlike anything that’s been written before much more eloquently and accurately. If an agent were using Storywise to identify what they had in front of them, the praise for ambition should be the thing they are paying the most attention to, rather than ignoring. This should be where the right agent or industry editor leans in to ask: do I have the chops to execute that level of ambition? Not asking me to simplify something that would not work in a simpler form.

II. AI can diagnose some problems BECAUSE it has engaged with the complicated structure instead of trying to rebuild something different.

It first identifies repetition as the biggest structural problem: Isarina repeatedly returning to betrayal, family stability, and imperial fragility without enough development between iterations. That is a specific, testable editorial claim.

This is a claim I would be happy to work on with an acquiring editor or agent who has sophisticated enough editing skills to get as far as this AI has in identifying the real problems in the manuscript that need correction.

Then it identifies pacing drift through proportion, and importantly says the ceremony, explanation, and sexual ritual are not inherently misplaced. Its claim is that too much narrative time is allocated to them relative to forward movement.

Again: sophisticated distinction, potentially actionable without simplifying the architecture.

Then character differentiation: too many characters allegedly speak in the same polished explanatory register despite differences in age, profession, region, and temperament.

Whether it proves that adequately is another question, but at least we know exactly what claim we’d go test.

In the Human vs. AI test here, the AI is surpassing the humans so far. The problem is Storywise generates a five-page report, which you cannot challenge to get corrections on the massive sections that are problematic. In fact, this was a case where AI jumble-speak became so confusing I needed AI to interpret it for me even to find out what it was saying, and which parts were wrong!

THIS IS WHAT THE INDUSTRY IS USING TO TELL THEM PRESCRIPTIVELY WHAT SUBMISSION IS IN FRONT OF THEM!

III. AI can see the problem but cannot articulate what the fuck it means

Here is where a real drawback is that you cannot engage with Storywise the way you can with a chatbot to demand correction and clarification when it is wrong and obtuse AF. The idea of an agent using this prescriptively scares the fuck out of me.

There’s an old joke about therapy: if your relationship with your therapist is so dysregulating that you need a second therapist to help you deal with your first therapist, it’s probably time to fire the first therapist.

Same goes for AI. If you have an AI that cannot communicate with humans, and you need to bring in a second AI to interpret the first AI for you, why are you using that first AI in the first place?

In practice: Storywise would make a broad diagnosis like “the antagonist architecture is underbuilt,” “the middle sprawls,” “the ending compresses,” “critical explanations arrive late,” or “answers arrive secondhand.”

Those sound meaningful. But several of them don’t tell you what Storywise actually perceived in the manuscript that caused the diagnosis, what reader effect it thinks is failing, or what successful execution would look like. And there is NO INTERFACE to pressure test Storywise’s analysis to get correct answers out of it.

At this point, a human who has already been told to pay attention because they have rare, ambitious IP in their hands, but who doesn’t have the ability to engage with the manuscript themselves, just gives up. AI says stuff I can’t make sense of, therefore the manuscript is too confusing.

Except NO ONE would be able to make sense of some of what the AI is saying. Out of a five-page report, maybe there is one combined page of analysis that is clear enough to act on.

So far, here’s the feedback loop in action:

complex manuscript > human delegates comprehension to AI > AI produces opaque analysis > human experiences confusion > confusion gets attributed back to the manuscript.

Why is there so much authority given to a machine in this situation?

Specific example: “The antagonist architecture is underbuilt. X’s betrayal is important enough to power the whole final act, but the novel withholds [the] psychology and operational methods too long, then explains them in a late summary. That robs the conspiracy of force. The reader should feel X closing around the story long before Isarina kills him.”

Analysis: The first sentences appear like a real diagnosis, even though the analysis is missing some key structure in this suggestion. Across the whole report, it repeats and elaborates on this suggestion, but it never shows its work. The final sentence is not even grammatically correct. “Closing around the story” isn’t a correct phrase.

So an agent could easily take this at face value because it sounds like a correct diagnosis, delivered with full confidence, with no ability to pressure test it, require it to check its work, or get underneath it. The language performs authority while providing no analysis that would justify it.

This is not a tool providing real information for an agent or editor. Hell, this tool doesn’t provide real information for me, the writer, or MY AI, to make heads or tails of what might be actionable revision execution. What I can do with MY AI is pressure test the suggestion against the manuscript’s architecture to see if MY AI would propose any revisions.

Is that what I am supposed to be doing in an industry where Publishers say NO USE OF AI on their submissions pages?

IV. Storywise is not just opaque, but becomes demonstrably wrong.

From the early section that made it seem like Storywise had engaged with the complex architecture, it gave me optimism it would correctly be able to describe it. I was wrong.

So far, Storywise was able to map institutions, to identify WHAT I built, but when it came to how I built it, its capabilities ended.

It is able to correctly describe macrostructure, but not narrative execution, yet states both with equal certainty.

Specific example: “The manuscript also lacks line-level discipline in point of view and narrative mode. Most of the book reads as close or mid-distance third person, but then it abruptly shifts into intrusive omniscient commentary such as ‘Reader, imagine if you will,’ which breaks immersion and sounds like a different book. That kind of tonal fracture is not a stylistic flourish here; it is a control problem.”

Answer: Storywise collapses three different things into one diagnosis. “Mid-distance third” is not a POV category. Narrative distance and omniscience are different categories. There is no “abrupt shift” into omniscience, because the book is omniscient. An omniscient narrator can move into a character’s interiority, pull back from it, and make statements about general sweeping knowledge. All of these tools are in the toolbox available for crafting a commanding omniscient POV.

Storywise actually recognizes the evidence of omniscient POV and then uses that evidence to conclude that omniscience is an error. “Reader, imagine if you will,” doesn’t prove a limited narrator accidentally broke POV. It demonstrates and establishes the readers’ expectation that they are being told a story with an Omniscient Narrator.

My question to the agent using this: can you clearly identify the errors when AI delivers this all in the same confident voice, claiming authority it does not have? The human has to be literate enough about literature to be able to tell the difference.

In this case, Storywise delivered a five-page report that just filled in extra space around the same errors. The five pages could be easily summarized, and when it went into detail, none of it explained the reasoning behind the deliverables. It just expanded and explained them more.

As with my AI chatbot, if I’m using it for some project for work, it can start rambling on and on just to hear itself talk about whatever topic it’s fixated on. That is not concise or accurate information, just more of it in overwhelming amounts where volume can hide the flaws.

Storywise still requires a human more sophisticated than it to get it to stop mass-producing more descriptions of its errors, and challenge, question, and check where it is wrong. A human can use another AI to do all of this, because even the delivery of this high volume of information jumbles fact with fiction, and delivers incorrect sentence structure and phrasing sequences.

Are the agents investing that effort, or just skimming an AI-generated report as if it were human-generated and accurate, for the bottom-line takeaways they can use for their decision-making?