resources
George LewisOperations4 min read

The pillars of quality for tech docs work for AI authors too

Ask an AI assistant to write a manual and it will; but it will do it differently each time. Use the information models, style guides and taxonomies your writers already have to control an AI assistant.

Ask an AI assistant to write a manual and it will; but it will do it differently each time. Use the information models, style guides and taxonomies your writers already have to control an AI assistant.

If you ask an AI assistant such as Claude, or one built into your customer support app to write your product documentation - it will. The text will read fluently, but it is unlikely to be what you expected. The structure will be wrong, the terminology will be incorrect, and each time you update, you’ll get a different document.

Researchers have measured how sensitive models are to the way a request is made. In tests on widely used models, they found that small changes in how a prompt was formatted swung results by up to 76 accuracy points.

We all know the standard advice: give the bot a clear prompt and plenty of context. Fortunately, technical writers already have that context. Information design assets such as an information model, a style guide with terminology, and a taxonomy define what the product documentation should contain, how it should read, and how it should be structured. These are best practice for a technical writer, and they should be best practice for an AI author agent too. If you can leverage the information design assets you already have, you are a long way towards controlling your author agents.

Consistency is a long-solved problem

Technical writers have treated consistency as a quality in its own right for decades. As far back as 1995, Shubert, Spyridakis, Holmback and Coney showed that procedures written in a controlled language, with defined style rules and an approved vocabulary, were significantly easier to understand when the procedure was complex, and that readers located information in them more easily.

Documentation teams deliver that consistency by following approved information design assets:

  • Information model
    Lists the sections and topics a publication contains, and what content and metadata each topic type must include.
  • Style guide and terminology
    Sets the sentence-level rules and names the approved terms for each concept, along with the terms to avoid.
  • Taxonomy
    Defines the product’s concepts and their relationships with each other.

Because the assets are agreed in advance, writers know what to produce and reviewers assess drafts against those same shared standards. The organisation gets compliance, and readers get consistency.

Treat AI author agents like a new member of the team

Anthropic’s guidance on prompting asks users to think of the model as “a brilliant but new employee who lacks context on your norms and workflows”. For a new technical writer, you would provide them the approved information design assets before they start. An AI author agent needs the same material.

Researchers at Universität Hamburg demonstrated this by generating code documentation either with basic prompts, or structured prompts that carried an output template and examples. The structured prompts produced documentation of “significantly higher quality … especially due to a consistent documentation format”.

However, copying and pasting the information design assets into prompts is error prone and inefficient - not conducive to an efficient, AI-enabled content supply chain. To realise the efficiencies, you need a governed source for the information design assets that your AI agents can easily access to get the latest versions of the assets, and to track what each agent received.

Managing information design assets in the AI era

Introducing AI agents into your content supply chain can provide efficiency gains. But as with introducing a new employee, you need to provide them with the appropriate tools to do the job, or you will spend more time fixing what they do than you save.

Well governed information design assets provide the guidance AI agents need. The question is how to make those assets easily available to the agents as well as the writers.

c8s-labs’ Content Ops Director enables documentation teams to manage their information design assets and make them accessible to AI agents. This provides them with the control overlay necessary to ensure documentation teams can leverage their existing assets to realise the efficiencies AI agents can deliver.

  • ai
  • information-design
  • governance
  • style-guides
Share
Back to resources