SLMs & fine-tuning
Manuscript-to-XML Structuring Pipeline
Rules first, a 20B open model for narrow questions, and a guard against lost text.
This pipeline converts an author's Word manuscript into publisher-standard XML for a scientific publishing services firm, logging every decision. Rules do most of the work, an open 20-billion-parameter model answers narrow questions that code then verifies, and a content guard blocks any silent loss of author text.
The problem
A scientific publishing services firm converts authors’ submitted manuscripts and files into publisher-standard XML. The conversion is detailed: structure, tables, references, file roles, each checked against a formal definition. The costly failure is not a crash. It is silently dropping or altering the author’s text, and nobody notices.
I wanted a pipeline that is explainable. Every decision should be logged, and nothing the author wrote should disappear without the system saying so.
What I built
The pipeline is rules first. Ordinary Python (lxml, python-docx) handles everything that a rule can handle. Where a rule cannot decide, the pipeline asks gpt-oss-20b, an open-weight model on a RunPod serverless endpoint, a narrow question, and then code verifies the answer instead of trusting it. Answers are cached. References are checked against CrossRef, and the output is validated against the DTD.
A content guard is the central design idea: it blocks any silent loss of author text.
I also delivered three installable modules to the firm in November 2026, covering file identification, references and tables. Reference checking was tested on small samples: DOI resolution on 13 of 13, and retraction recall moving from 4 of 5 to 5 of 5.
The model here is a small open model used as a component, not a fine-tuned one.
How it performed
All 411 test articles build DTD-valid, and 423 automated tests pass. Against the firm’s own final files, the pipeline matched 27,646 article parts, 8,714 of 9,186 table parts (94.9%), 2,529 of 2,530 references and 189 of 189 file roles. An error study found 96.3% correct. A paid test used 5 articles and 225 model questions.
These are agreement figures against the firm’s finished files, which is a useful bar but not the same as a production audit. The three modules were handed over, and no deployment outcome is recorded. I will not name articles, journals or authors, because they are the firm’s unpublished material.
What I learned
Ask a model narrow questions and verify the answer. A 20-billion-parameter model is good at a clear, bounded question, and unreliable when asked to do the whole job. Putting it behind code that checks its output kept it useful and removed the need to trust it.
A guard against loss is worth more than a gain in cleverness. A converter that is 95% right and says so is safer than one that is 99% right and hides the remainder.
Comparing against the client’s own finished files is a hard, honest test, because the files encode years of their conventions. It also showed me where I fell short: tables are the weakest of the four measures I report.
Rules first is a cost decision as well as a quality one. Every question I did not need to ask the model was cheaper, faster and fully explainable.