Too many nested apply-templates calls Error When Running a Transformation
Problem
I'm getting the error message Too many nested apply-templates calls when I try to transform my DocBook file to HTML using default Oxygen XML Developer plugin DocBook to HTML transformation scenario.
Solution
Most likely, this is the result of a masked stack overflow error that can be solved by increasing the stack size (-Xss) to 4MB. Try setting a new VM option with the value -Xss4m. You can try to slowly increase this to larger values (e.g. -Xss5m or -Xss6m). Note that this consumes memory on a per thread basis (Oxygen XML Developer plugin can have tens of threads), so using a very large value here can backfire and leave Oxygen XML Developer plugin without memory.