We upgraded our modx 2.7.3 to 2.8.6 (running on nginx and php5.6 fpm) and now the ProcessElementTags is running 400% slower than before
Step to reproduce
How to reproduce the issue, including custom code if needed.
Take an existing modx 2.7.3 installation, create a resource that calls processElementTags 900 times for example (I know not great), time the loading time.
Upgrade that modx to 2.8.6 and run the same page, and look at the loading time
Observed behavior
How it behaved after following the steps above. Annotated screenshots here are excellent.
Expected behavior
How it should behave after following the steps above.
Before the same amount of processElementTags calls was taking 9 seconds
Environment
MODX version tagged when creating the issue, Apache or NGINX with version, MySQL version, browser, etc. Any relevant information.
Modx 2.7.3 before
Modx 2.8.6 after
the rest below is the same as before and after
nginx version: nginx/1.14.0
mysql version: mysql Ver 14.14 Distrib 5.6.41-84.1,
os: Ubuntu 16.04.5 LTS
php: 5.6 fpm
I checked the code of the function modParser::processElementTags with git blame and it seems that nothing in this function changed in the last 8 years.
Literally nothing changed in the environment, I downloaded version 2.8.6, unziped, copied the core and setup folders over the existing modx, and ran the setup.
Is there anything else I can look at to investigate, because even if the code didn’t change, nor did the environment, the slowdown is there, it’s happening. The images are from newRelic which hasn’t changed either
If I read the printscreen you posted correctly, then most time is spent executing modScript::process. This function runs a snippet (or plugin) and hasn’t changed either.
Is there a more detailed breakdown further down (that’s not visible in the posted printscreen)?
That’s about as highest definition I can get to… but you can see the Custom/modX::getChunk call is taking around 86ms of that 81ms is the Custom/modChunk::process, of that 71ms is the Custom/modChunk::processElementTags which seems to be called twice.
You mentioned the code that runs a plugin hasn’t changed, but have the plugins changed? is there some kind of new plugin that’s been added, maybe some kind of debugger or verbose logging that might have been added which might not show up in the newRelic measurements?
Plugins aren’t part of the MODX core. They are added by extras you install or by yourself.
I couldn’t find any code that was changed in the MODX parser that would explain why your code slows down so much after the upgrade.
What you could try:
Install the extra pdoTools. pdoTools has its own parser (pdoParser) as an ‘enhancement’ of the MODX parser (modParser). pdoParser overrides the function processElementTags(), so there is a possibility that your code will run faster.
If the snippet that runs so slow on your site is a custom snippet that calls $modx->getChunk() multiple times, you could try creating your own function. Many MODX extras have their own getChunk() function that saves the chunk-content in a variable to save time when it’s called repeatedly.
Here’s an example:
Or use the getChunk function from pdoFetch (part of pdoTools) instead.
Thanks, Basically I am reworking my whole solution to not use any modx chunks. I have been using modx since 2014 and as you can see my tech stack is old. I was planning to move to a more modern modx stack later this year. However this upgrade fiasco has given me serious doubts in continuing using modx as a base system. So by moving to a non-modx templating system, I am one step closer to moving to another stack.
Are you really on PHP 5.6? MODX 2.8.6 will run on PHP 5.6, but a version at or above PHP 7.2 is recommended – (MODX Server Requirements).
I have noticed MODX 2.x core code that behaves differently in various earlier versions of PHP, so that might be affecting your times. I noticed a big decrease in some page-load times when I went to PHP 7+.
Hi Bob, the thing is that we had a stack set up by a modx cloud specialist in 2014 when we first built out this system. It was running on an older ubuntu and direct upgrading to php 7.2 wasn’t supported on that version of ubuntu.
When we did the upgrade in our dev environments the modx output modifiers wouldn’t work and we had lots of chunks with modifiers within modifiers, and they just didn’t work, it was going to be a huge job and we would have effectively had to rebuild the whole stack. We’re a non profit so it wasn’t a priority so we kept the php 5.6 fpm version.
Even now with our current setup, the output modifiers are flaky at best, but we have a stable system. Over the years I moved more and more of our templating to handlebars, which is a more robust and stable templating engine.
Our system has a API backend, so we have a passthrough script that effectively means that we pull data from the API to the browser and present it there. It means that our system can present a page a lot faster.
Before the upgrade it was at 11 seconds before the html would end up at the browser and afterwards it had to call a few ajax commands, after the upgrade it went up to about 44 seconds for the html to make it to the browser, now moving the rendering to the browser of those objects we have a page load time to the browser of 1475 ms, and then the with the ajax the page full time it takes to get to be in a usable state is about 13 seconds.
So the difference is considerable, even before the upgrade we should of done this ajax conversion of the last table of data, but as we are a non profit we haven’t had much bandwidth for performance improvements especially on pages that don’t get much usage.
That definitely makes sense, but the output modifiers are notoriously slow. So replacing them would make even more sense in your case. They’re also prone to flakiness due to the handling of cached and uncached tags (tags with, and without a ! token at the beginning). In that case, they may work right after the cache is cleared, but not later, or vice versa. In a number of MODX upgrades, the MODX parser is rewritten, which can fix some parsing problems but cause others.
The best solution, if you can do it, is to replace the conditional output modifiers with custom snippets, but if you have a lot of them, that could be a pretty big job. I personally don’t use the output modifiers – ever. I suspect that if you did this, though, you could upgrade MODX and PHP without all the problems. You could start with the ones that you see not working all the time.
Ditto. I have a client who had a script that accessed chunks with tons of output modifiers in them. It was taking the page 10 seconds to load. I rewrote the script to be 100% php-based, with no chunks, and now it loads in under a half second.
It’s frustrating that a key part of the CMS is so slow that we shouldn’t be using it. As a development platform it’s not very reliable. Templating is a key feature for any platform, so apart from using ppTools, is it possible to switch out the templating in modx in the platform?
I don’t consider the output modifiers to be a key part of MODX. And although I don’t use them, many people do so and consider them reliable if you don’t nest them too deeply and do get the caching right. There are also ways to make them much faster, even without PHP (the info at that link is from the primary MODX architect).
They are there partly for legacy/BC reasons, and partly to provide a way to allow people without PHP skills to add run-time dynamic logic to a page.
As for templating, I’m not sure what you mean. MODX has Templates that wrap each page each page, but MODX Chunks also serve as templates (often called Tpl Chunks). The Chunks can contain MODX tags, as well as placeholders that can be replaced by code in a snippet.
MODX itself uses Smarty as a templating engine for the Manager, I think some people use it in the front end.
I believe you could use your own templating engine of choice with MODX. I’ve never found the need, because PHP is almost always faster, but I’m sure there are people who have done it.
MODX Plugins allow you to alter pages before and after they are parsed and before they are rendered for the browser. So the code of a templating engine could be brought to bear on all pages, or any subset of those pages.
In my experience, you can do just about anything you want with MODX as long as you have the skills. MODX is an extremely agnostic platform that is built to give you complete freedom to use whatever tools and methods you choose.
I do use output modifiers for certain things, when I know they won’t slow down page loading, because they’re convenient. Everything in MODX has its place. The same site for which I rewrote that script also uses a lot of output modifiers, but they are brief and in places where the output can be cached.
