MODX3: broken Output Filter/Modifiers

General
1

Hello, I want to report something about my weird problems and discoveries with the new MODX 3 when using output-filters. Maybe someone has the same problems or hints?

It might have been a combination of things but now it works (solved, but not satisfactory).

I like MODX and think it is a very good CMF which is developed and maintained by many volunteers. This costs a lot of free hours. Thanks!

Due to the community and github I have a fresh MODX 3.0.1 install running without error logs. The annoying thing is that it runs smoothly with 2.8.x without these ā€œbugsā€ like in 3.0.1 and I some things cannot understand, e.g. square brackets: sometimes I have to double bracket (mosquito method) a query to make it work (see below).

I use MODX 2 for some private and community pages since this version exists and have always kept it up to date. Iā€™ve never had so many problems switching to a higher version as I have with the new version 3.0.x, therefore I keep my current 2.8.4 versions.

Now version 3.0.1 is out and I have installed it on a new apache server with PHP 8.1.11 and only a few snippets installed. Many snippets are here from the community, github or self created, which all run without problems on MODX 2.8.4. Yes, I am aware that there are problems with such major updates, also because of new PHP 8.1.

Especially I like the output-filters from MODX (one of my killer feature) which I use in many chunks, even nested output filters with chunks and snippets run almost without problems under 2.8.4, unfortunately this is not usable with 3.0.1.

I have found many links about this matter or similar and one is very promising (last link):

The last link fix works after changing the line 56 in modInputFilter.php (hope this will be included in MODX 3.0.2) and something like this works again:

[[*tvGalleryPath:notempty=`<div class="photos gallery">[[$webPhotos? &tvPfad=`[[*tvGalleryPath]]`]]</div>`]]

ā€¦ except for instance:

[[+fc_0_3:gte=`-99`:then=`[[$webJSWetterChart1]]`:else=`[[$webJSWetterChart2]]`]]

which works in 2.8.4, but 3.0.1 is simply rendered as text and not parsed.

Today I set the 2 chunks to uncashed and it works:

[[+fc_0_3:gte=`-99`:then=`[[!$webJSWetterChart1]]`:else=`[[!$webJSWetterChart2]]`]]

TV with notempty filter and chunk (mosquito method):

[[[[*tvRandPicture:notempty=`$webRandPictureRekursiv`]]]]

similar, but this works without mosquito method:

[[*tvGaleriePfad:notempty=`<div class="photos gallery">[[$webPhotos? &tvPfad=`[[*tvGaleriePfad]]`]]

All this is crazy for me and as mentioned above, this could be a combination of several factors, which I have never had with MODX 2.

mySystem:

  • MODX 3.0.1 installation on Linux Server (Ubuntu)
  • PHP: 8.1.11
  • mySQL: 8.0.30
  • Server API: FPM/FastCGI
  • new site with this

btw, the new homepage with MODX 3.0.1: bitburg-masholder.de

Best greetings from Bitburg (Germany)

2

Sorry, I had forgotten the code block and markh has fixed it. He was trying to reply to the post yesterday when it was marked as spam apparently by an automated system. Thatā€™s why he wrote me a personal message and now my post is unlocked again, tks to the AI :blush:

According to Mark is the fix planned to be included in 3.0.2 and should fixed all the parsing issues.

I can confirm it is much better with this fix, just strange for me that I have to set some chunks to uncached (!) or must use double square brackets (mosquito method) like this:

[[[[*tvRandPicture:notempty=`$webRandPictureRekursiv`]]]]

Thanks! Jo

3

Hey @krischel, as Iā€™ve prepared an alternative fix that should work more reliably in different use cases, would you be so kind to try Refactor input filter parsing to properly support nested tags by Mark-H Ā· Pull Request #16288 Ā· modxcms/revolution Ā· GitHub and report back if that works for your use cases that were broken?

4

Hi @markh, I think thatā€™s a bit better. Now I can use it again with cache like in MODX 2.8.x (as it should be). But for this I had to modify the call.

In my MODX 3.0.1 problem I use Chart.js to visually display weather data from the German Weather Service (my snippet without the chart: dwdWeather). For this I use multiple nested chunks like this (JS script included in template):

var ctx = document.getElementById('webJSWetterChart');
[[[[+fc_0_3:str2float:gte=`-99`:then=`$webJSWetterChart1`:else=`$webJSWetterChart2`]]]]

runs with MODX 3.0.1 and your fix, but I get this message in the Error Log:

ā€¦/modx/core/src/Revolution/modParser.php : 509) Could not find snippet with name [[+fc_0_3:str2float:gte=-99:then=$webJSWetterChart1:else=$webJSWetterChart2]].

This call works with MODX 2.8.x:

[[+fc_0_3:str2float:gte=`-99`:then=`[[$webJSWetterChart1]]`:else=`[[$webJSWetterChart2]]`]]

but the code is not parsed in MODX 3.0.1 with your fix and it is readable as text in the HTML source code.

Chunks webJSWetterChart1 and webJSWetterChart2 contain these lines which make problems for me:

        [[+fc_0_1:is=`00:00`:then=`[[$webJSWetterChartTPL00]]`:else=``]]
        [[+fc_0_1:is=`06:00`:then=`[[$webJSWetterChartTPL06]]`:else=``]]
        [[+fc_0_1:is=`12:00`:then=`[[$webJSWetterChartTPL12]]`:else=``]]
        [[+fc_0_1:is=`18:00`:then=`[[$webJSWetterChartTPL18]]`:else=``]]

These 4 templates contain JS code only and I changed it to:

        [[[[+fc_0_1:is=`00:00`:then=`$webJSWetterChartTPL00`:else=``]]]]
        [[[[+fc_0_1:is=`06:00`:then=`$webJSWetterChartTPL06`:else=``]]]]
        [[[[+fc_0_1:is=`12:00`:then=`$webJSWetterChartTPL12`:else=``]]]]
        [[[[+fc_0_1:is=`18:00`:then=`$webJSWetterChartTPL18`:else=``]]]]

(DWD provides some special data only at these times)

Yepp, it works! Unfortunately also with this errors:

ā€¦/modx/core/src/Revolution/modParser.php : 509) Could not find snippet with name [[+fc_0_1:is=00:00:then=$webJSWetterChartTPL00:else=]]. .../modx/core/src/Revolution/modParser.php : 509) Could not find snippet with name [[+fc_0_1:is=`06:00`:then=`$webJSWetterChartTPL06`:else=]].
ā€¦/modx/core/src/Revolution/modParser.php : 509) Could not find snippet with name [[+fc_0_1:is=12:00:then=$webJSWetterChartTPL12:else=]]. .../modx/core/src/Revolution/modParser.php : 509) Could not find snippet with name [[+fc_0_1:is=`18:00`:then=`$webJSWetterChartTPL18`:else=]].

I guess my syntax is wrong. These double square brackets (mosquito method) and the internal chunk without square brackets is probably more by accident that this works so? :innocent:

I donā€™t know, but maybe this report is helpful.

Thanks!

5

Thanks, Iā€™ll try to look into those cases in the next few days.

Just to confirm, is it possible [[+fc_0_3]] is not yet set when that gets called? That would cause the inner tag to end up unprocessed in the outer tag and cause that ā€œCould not find snippetā€ error.

6

Hi Mark, for instance this error:

ā€¦/modParser.php : 509) Could not find snippet with name [[+fc_0_3:str2float:gte=-99:then=$webJSWetterChart1:else=$webJSWetterChart2]]

I just checked the placeholder [[+fc_0_3]] which has now the value 11 Ā°C and [[+fc_0_3:str2float]] the value 11.0 (it seems fine for me).

This is the placeholder for the forecast minimum temperature, which is always available (if not, the value has -273.2 ).

btw: To be sure, after your hint I added a default filter to my query:

[[[[+fc_0_3:str2float:default=`-273.2`:gte=`-99`:then=`$webJSWetterChart1`:else=`$webJSWetterChart2`]]]]

Unfortunately this does not help either and the error log still appears.

Q: Is this syntax of the query basically OK? I mean the 2 times double brackets and the two chunks without double brackets?

Thanks for your time and investigation.

7

Yeah, thatā€™s the mosquito syntax.

However the error youā€™re seeing is that the outer tag [[ ... ]]] is getting processed first looking for a snippet with the name of [[+fc_0_3....]], which suggests the inner tag isnā€™t parsed before the outer tag is.

The only reason I can think itā€™s doing that is because it doesnā€™t have the value for [[+fc_0_3]] when it first finds it, with it being set (for example by an uncached snippet) at another time. It may still render correctly because the unreplaced tag stays there for a bit until it does get the value or when it cleans up unprocessed tags, but it does cause these errors.

8

OK, thanks for your answer. Yes, calling only the placeholder or the chunk works, but not together with an if/then query.

9

Hello Mark, today I updated to MODX 3.0.2 and tested all my Output Filters, unfortunately I still have the error message in my Log.

I can us [[+fc_0_3:str2float]] or [[$webJSWetterChart1]] or [[$webJSWetterChart2]] without Error-Log but not in a nested chunk like:
[[ [[+fc_0_3:str2float:gte=-99:then=$webJSWetterChart1:else=$webJSWetterChart2]] ]]

Therefore, I also assume that as you mentioned, the placeholder value [[+fc_0_3]] will not be found the first time, so I included a small snippet to read it first and et voila, it works without error message :slight_smile:

[[ [[dwdChartNumber:is=1:then=!$webJSWetterChart1:else=!$webJSWetterChart2]] ]]

I need this filter to use 2 different charts, if the value of [[+fc_0_3]] is greater than -99, chart 1 should be used, otherwise chart 2.

Thanks for the hint!

10

A pity, I celebrated too earlyā€¦ I need to run the snippet without cache and so I get the same error messages in my Log.

(ERROR @ .../core/src/Revolution/modParser.php : 509) Could not find snippet with name [[!dwdChartNumber:is=1:then=$webJSWetterChart1:else=$webJSWetterChart2]]

So I use again:

[[ [[+fc_0_3:str2float:gte=-99:then=$webJSWetterChart1:else=$webJSWetterChart2]] ]]

This placeholder wants to piss me off, it works, but fill my Error Log.

11

Hi, here is my solution via snippet (runs since yesterday without error message):

<?php
$output = '';
$fc_0_3 = $modx->getPlaceholder('fc_0_3');

$number = abs(intval($fc_0_3));

if ($number == 273) { 
    $output = $chunk = $modx->getChunk('webJSWetterChart2');
  } else {
    $output = $chunk = $modx->getChunk('webJSWetterChart1');
}

return $output;

btw: use abs value so that I do not have to query with minus numbers (but this does not matter in this case)

1 Like
12

I have a very similar problem using MODX 3.0.2-pl:

There is a snippet which is called uncached within a template. In a certain condition the snippet sets the placeholder

$this->modx->setPlaceholder('fields_hidden', '1');

The both nested placeholders phA and phB are filled with different output.

In my template I have this part

[[!+fields_hidden:is=`1`:then=`
    [[!+phA]]
`:else=`
    [[!+phB]]
`]]

The problem is none of the both nested placeholders are rendered!

Whatā€™s even worse is - when I set

$this->modx->setPlaceholder('fields_hidden', '0');

0

is rendered to the output!

Using only one of the nested placeholders ([[!+phA]] or [[!+phB]] without the condition) works just fine! So they both have content!

This all works just fine in MODX 2.8.x! Shouldnā€™t this problem be already fixed in MODX 3.0.2?

13

Do the placeholders phA and phB contain single square brackets?

There is a bug in MODX 3.0.2 that affects output modifier and square brackets:

14

Yes it does!

This is the content of phA:
(the gongroups[] part should be the problem)

<fieldset class="gongrpfieldset">
    <label>
        <input type="checkbox" name="gongroups[]" value="3" aria-describedby="gongrp3Help">
        Newsletter
        <small id="gongrp3Help">(News from our site)</small>
    </label>
</fieldset>

I applied the fixes from @markhā€™s pull request in modInputFilter.php but it still doesnā€™t work!

15

To make it more clear - here is the full (real) code from template which fails:

[[!+fields_hidden:is=`1`:then=`
    [[!+grpcatfieldsets]]
`:else=`
    <fieldset>
        <legend>Newsletter Topics</legend>
        <div class="label[[!+error.gongroups:notempty=` gongrpfieldserror`]][[!+error.goncategories:notempty=` goncatfieldserror`]]">
            <p>Please choose the newsletter topics you are interested in.</p>
            [[!+error.gongroups]]
            [[!+error.goncategories]]
        </div>
        <input type="hidden" name="gongroups[]" value="">
        <input type="hidden" name="goncategories[]" value="">
        [[!+grpcatfieldsets]]
    </fieldset>
`]]
16

I canā€™t reproduce this.
The PR from Mark fixes the problem when I test it.

17

I run a few test:

$this->modx->setPlaceholder('fields_hidden', '1');

Test 1:

[[!+fields_hidden:is=`1`:then=`
    hidden
`:else=`
    visible
`]]

Output:
visible

Test 2:

[[!+fields_hidden:is=`1`:then=`
    hidden
`:else=`
    []
`]]

Output:
(no output)

Test 3:

[[!+fields_hidden:is=`1`:then=`
    hidden
`:else=`
    [
`]]

Output:
(no output)

Test 4:

[[!+fields_hidden:is=`1`:then=`
    hidden
`:else=`
    ]
`]]

Output:
]

Here is the content of my modified core/src/Revolution/Filters/modInputFilter.php:

<?php
/*
 * This file is part of the MODX Revolution package.
 *
 * Copyright (c) MODX, LLC
 *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 */

namespace MODX\Revolution\Filters;

use MODX\Revolution\modElement;
use MODX\Revolution\modTag;
use MODX\Revolution\modX;

/**
 * Base input filter implementation for modElement processing, based on phX.
 *
 * @package MODX\Revolution\Filters
 */
class modInputFilter
{
    /** @var modX A reference to the modX instance. */
    public $modx = null;
    /** @var array An array of filter commands. */
    private $_commands = null;
    /** @var array An array of filter modifiers. */
    private $_modifiers = null;

    /**
     * Constructor for modInputFilter
     *
     * @param modX $modx A reference to the modX instance.
     */
    function __construct(modX &$modx)
    {
        $this->modx = &$modx;
    }

    /**
     * Filters a modElement before it is processed.
     *
     * @param modElement|modTag &$element The element to apply filtering to.
     */
    public function filter(&$element)
    {
        /* split commands and modifiers and store them as properties for the output filtering */
        $output = $element->get('name');
        $name = $output;
        $splitPos = strpos($output, ':');
        if ($splitPos !== false && $splitPos > 0) {
            $name = trim(substr($output, 0, $splitPos));
            $modifiers = substr($output, $splitPos);


            $chars = mb_str_split($modifiers);
            $depth = 0;
            $command = '';
            $commandModifiers = '';
            $inModifier = false;
            $skipNext = false;
            foreach ($chars as $i => $char) {
                if ($skipNext) {
                    $skipNext = false;
                    continue;
                }

                switch ($char) {
                    // `[[` indicates the start of a nested tag, which increases the depth.
                    // The character is added to either the command or the commandModifiers
                    case '[':
                        if ($chars[$i + 1] === '[') {
                            $depth++;
                            //$depth === 0 ? $command .= $char.$char : $commandModifiers .= $char.$char;
                            $inModifier ? $commandModifiers .= $char.$char : $command .= $char.$char;
                            $skipNext = true;
                        }
                        else {
                            $depth === 0 ? $command .= $char : $commandModifiers .= $char;
                        }
                        break;

                    // `]]` indicates the end of a nested tag, which decreases the depth.
                    // The character is added to either the command or the commandModifiers
                    case ']':
                        if ($chars[$i + 1] === ']') {
                            //$depth === 0 ? $command .= $char.$char : $commandModifiers .= $char.$char;
                            $inModifier ? $commandModifiers .= $char.$char : $command .= $char.$char;
                            $depth--;
                            $skipNext = true;
                        }
                        else {
                            //$depth === 0 ? $command .= $char : $commandModifiers .= $char;
                            $inModifier ? $commandModifiers .= $char : $command .= $char;
                        }

                        break;
                    // The `=` sign (equals) is the separator between the command and its modifiers
                    // The character is only added to the modifiers when we're inside a nested tag;
                    // otherwise the command name would include the `=` sign when opening the modifiers
                    case '=':
                        if ($inModifier) {
                            $commandModifiers .= $char;
                        }
                        break;
                    // The ` sign is either the start or end of a modifier.
                    // However, we may encounter a ` inside a NESTED tag, which we need to leave alone.
                    // That's why only when we're at the ROOT of the tag we toggle the inModifier flag.
                    // The character is also added to the modifiers to preserve nested tags.
                    case '`':
                        if ($depth === 0) {
                            $inModifier = !$inModifier;
                        }
                        else {
                            //$commandModifiers .= $char;
                            $inModifier ? $commandModifiers .= $char : $command .= $char;
                        }
                        break;
                    // The `:` sign (colon) is a separator between multiple commands in a string.
                    // We may also encounter it inside a NESTED tag, in which case need to preserve it.
                    // At the root level, this saves the command and its modifiers and resets it
                    // for the next pass to process further.
                    case ':':
                        if (!$inModifier) {
                            if (!empty($command)) {
                                $this->_commands[] = $command;
                                $this->_modifiers[] = $commandModifiers;
                            }
                            $command = $commandModifiers = '';
                        }
                        else {
                            $commandModifiers .= $char;
                        }
                        break;

                    // Any other characters are plain strings and thus need to be added to either
                    // the modifier or root command we're currently processing
                    default:
                        $inModifier ? $commandModifiers .= $char : $command .= $char;
                        break;
                }
            }

            // After a pass over the entire tag, make sure to save the last command and its modifiers if set.
            if (!empty($command)) {
                $this->_commands[] = $command;
                $this->_modifiers[] = $commandModifiers;
            }
        }

        $element->set('name', $name);
    }

    /**
     * Indicates if the element has any input filter commands.
     *
     * @return boolean True if the input filter has commands to execute.
     */
    public function hasCommands()
    {
        return !empty($this->_commands);
    }

    /**
     * Returns a list of filter input commands to be applied through output filtering.
     *
     * @return array|null An array of filter commands or null if no commands exist.
     */
    public function & getCommands()
    {
        return $this->_commands;
    }

    /**
     * Returns a list of filter input modifiers corresponding to the input commands.
     *
     * @return array|null An array of filter modifiers for corresponding commands.
     */
    public function & getModifiers()
    {
        return $this->_modifiers;
    }
}

Could it be that there is anything else what needs to be changed in MODX source - not only this fix?
I have the origin MODX 3.0.2-pl source with this one fix!

EDIT: sorry, I missed one changed line! Iā€™ll check it again and will report back!

18

It looks like you missed 1 change from the PR (on line 79).

Just copy the whole new file modInputFilter.php to core/src/Revolution/Filters without trying to make all the changes manually.

1 Like
19

Jep, just saw itā€¦ sorry!
Iā€™ll report back if this works.

20

With @markh 's fix It works now!

Sorry for stealing your time! I first didnā€™t want to replace the full file as I didnā€™t know if there are other changes (previous commits) included which could be problematic.