pdoUnique - Using pdoTools to return only unique/distinct values of a TV

I have a bunch of resources (about 200) each representing a house or flat for sale. They each have a TV which called “locTown”. Unsurprisingly, this TV stores which town the flat/house is located in.

On my home page I wanted to put a nice, clickable list of all the towns. I searched far and wide for a way to return only unique/distinct values of a TV (rather than a list which just repeated the same towns 20 or 30 times).

As I was using pdoTools I experimented a bit with the &where statement, tries to trick it into allow me to use a DISTINCT query but all to no avail. Thus began my quest to write a small utility snippet for pdoTools which does the job - and here it is:

<?php
/**
 * pdoUnique snippet
 *
 * Usage:
 * [[!pdoUnique?
 *     &parents=`2`
 *     &includeTVs=`locTown`
 *     &processTVs=`1`
 *     &uniqueField=`tv.locTown`
 *     &uniqueLimit=`8`
 *     &tpl=`@INLINE <li>[[+tv.locTown]]</li>`
 *     &limit=`0`
 * ]]
 */

// Extract snippet properties
$uniqueField = $modx->getOption('uniqueField', $scriptProperties, '');
$uniqueLimit = (int) $modx->getOption('uniqueLimit', $scriptProperties, 0);
$tpl = $modx->getOption('tpl', $scriptProperties, '');
$tplWrapper = $modx->getOption('tplWrapper', $scriptProperties, '');

unset($scriptProperties['uniqueField'], $scriptProperties['uniqueLimit']);

// Enforce raw data return for further processing
$scriptProperties['return'] = 'data';

// Fetch data using pdoResources
$data = $modx->runSnippet('pdoResources', $scriptProperties);

// Ensure data is an array
if (!is_array($data)) {
    return 'Error: pdoResources did not return an array. Output: ' . print_r($data, true);
}

// Deduplicate results based on uniqueField
if (!empty($uniqueField)) {
    $seen = [];
    $rows = array_filter($data, function ($row) use ($uniqueField, &$seen, $uniqueLimit) {
        $value = $row[$uniqueField] ?? null;
        if ($value === null || in_array($value, $seen, true)) {
            return false;
        }
        $seen[] = $value;
        return $uniqueLimit <= 0 || count($seen) <= $uniqueLimit;
    });
} else {
    $rows = $data;
}

// Load pdoTools for chunk rendering
/** @var pdoTools $pdoTools */
$pdoTools = $modx->getService('pdoTools');

// Render output
$output = '';
foreach ($rows as $row) {
    $output .= empty($tpl) 
        ? "<pre>" . print_r($row, true) . "</pre>\n" 
        : $pdoTools->getChunk($tpl, $row);
}

return $output;

Basically, it calls pdoResources and then sends the results though a loop which filters out the duplicates based on the TV field you specify, then returns what’s left over into whatever chunk you want.

Note that this is made somewhat easier by an under-documented feature in pdoResources called &return=data which will return the output of your snippet call as a raw php array - useful when you want to manipulate this output with another snippet.

Not the prettiest or most elegant code, but it works pretty well for me so far.

Happy to hear feedback and hints on how to improve it. Thx.

Thanks for sharing.


It would be more efficient to only return the unique values from the database (than removing the duplicates in PHP).

Using xPDO, the class xPDOQuery has a function distinct() that can be used.

For pdoTools, you could probably use the groupby property:

...
$scriptProperties['groupby'] = 'TVloctown.value';
$scriptProperties['select'] = 'MIN(modResource.id) AS id';
...

Thanks @halftrainedharry . I did spend some time messing about with &groupby in the pdoResources call but did not manage to get it to do what I needed.

Are you saying this could be done with just pdoResources alone using groupby and select? What would the call look like there?

This is the call I now use for pdoUnique:

[[pdoUnique?
&parents=2
&includeTVs=locRegion
&processTVs=1
&uniqueLimit=8
&limit=0
&tpl=@INLINE <div class="box-location hover-img item-[[+idx]]"> <div class="image-wrap"> <a href="[[~[[+id]]]]"> <img class=" ls-is-cached lazyloaded" data-src="[[+tv.image]]" src="[[+tv.image]]" alt="Some Description"> </a> </div> <div class="content"> <h6 class="text-white">[[+tv.locRegion]]</h6> </div> </div>
&uniqueField=tv.locRegion
&fastMode=1
]]

Yes I think this should work:

When I use this snippet tag

[[!pdoResources?
    &parents=`2`
    &includeTVs=`locRegion`
    &processTVs=`1`
    &limit=`0`
    &tpl=`@INLINE <div class="box-location hover-img item-[[+idx]]"> <div class="image-wrap"> <a href="[[~[[+id]]]]"> <img class=" ls-is-cached lazyloaded" data-src="[[+tv.image]]" src="[[+tv.image]]" alt="Some Description"> </a> </div> <div class="content"> <h6 class="text-white">[[+tv.locRegion]]</h6> </div> </div>`
    &fastMode=`1`
    &groupby=`TVlocRegion.value`
    &select=`MIN(modResource.id) AS id`
]]

pdoResources generates an SQL query like this (the list of IDs for modResource.parent and the TV-ID will be different on your system), that seems to work when I test it:

SELECT MIN(modResource.id) AS id, 
IFNULL(`TVlocregion`.`value`, '') AS `tv.locRegion` 
FROM `modx_site_content` AS `modResource` 
LEFT JOIN `modx_site_tmplvar_contentvalues` `TVlocregion` ON `TVlocregion`.`contentid` = `modResource`.`id` AND `TVlocregion`.`tmplvarid` = 3 
WHERE  ( `modResource`.`parent` IN (2,22,23,24,25,26,27,28,29,30) AND `modResource`.`published` = 1 AND `modResource`.`deleted` = 0 )  
GROUP BY TVlocRegion.value 
ORDER BY modResource.publishedon DESC

Sadly, the snippet call does not for me. :frowning: But appreciate the suggestion.

In any case, it would be awkward to limit the output to a set number of reults e.g. just return 8 distinct values rather than all of them. This is something that I found useful with pdoUnique.