getPage Infinite scroll duplicate results problem

[dubbs]

Hi - I am using getPage to pagination a getResources call, and then using an infinite scroll to mean the pagination is on the same page rather than a bunch of pages as per code below… This works nicely… but… I am noticing that the results it outputs are often duplicates of the results already fetched on the first return of results… Any ideas why this is happening and how I can resolve it?

[[!getPage@custom? &pageNavOuterTpl=`[[+next]]` &pageNextTpl=`[[$ELEMENT_InfiniteScroll_customNextTpl]]` &element=`getResources` &limit=`16` &parents=`2` &resources=`-[[!GetRestOfWorkIndexProjects-ALL-NEWPAGE]]` &tvFilters=`ELEMENT-2019Project-WorkIndexSelections==%779%` &tpl=`ELEMENT-2019Work-Grid-ProjecTpl` &showHidden=`1` &pageNavVar=`pagination` &includeContent=`1` &includeTVs=`1` &processTVs=`1` &tvPrefix=`` &sortby=`RAND()`  &includeTVList=`ELEMENT-2019Project-WorkIndexProjectBlurb,ELEMENT-2019Project-WorkIndexSelections,ELE-2019Project-WorkIndexCoverIMG-ALL,ELE-2019Project-WorkIndexCoverIMG-EXPERTISE,ELE-2019Project-WorkIndexCoverIMG-EXPERTISE-CAT1,ELE-2019Project-WorkIndexCoverIMG-EXPERTISE-CAT2,ELE-2019Project-WorkIndexCoverIMG-EXPERTISE-CAT3,ELE-2019Project-WorkIndexCoverIMG-EXPERTISE-CAT4,ELE-2019Project-WorkIndexCoverIMG-EXPERTISE-CAT5,ELE-2019Project-WorkIndexCoverIMG-INDUSTRY,ELE-2019Project-WorkIndexCoverIMG-INDUSTRY-CAT1,ELE-2019Project-WorkIndexCoverIMG-INDUSTRY-CAT2,ELE-2019Project-WorkIndexCoverIMG-INDUSTRY-CAT3,ELE-2019Project-WorkIndexCoverIMG-INDUSTRY-CAT4,ELE-2019Project-WorkIndexCoverIMG-INDUSTRY-CAT5`]]

[[!+pagination]]

pageNextTpl is as below:

<div class="InfiniteLoaderLoadingMessage">
<img src="/assets2019/images/core/ajax-loader.gif" width="16" height="11" />
<p>Loading...</p>
</div>
<span class="InfiniteLoaderNextContainer">
<a class="loadMore" id="loadMore" href="[[+href]]">See More</a>

<script>
       // $('#loadMore').click(function(){
        $('#loadMore').on('click',function(){

$('.InfiniteLoaderLoadingMessage').addClass('showME');


            // gets url from clicked button
            var moreURL = $(this).attr('href');
           // calls url to get new data
            $.get( moreURL, function( data ) {
                // inspects data for desired elements, so that whole page doesn't render
                var newData = $(data).find('.page-content').children();
                // inserts desired data after existing content
                $(".page-content").append(newData);

$('.InfiniteLoaderLoadingMessage').removeClass('showME');

            });
            // removes button and script so that there are not duplicates, as ajax call brought in a new button and script after loaded elements
            $(this).parent().remove();
            
            return false;

// call hover jquery in 2019/custom.js
workIndexRollOvers();

event.preventDefault();

        });
</script>
</span>

The snippet GetRestOfWorkIndexProjects-ALL-NEWPAGE is a list of values which this getPage/getResources call should not display.

Any ideas?

[markh]

Probably because you’re sorting results randomly:

&sortby=`RAND()`

It may not seem as random because getPage will cache results one-page-at-a-time, but any next results that is requested will have a different order than the initial requested results.

[dubbs]

OK thanks Mark - will give it a go without Random sorting… see if that fixes it… I’d like it to be random though…

[markh]

It is random at the moment - but that does mean it will randomly include results that may have already been used before. Kind of a “computer says no” situation where you can’t have one thing, and also another.

[dubbs]

OK - yes works without the Radom sort by… any idea how I can achieve a random order whilst maintaining this to work as it does currently?

[nuan88]

It seems to me you should be eliminating dupes prior to sorting, or heck after sorting works too.

Could this be handled on the output side? You’ve had multiple duplicates issues and it *should be trivial to remove them

I guess what I am saying is the duplicates are a separate issue from anything else, including sorting

[dubbs]

It seems to me you should be eliminating dupes prior to sorting, or heck after sorting works too.

Could this be handled on the output side? You’ve had multiple duplicates issues and it *should be trivial to remove them

Thanks - but how can I eliminate the duplicates? Its a bug in getPage I think rather than something I am doing - I’m just using getResources to display a set of results from a defined parent, displaying them in paginated format using getPage - but rather than via separate pages, rendering them all on the same page using an AJAX call to call them in that way.

[markh]

It’s really not a bug. When you ask for a random sort, the order is different each time the snippet runs so it can’t know what was already used. Random + pagination without seeing duplicates just doesn’t exist.

You’ll have to rethink how you want things to be ordered to make this work.

Perhaps you can sort it on menuindex, or a published date, but that might not be what you want.

Or, you could use a different snippet that first returns all possible resource IDs in a randomised order. Then you then pass to the snippet as a FIELD sort, which basically defines a hardcoded order:

&sortby=`FIELD(id, [[getRandomlySortedIDs]])`

The snippet return something akin to 5,2,9,3

With that, pagination should work without the duplicates, because the order is consistent (altho not based on any particular field) no matter the request. If you want to periodically switch up the order, you could use the getCache snippet or implement a custom cache with an expiry into the snippet.

Looks like you already use a snippet for the resources to include ([[!GetRestOfWorkIndexProjects-ALL-NEWPAGE]]) - maybe you can reuse that for the sort as well.

[dubbs]

OK thanks MArk - will see if I can get that approach to work

[dubbs]

Bingo! works a treat. Created new snippet with title “getRandomlySortedIDs” - then called it in getResources call as per

&sortby=`FIELD(id, [[getRandomlySortedIDs]])`

Snippet code:

$resource = $modx->getObject('modResource', 2);
if($resource->hasChildren()) {
$children = $resource->getMany('Children');
}
$resourceIds = array();
foreach ($children as $child) {
$resourceIds[] = $child->get('id');
}

// randomise ids
shuffle($resourceIds);

$output = implode(',', $resourceIds);
echo $output;

[markh]

In a snippet, always use return and not echo. Consider a snippet a PHP function which needs to return its value to the caller, while with echo its position is fairly arbitrary. It may work in some cases but not in others.

You can also boost performance by using the getChildIds method rather than looping over resources yourself. With $modx->getChildIds, the cached resource map is used.

Also simplifies the code:

$resourceIds = $modx->getChildIds(2);
shuffle($resourceIds);
$output = implode(',', $resourceIds);
return $output;

[dubbs]

$resourceIds = $modx->getChildIds(2); shuffle($resourceIds); $output = implode(’,’, $resourceIds); return $output;

Thanks Mark - I am not a PHP chap - so good to learn your method was more efficient // thanks