MODX Community

Importing 3rd party content. Where to Start?

I received a request to pull in data from another location (intranet specifically) and use that to update content within the site in a few places:

  1. A sorting method to filter a list (front end)
  2. A TV listbox (multi select) (back end)

Currently, this is a manually created & updated list using MIGX in the back end. Ideally, I’d like to make sure that list is updated regularly from the external source. Then, I’d like to pull them in to individual resources from an external source too.

For example: Say I have 500 people who list what schools they’ve graduated from in their biographies. Rather than manually keeping this ever-changing list updated manually, I’d like to pull this from another source (via a feed or something) to keep our main site synced with our intranet. Then, i’d smilarly pull into each person’s resource page that list of schools they specifically attended.

If this is possible, what would be the best way to go about this?

thanks!

I think you’re looking at one or more custom snippets that gather the data and update things, possibly with a cron job that runs the job at a set interval.

The method for gathering the data will depend on how it’s currently kept on the external source. It it’s in HTML, this and/or this could be helpful.

Is it possible to pull this content into the database from a CSV file?

So it looks like i may have to use CSV files. So suppose that our 3rd party system spits out csv files and uses FTP to add them into a directory. It looks like you can read CSVs into a mySQL database with something like this: https://phppot.com/php/import-csv-file-into-mysql-using-php/.

Let’s say also that this would update employee biographies, which are populated by many TVs. How would I even start to read a directory of CSV files into the DB, create new resources if that user ID didn’t already exist, modify one if it already did exist and publish them all automatically?

Thanks!

Take a look at xPDO. In MODx you can relatively easy create/manipulate resources.

$r = $modx->newObject('modResource');
$r->set('parent', $parent);
$r->set('pagetitle', $pagetitle);
$r->setContent($content); 
...
$r->setTVValue('myTV', $tv_value); //setting a TV
$r->save();

You would need a unique identier for the employees and store that somewhere in the resource, so that you can query first, if a resource for this employee is already available.

1 Like

Hmm that makes some sense to me. So yeah each person would have an employee ID in there already. Like mine could be jsydor1 and my brother Jon could be jsydor2. We already have that as a TV called ‘employeeID’.

So could I query all employees with PDOresources or something to get an array of currently used employeeIDs and then reference that to see if it already existed?

[[pdoresources? 
   &parents=`xxx` 
   &tpl=`@INLINE [[+tv.employeeID]], ` 
   &includeTVs=`employeeID`
]]

You probably could use pdoResources or you could do the query yourself (with xPDO):

$query = $modx->newQuery('modTemplateVarResource', array('value' => $employee_id,'tmplvarid' => $id_of_tv_with_employee_id));
$query->select('contentid'); 
$resource_id = $modx->getValue($query->prepare());
if (!empty($resource_id)){
    //resource already exists
    $r = $modx->getObject('modResource', $resource_id);
} else {
    //create a new resource
    $r = $modx->newObject('modResource');
}

If you use pdoResources, you’ll get back one big chunk of text that you would have to parse. I think using xPDO would be more appropriate.