MODX3 Extras, backward compatability and build-script compatibility

Development
1

I’ve been trying to catch up on the recent development changes as we’re getting closer to MODX3 release, and I noticed the documentation seems to be incorrect or not updated for building an Extra. For instance, it displays the same build scripts and articles for the 3.x branch and the Doodles tutorial. That tutorial does not work under 3x.

Some questions I have:

  1. Are build scripts backward compatible?
    1. In testing the Doodles tutorial, the “parseSchema” function seems to result in the new file structure and namespace usage. But that is then not compatible with “addPackage” it seems.
    2. I found that I can install my Extra in 3.0 and it works great, until it comes to the build process. So the manager page, connectors and processors are all compatible.
    3. I understand this is a Breaking Change, but just looking for clarity around Extra contributors and maintainers. The communications give the impression that you might be good to go until 3.2 or 3.3 when deprecated classes are fully removed. In my testing, it looks like that could be true if you kept a 2.x instance of MODX to do your Extra development on and run your build. Then install it to 3.x and test it, then publish it.
    4. Clear documentation on what should be backward compatible will allow reporting of bugs rather than reporting things that are not planned to be backward compatible.
  2. Are Docs updates and enhancements going to happen post launch?
    1. I’m sure this is an “it depends” type of answer based on community involvement, etc.
    2. If I can get some clarity around a vanilla simple MDX3 extra I can take on writing an updated tutorial for MODX3 docs. It might be time to retire “Doodles” and replace it with something else? Or just expand on that concept.
  3. Anyone have some clarity on the Autoload vs. the system property (extension_packages) that was used to include an Extra at instantiation?
    1. Does the “bootstrap.php” file replace the need to add the “extension_packages” system property?
    2. What would be the format to add a package and service from within a Snippet now? So if I only wanted my Extra to load when needed in the Snippet. From an efficiency perspective, we wouldn’t want it to load at instantiation.
  4. In my case I’d like to rewrite my extra to follow MODX3 and that version would require 3.0 minimum. Any suggestions on Github process, like forking vs. branching.
    1. I noticed that things like Tagger have a 3.x branch. But I would think that means when 3 goes live, it would be merged in. The assumption there would be that no further fixes would be supported on 2.x?
    2. If you forked the repo and made all the 3.x changes there, would that cause confusion anywhere?

I know I’ve asked a ton of questions here… sorry! Pick your favorite and answer whatever you want :slight_smile:

References:
https://docs.modx.com/3.x/en/extending-modx/tutorials/developing-an-extra

This one is 4 years old, but was helpful and works under 3.0.0-beta2. It was a little unclear on how to execute the snippets. I just added in a line to include the index from the site root so I had access to $modx then ran them from the command line.

And maybe most people already understand PHP namespaces and PSR-4, but I found this introduction understandable, and useful as an intro:

4 Likes
2

I don’t know the answers to any of your questions, but I wanted to say how much I appreciated seeing them asked having them put so well. As an extra author, I also have some questions, but your list has some excellent questions I hadn’t thought of.

3

Thanks Bobray, I appreciate the comment. I’ll post updates if I find answers elsewhere or figure things out. I think most people with the knowledge are hard at work on the meat and potatoes instead of the side dishes :wink:

4

Help with updating documentation (especially these older tutorials) is much appreciated - there’s an edit button at the top and bottom of each page in the documentation. :wink:

Under the Upgrading to 3.0 documentation you’ll find a reference of the things that changed (notably the breaking changes), although that’s not intended as a “how to upgrade your extra” documentation. It just really depends on the extra how complex it is to migrate, and for many extras it doesn’t even require any changes at all. There’s an issue here about this as well.

The key things I’ve personally found in my extras that need updating are type checks (if ($chunk instanceof modChunk) { ... }) and extended classes (customProcessor extends modResourceCreateProcessor). Those things are fairly straightforward to make cross-version-compatible by extending the checks to also include the 3.x class name, or adding a conditional class_alias() call.

It is important to note there’s a BC layer for class names in place that’s intended to be removed in 3.3. Detailed here.

As for your specific questions…

1. Build Scripts

In a way, yes they’re backwards compatible, but it might be better described as forwards compatible.

When building on 2.x it will install fine on 3.x. However, building on 3.x will not install on 2.x.

When building on/for 3.0+, build scripts may need to be changed a bit, in particular when referencing the classes directly. This probably needs to be updated in the documentation.

One of the cool things when you’re only building for 3.0+ is that you can use the xPDO 3 CLI to build schemas without a build.schema.php. I like to add it to composer.json, e.g. on a standalone xpdo project:

    "scripts": {
        "build-schema": "vendor/bin/xpdo parse-schema mysql sitedash.mysql.schema.xml src/ -v --update=1 --psr4=SiteDash"
    }

The model has `package="SiteDash\Model", so with that command it builds an xPDO 3 model into src/Model with namespace prefix SiteDash\Model. That command (the paths especially) will likely need to be tweaked for packages.

Yes, unless you rewrite the build script and code itself to be 3.0+ only. My personal focus is on cross compatibility for now, but you could release separate version lines for 2.x/3.x right now.

Documentation is a continuous and never ending project. Definitely very much dependent on community involvement. I’ve tried to document as many of the breaking changes as possible but there are thousands of documentation pages and I hope more people will be excited to help now we’re getting to release candidate stage.

That would be fantastic, and I’m happy to try and clarify anything you need clarified. In general it should just be a matter of updated class names. I have no strong opinion on Doodles vs something new, but if Doodles no longer works it makes sense to fix that to avoid similar but different sets of documentation.

It is indeed an alternative way to do that. The main difference is that extension_packages are specific to load a package (or service class, depending on the options) while the bootstrap is more of an open-ended entrypoint.

The namespace bootstrap can be used to load an autoloader, instantiate a service/package, preload stuff, etc.

For cross-version compatibility, the same getService/addPackage can be used.

For 3.0+, if you place the addPackage call in the namespace bootstrap.php you may no longer need the service class at all. getService will also still work the same, even though it has been deprecated and it’s worthwhile looking at injecting your service into the dependency injection container instead.

That’s up to you, tho I’d definitely suggest branches over forks.

Personally I’m going for cross-compatibility from a single code base for now (as long as possible) to avoid needing two separate release branches, but I can definitely imagine the decision to just deprecate the 2.x version and release an update for 3.0+ instead.

Wether your merge a 3.x branch into 2.x just depends on how you want to manage the extra. I do personally use and would suggest version-based branches (release-1.11, release-1.12, release-2.0; or just v1.11, v1.12) rather than the MODX version it supports.

1 Like
5

Thanks @markh! That is super helpful.

I’ll take a look at Doodles and updating the docs. I think there may be an additional documentation challenge in the sense that the main Doodles example could be updated to be 3x compatible only. But it might also be useful for a time to also document how you could achieve the cross-compatible package. I’ll look at both.

6

We do have version-specific documentation, so the 3.x docs could be for 3.0+ with a link/callout to the 2.x docs on how to do it the 2.x/cross-compatible way?

7

FWIW, people have struggled with the Doodles tutorial for years, and I don’t think the problem is with the docs. It’s just so complex that many people give up before getting it working. A simpler example might be more helpful, imo.

1 Like
8

@markh, do you have any tips on cross-compatibility?

Specifically:

  1. How are you implementing class checks like:
if ($doc instanceof modResource) {}
  1. What are you doing with includes and requires?
9

@markh

I’ve made good progress on understanding how the build process works in MODX3 vs previously. In digging into it I found that 2.0 supports something similar to the namespace concept and you can use “.” to create a directory structure for the output. That’s another topic though… on to my question:

Can you use “src” in your namespace? (namespace MyApp\src\Model)

  1. Is there a negative impact to including the “src” directory in your namespace? A common pattern for v3 appears to be a “src” directory to house all the class files and a “Model” directory within that. When building in a “project” folder in the web root for development, output structure we want would be the following. The “src” and “schema” folders would be direct children of “StoreManager”, or whatever your package is.

    • project1/
      • StoreManager/
        • src/
          • Model/
        • schema/

I found there are two ways to structure the schema and build script to get the desired output. This is based on the “package” and “phpdoc-package” attributes which work together with the argument provided for the “target” (or historically the model directory).

I wasn’t sure if “src” was included if it would somehow conflict with what the PSR-4 autoloading would be doing? I see in the MODX composer.json that it has a configuration that effectively creates an “alias” or namespace prefix that maps to the source directory.

"autoload": {
    "psr-4": {
        "MODX\\": "core/src/"
    }
},

I assume that if the “namespace” in the class files contained “\src” the composer.json file, if used, just wouldn’t need that mapping.

Option 1

If you pass the full directory structure, this works great. The only down side is that because MODX uses the “package” attribute as the namespace in your generated files, you have “MyApp\src\Model” instead of “StoreManager\Model”.

<model package="StoreManager\src\Model\" baseClass="xPDO\Om\xPDOObject" platform="mysql" defaultEngine="InnoDB" phpdoc-package="StoreManager\src\Model" version="3.0">

The corresponding parse and build calls do not leverage any namespace prefix option. I have these as separate files, just consolidated here to one.

// Define the schema file path and model paths
$projectRootDir = MODX_BASE_PATH . 'project1' . DIRECTORY_SEPARATOR; //{base_path}/project1/
$corePath = $projectRootDir . 'StoreManager' . DIRECTORY_SEPARATOR; //{base_path}/project1/StoreManager/
$schemaFile = $corePath . "schema" . DIRECTORY_SEPARATOR . "storemanager.mysql.schema.xml"; //{base_path}/project1/StoreManager/schema/storemanager.mysql.schema.xml

// Parse the schema to generate the class files
$generator->parseSchema(
	$schemaFile,
	$projectRootDir,
	[
		"compile" => 0,
		"update" => 0,
		"regenerate" => 1
	]
);

// Add the package
$modx->addPackage('StoreManager\src\Model', $projectRootDir);

echo(print_r($modx->packages, true));
if (class_exists("StoreManager\src\Model\smStore")) {
	echo("Class Exists after addPackage.");
	// Create the tables
	$manager = $modx->getManager();
	$manager->createObjectContainer('StoreManager\src\Model\smStore');
}
else {
	echo("Class does not exist after addPackage.");
}

Option 2 (The @theboxer method)

This may be the method someone else came up with, but I reviewed a few of John’s Extras like Git Package Manager, along with Tagger and Collections to figure out what he’s doing here. Looks like he’s been doing a fair amount of work to update his.

EDITED: Forgot to include the schema format. Also, the phpdoc-package and subpackage attributes don’t appear to have any functional purpose at this point. Setting it to a random string has no effect on MODX creating the class files and table. Everything is based on the “package” value.

<model package="StoreManager\Model\" baseClass="xPDO\Om\xPDOObject" platform="mysql" defaultEngine="InnoDB" version="3.0">

For this method you pass the root portion of your namespace as a “namespacePrefix” even though it’s really not. This seems to be a work around to get MODX to do something sensible, like allow us to NOT have nested directories with the same name :slight_smile: (ie: MyApp/src/MyApp/Model/).

Only the differences are shown here which is just the parse, addPackage, and build parameters. For parse we supply the prefix which MODX then removes from the “Package” value and then converts to a path.

For the addPackage call, MODX removes the prefix value from “StoreManager\Model” and then goes to the “src” directory, and adds on the remaining value “\Model”. This leaves us with a path of “…/src//Model”.

For the create call, we pass in the Class which now matches our namespace and does not contain the “\src” within it.

// Parse the schema to generate the class files
$generator->parseSchema(
	$schemaFile,
	$corePath . 'src' . DIRECTORY_SEPARATOR,
	[
		"compile" => 0,
		"update" => 0,
		"regenerate" => 1,
		"namespacePrefix" => "StoreManager\\"
	]
);

// Add the package
$modx->addPackage(
	'StoreManager\Model',
	$corePath . 'src' . DIRECTORY_SEPARATOR,
	null,
	"StoreManager\\"
);

$manager->createObjectContainer('StoreManager\Model\smStore');
10

Basically:

if ($doc instanceof modResource || $doc instanceof \MODX\Revolution\modResource) {}

Depends on the context; in many cases they may no longer be needed in 3.0 and you might do something like:

if (!class_exists('someClass') && file_exists('path/to/classfile/in/2.x') {
   include 'path/to/classfile/in/2.x';
}

I do not believe phpdoc-package has any impact on the actual generation or loading of model classes; that’s meant for documentation purposes (in case you use phpdoc for that).

The key bit in 3.x to use StoreManager\Model\ is the namespacePrefix, and setup an autoloader with your namespace and path (relative from core/components/yourpackage/) so that resolves correctly.

11

Ah, whoops. You’re right. The phpdoc-package is set on the model object, but doesn’t appear that it is ever used.

12

Thanks,

I’m wondering about the implication of this for extras that can run outside of MODX. Wouldn’t that mean you’d have tp require or include an autoload file (which I’m assuming is not necessary inside of MODX)?

13

If you’re using the standard way to load MODX externally that automatically includes the autoloader for 3.

14

Erm … I know more than one way to load MODX externally, but I don’t know which is the “standard” one.

15

I don’t know the “standard” either. And actually I think the docs are outdated and should be updated. The docs say that “MODX_API_MODE” is deprecated way back in 2.x. But it’s still in the index and still behaves as it should. It’s only used to NOT load the request handler.

I have a pull request out for an updated tutorial for this article in Docs: Using Custom Database Tables - Tutorials | MODX Documentation

Within that I’m using the API Mode and just including the index.php from the MODX root directory. I think that should be the standard really. It’s simple.

The below is from my build.tables.php file that is in the new tutorial.

<?php

// Set API Mode
define('MODX_API_MODE', true);

// Shortcut for directory separator
$ds = DIRECTORY_SEPARATOR;

// Include the index to load MODX in API Mode
@include(dirname(__FILE__, 3) . $ds . 'index.php');

/**
 * @var \MODX\Revolution\modX $modx
 * 
 */

// Classes to loop through
$classes = [
	'ToDo\Model\tdList',
	'ToDo\Model\tdTask'
];

// Get the manager
$manager = $modx->getManager();

// Loop through our classes
foreach ($classes as $class) {
	// Check if the class exists
	if (class_exists($class)  ) {
		// Create the table
		echo("Creating table for class: $class".PHP_EOL);
		$manager->createObjectContainer($class);
	}
	else {
		echo("Unable to load model class: $class".PHP_EOL);
	}
}

This is my first time contributing documentation… we’ll see how it goes :slight_smile:

1 Like
16

The docs here say that including index.php is deprecated (though I don’t know if that’s accurate).

Also, the docs on that page say to call getService(), which is deprecated in MODX 3.

17

Yep, I saw that too. The two methods are almost identical in functionality.

Both methods include vendor/autoload.php (slight difference of include vs include_once). It seems like loading the “old” modx.class.php file would be the deprecated method since that actually triggers MODX to load the include/deprecated.php class aliasesto map old class names to new namespace classes.

The “index” method creates the instance of MODX for you so that you don’t need that line of code. Both methods seem to most often initialize web as the context. That could be one reason not to use the “index” method. From a consistency perspective, it might be better to not have to change how you do it for certain Extras. But if the majority use “web” still seems like the preferred method from a simplicity perspective.

So, if you don’t need the deprecated.php functionality, then including the index would be fine. Although even here you could just include it after the index.

I definitely think more clarity on “why” would be great to support a particular method or approach. It seems like it doesn’t really matter. No matter which way you go you’ll need to do testing and validation. This only affects the build script and no other portion of the functionality. If you’re building on 3.x you’ll likely have to update your build script anyway and classes, etc.

18

I like the @markh idea of writing extras and their build scripts so they’ll run on both MODX 2 and MODX 3. Hopefully, he’ll enlighten us further on what he meant by the “standard way” to load MODX externally.

19

I was referring to the documented approach, altho reviewing that does make me wonder if that still works…

20

I know getService() is deprecated in MODX 3. I know I changed it in some code of mine, but I can’t remember if it actually caused trouble, or if it just generated a deprecation warning.