There’s very little you can’t do in MODX. WebP images by themselves should work fine for browsers that support them. IE does not, and older versions of other browsers don’t, but current browsers almost all do (with the possible exception of Safari). That said, photo display extras like Gallery, may not handle webP images without some modification and you’d have to change the MODX System Setting (upload_files) to allow you to upload them in MODX.
You can use HTML to show WebP images to browsers that can handle them and fall back to .jpg, or .png images as a fallback using the trick described here.
Make sure your images are being displayed directly or cached rather than being pulled in by a snippet that’s called uncached.
Keep in mind that the slow performance may be due to your server, especially if you’re on shared hosting.