Ideas for Arc XP

Build process & dependency manager

I’ll preface this by saying that while I have worked on some features, I may be widely ignorant of best practices and tools available, so maybe a lot of this exists already, or different tools exist that can solve the same problems. If that is the case, feel free to school me and send me RTFM.

So the basic idea is two folds.

Build process

Allow a “build” step for features. Could be Gulp, Grunt, Broccoli, Webpack, … whatever. Just allow “some form of processing”. It seems that right now, all we can play with is 1 JSP and 1 render.js. For anything but the smallest feature, this is fairly limiting and prevents any sort of code organization. In my wildest dream, I could write my code in ES2015, with modules, classes, fat arrows and all that goodness, pass it to Babel, and finally end up with a compiled render.js file. Similarly, I could write my CSS in SASS or Less, and just have it render as one style.css file in the end.

Dependency management

Once we have a build step, I think it would be nice to have some sort of dependency management at the global level. Most features (it seems to me), are still built using jQuery and string concatenation. Code is littered with statements such as:

if (window.localStorage) { do stuff }

It would be much nicer to have one lib in charge of “storage”, which would take care of the implementation details, so I can write this and not have to worry about the existence of localStorage or not:

store.get(..)

The issue is that I don’t see a simple way at the moment to maintain that kind of code, document it, etc. Large organisations tend to naturally evolve their own set of libraries, micro-frameworks and the like, and “some” way to make this code shareable and discoverable would be nice.

This is also true for external dependencies. In particular, at least one library in charge of “templating” would be a nice to have, so we can go beyond concatenating strings manually in our own code. I would personally like something a little more advanced, like React or maybe Knockout. There are smaller contenders if size is an issue. But something. Make a poll, ask people what they like, and make it available. Using this type of lib of course requires the “build” step above, as you would want to have separate files for templates, etc. Then each “feature” could simply say:

var react = require("react");
var moment = require("moment");
var store = require("wapo-localstorage");
...

and the dependency manager would take of building the final stuff, taking into account the needs of each feature, order of loading etc.

Risks

The risk of course, once you open the doors to more external libs, is to have them proliferate, and end up with 5MB of JS. Maybe we could just have a list of “blessed” dependencies and force dev to use those. Evaluating new ones as need pop up.

Conclusion

I’m sure there are a lot of technicalities involved, and this is just a suggestion. I guess all I want is simply access to “more modern” tools to write JS. I’m not attached to any particular lib, or builder, or dependency manager. I just want “something”.

Again, maybe some of this exists already, and I’m simply unaware of it. If so, it’s even better :)

Thanks!

  • Guest
  • Jul 17 2015
  • Will not implement
Categories Developer Tools
I need it... Not sure—just thought it was cool!
  • Attach files
  • Greg MacGregor commented
    December 18, 2015 15:28

    This a very attractive proposition to myself as well ... Big +1

  • Guest commented
    July 30, 2015 18:29

    Hey Luke,

    When I wrote this, I was unaware that you could "depend" on external files via the config file, and that dependencies would only be included once, even if required by multiple features. There are many pages on the wiki, that would seem to cover some of this, but they are empty:

    Similarly, on the external wiki, most pages are empty:

    (I know that writing documentation is a hard problem, and that in a very fast moving environment, it's difficult to dedicate time to it, so I'm not throwing the stone here. Just mentioning that this was a feature I was simply unaware off.)

    "Dpendencies" cover 80/90% of what I wanted to do, so that's sweet, and I'll start using this soon as I want to completely rewrite some of the features I inherited.

    I am still interested in some more advanced stuff, like maybe automatic ES2015 transpiling, but this is not super important. Plus it may make it more difficult for newcomers to make changes to existing code if they are not familiar with the new syntax. I could easily have a build script in my subfolder, and use a README and a git hook to compile locally before commit, but that seems cumbersome and error-prone.

    The other thing I was interested in was to use SASS or LESS for CSS. Is that a feature available as well?

    Thanks.

  • Luke Mason commented
    July 30, 2015 18:11

    The scope of this ticket is too broad for the product as a whole. Most of the process you discuss would already be possible, and in fact easy to do, inside the current pagebuilder configuration. If when setting up a process as a site team you run into limitations that are imposed by pagebuilder open a specific idea and explain what needs to be done.

  • Guest commented
    July 17, 2015 20:10

    Then again, I'm not expecting this to be built tomorrow. Just putting the seed of the idea out there, and seeing where that takes us..

  • Guest commented
    July 17, 2015 20:10

    I would add that this is also part of a "larger" idea: using modern tools allow to attract better talents. Good JS developers today want to work with stuff that is "sexy", not with gigantic codebase of string building jQuery. So I think the tools we use are important. Not only for technical debt, but also for the kind of people/developers you want to hire/keep.

  • Guest commented
    July 17, 2015 20:03

    Right. But I'm not talking about anything complicated. Even very simple modules, like "Post Recommends", can have somewhat complex code: https://github.com/WPMedia/PageBuilder-Features/blob/master/resources/contentfeed/build/postrecommends/contentfeed.js. This is the kind of stuff that I would like to rewrite/refactor in much smaller pieces. I can do it all in one file, but it's kinda tedious. Small files are more readable. Also, even if we don't use the advanced concepts of a library like React, it emphasizes writing very simple components, and building bottom-up. So in a component like this one, you have all sorts of decisions to make, design wise, based on the input, on how to render (ie what HTML). This could be abstracted into several hierarchical levels, so that each is individually simple to understand and verify.

    Using a lib would also simplify event delegation, inheritance, and all sorts of little things/annoyances that you find yourself in when you want to reuse someone else's code.

    Still staying with Post Recommends, it is built on "ContentFeed.js" which has its separate repo and generates 2 builds in PB. But because of the way its structured, you can't really make modifications to it without affecting the other consumer. If you have real components, classes, etc, then you can can modify only the pieces you need, while still sharing lower level components and or logic.

    I guess all I'm saying is that you don't need to be writing a full SAAS app to benefit from better tools. Even a seamingly modest module can use it.

    And this is a server side problem. Because in the end, you want to deliver 1 JS, and 1 CSS (or X chunks for performance). This is why major frameworks end up integrating this kind of tooling (think of Rails with the asset pipeline for example). If it can't be done by PB, I guess I can still add my own "build" file, a README, and a git hook to auto-build on commit. I mean its doable. I just think that there is a benefit to provide it in PB.

     

    Finally, once you have better tooling, you open the door to more  advanced stuff, such as using a transpiler (Babel) to write your JS in ES2015 syntax. This in turns improve developer productivity and drives best-practices.

  • Guest commented
    July 17, 2015 19:46

    "Build process" is definitely something that we are looking into. However, the spirit of PageBuilder is to create simple and small Features and combine them to create a web page. If you are building a complex Feature, maybe something that resembles a Single Page App, then you might be approaching things in a "wrong" direction. (Wrong is in quotes because it's only wrong from one perspective of PageBuilder.)

    There's nothing in PageBuilder that prevents you from a organization wide way of doing resource management. It's just that vanilla PageBuilder doesn't come with one way of doing client side resource management. (maybe it should.)

    I think the take away is that PageBuilder is first and foremost a SERVER SIDE rendering engine. It was never designed to be a client side framework that helps you build fully interactive pages like single page application.

    Maybe PageBuilder should somehow accommodate rendering sites that are primarily client side rendering application

    Hope this helps.