Implementing webpack raw-loader as babel-plugin-macros
This weekend I was working on my side project to list and display all my future tech talks. This is an overengineered web app built using CRA and Spectacle, with a little bit of React Router (I'll write about it in another post). I want to be able to display talk information in both inside app and in GitHub (where I host the repository), so I decided to put talk title and description in
README.md inside talk directory to prevent duplication.
My original solution to this problem was using
preval.macro which uses
babel-plugin-macro. Using preval is straightforward. Preval allows me to write JS code with filesystem access that will be executed on build step (via babel). It's really cool, you should try it yourself.
Basically you just import the macro, use it via tagged template, and put the code you want to execute as string.
This works but I don't like the boilerplate of requiring
path. It's also not reusable because I can't just pass file path.
Then I remember about webpack
raw-loader. CRA uses webpack under the hood, so
raw-loader should work. Unfortunately CRA doesn't yet have a solution regarding importing raw text file. I know this is unsupported method, but maybe still worth trying.
First I need to install
devDependencies because CRA doesn't include it by default, then I need to use webpack loader require syntax because CRA doesn't allow me to extends webpack (for good reason). The result is slightly better than preval (in term of boilerplate).
This fails the build of course, because CRA enable eslint rule to disallow webpack loader syntax. I need to disable it via eslint comment. Not pretty.
Then I had another use case for importing code example to spectacle code slide. This is where it gets weird. Somehow, importing it using raw-loader causes some newlines to be removed unintentionally. This is really frustrating because I need to annotate the exact line number to highlight some parts of code. This inconsistencies of removing newlines means I need to know the line number AFTER it was removed. Not pretty.
Then I was thinking of creating my own
babel-plugin-macros to replace
raw-loader. I searched it first, but no luck.
It's time for another side project.
This is the first time I write macro using
babel-plugin-macros, so I didn't have a clue what to do. I do know AST a little, but still. Fortunately I have a lot of inspirations from Kent C. Dodds. He has a lot of repositories using babel macro. (of course, he was the author of
preval.macro codebase and learn basic macro creation, but the functionality is quite different, so not much to copy/paste (sorry Kent). Then I saw
import-all.macro codebase and it's pretty similar to what I want to achieve. A function accepting arguments that gets compiled to another thing.
First step of creating macros are making sure all references in codebase get replaced.
createMacro function that accept function with some parameters:
Our function will be given one argument (it's an object) that includes
babel. We use
references to perform AST transform for all macro references in the codebase.
state provides us with information regarding current processed file (we mostly interested with its filename, more on that later), and finally
babel is mostly used to create new AST via babel-types available as
babel.types (people usually rename it to
We want to create macro like this
That will be converted to this (depends on
So we know that the only valid usage is function call (or
CallExpression in the AST). First we need to process all references
Then we need to get function arguments We only need one because the function only accepts one argument. We also need to make sure that the file path is deterministic (not a variable where we need to know the value).
Reading file contents
After getting function argument, we need to resolve its absolute path. This is where filename from
state comes in handy. We can get directory of the filename using
path, and merge the path with the relative path from previous step. We can use full path to get file contents via
fs. We need to use sync method because macro needs to be resolved synchronously.
The last step is replace original AST. We'll use babel.types to create new AST. What we replace is the function call, with an expression containing string literal. We use
replaceWith method on AST node that we want to replace.
That's it. We only need a few lines of code to replicate webpack
Finally we need to add some tests to make sure that our macro works as expected. The easiest way to unit test macro is by using
babel-plugin-tester with snapshot enabled. We provides some code, and
babel-plugin-tester will write a new snapshot containing our code and our transpiled code. You should learn about Snapshot Testing if you haven't already. It's really cool, useful, and will probably saves you a lot of time testing something.
It will create this snapshot:
As you see in the snapshot, the macro works as expected.
You can see end result in my repository: https://github.com/pveyes/raw.macro
I was really amazed on how much we can do with babel, especially with babel macros. The possibilities are endless. I think I'm going to write another babel macro soon.