Goals

So… my last day of working on this was interrupted by the battery in the display of my Surface Book deciding to bulge, requiring me to go through a several day process of safely backing it up, finding my replacement, and upgrading my replacement to the latest Windows 10.

The good news is I did recover a lot of context. The bad news is that’s about all I did.

Tonight’s goal is to get the new syntax highlighting plugin for kramdown to do some syntac highlighting (probably just turning all text red).

Process

1. Run bundle gem kramdown-syntax-textmate from the repository’s parent directory to create the gem’s skeleton.
2. Adjust the skeleton to be more like kramdown-syntax-coderay since that’s a known working plugin.
3. Modify the code copied from the CodeRay plugin to just wrap the text in a span with color red
4. Get Jekyll to use the new gem for syntax highlighting
5. Do a bunch of debugging
   • Try to use exceptions to track when things were being loaded (no dice)
   • Learn about kramdown’s warning system and how to enable that in Jekyll
   • Discover that I needed to add the gem to the Jekyll “plugins”, which is really just a code word for additional gems
   • Discover that the requiring of the gems requires the root file in the lib matching the gem name
   • Discover that I needed to remove the Jekyll Markdown cache (.jekyll-cache/Jekyll/Cache/Jekyll--Converters--Markdown)
   • Discover that I needed to call add_syntax_highlighter
   • Debug the string I was returning
6. That’s it! Today’s goal has been accomplished with 019491e2245610f1fd05261d2bb6786ad6d68d3d

BONUS

Since I still have some time left while I wait for my laundry to finish (sheets, can’t go to sleep without them), I’m going to find the updated code that VS Code uses to do its syntax highlighting.

It looks like things have moved around a bit, but not too terribly.

• grammar loading
• rust grammar
• parsing the grammar appears to start here
• with the actual parsing code being in vscode-textmate
• coloring happens here
• TextMate’s tokenization is here
• This is where it looks like the loaded grammar gets converted into an IGrammar for use with the tokenization
• And the grammar registry takes us back to vscode-textmate’s Registry

It appears that really the core of the language grammar processing is in vscode-textmate, from the grammar reading and parsing, to the tokenizing. The only piece I haven’t found yet is the part that converts the tokens into CSS styles.

• The classes are consumed here, so they have already been assigned by the time _renderLine() is called
• I’m pretty sure the classes are not part of vscode-textmate
• It looks like the transform from tokens to class names happens here
• lineTokens.ts seems to be the file that does the transformation
• The actual translation seems to handled through shared metadata between VS Code and vscode-textmate
• It looks like the controlling aspect is the theme

I’m going to put a pin in the theme for now, but it looks like I’ll want to rename this syntax highlighter from TextMate to VS Code, as although the source grammar files are in the TextMate format, the processing that happens using them seems like it is core to VS Code.

I think that makes for a good stopping point. I did some code reading, some actual coding, some debugging, and made a lot of progress. Next time I’ll start with the syntax highlighter rename (which should be pretty fast) and then start on the grammar processing.

Test code

this is some rust code