Add support for Magik

[sebastiaanspeck]

Closes #4008

[netlify]

✅ Deploy Preview for dev-prismjs-com ready!

Name	Link
🔨 Latest commit	40364d1
🔍 Latest deploy log	https://app.netlify.com/projects/dev-prismjs-com/deploys/69d4f17413568e000895367e
😎 Deploy Preview	https://deploy-preview-4020--dev-prismjs-com.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[github-actions]

No JS Changes

Generated by 🚫 dangerJS against 40364d1

[sebastiaanspeck]

@LeaVerou Is there something we can do for getting this merged? Does a plan exist, when this might be done?

[DmitrySharabin]

@sebastiaanspeck, I'm on it. I'll be back with the review results shortly.

[DmitrySharabin]

Foremost, thank you for putting effort into adding a new language to Prism!

After reviewing this definition against the official Prism extending docs, I've spotted a few issues worth addressing.

The docs state:

“All tokens before a greedy token should also be greedy” and “the greedy tokens should be the first tokens of a language.” (see https://prismjs.com/extending#greedy-matching)

Currently, non-greedy tokens (keyword, builtin, boolean) sit between greedy ones (comment and char), which violates this rule and can cause subtle tokenization bugs.

I'd suggest moving all greedy tokens to the top of the grammar, followed by all non-greedy tokens.

But before reordering the tokens, we need to remove greedy: true from some of them:

• No characters in a number literal start another greedy token, so there's no need in greedy: true.
• Moving operator before keyword and property before operator in the grammar eliminates the need for greedy: true in those tokens.
• Since Prism matches the text against tokens sequentially, we need to adjust the order of some operators within the operator array: move the compound assignment patterns (_and<<, etc.) before the plain << and \^<< patterns. With that change in place, we don't need greedy: true there either.

In short, I'd recommend the following order of tokens in the grammar:

1. comment (greedy)
2. char (greedy)
3. string (greedy)
4. regex (greedy)
   --- Remove all greedy: true from everywhere in the following tokens ---
5. property
6. operator
7. keyword
8. The rest: builtin, boolean, variable, symbol, number, punctuation

Please verify that with all the proposed changes in place, Prism highlights the language correctly.

[sebastiaanspeck]

Thank you for the review. I will have a look if those changes still reflect the same thing as it does now.

[sebastiaanspeck]

I am sorry, we had several fixes in v1, but didn't port them to v2. Did that now, but now it probably violates the docs again. Would you be so kind to help me review this PR to get it to a correct state?

[DmitrySharabin]

Let's try to apply the suggestions from the previous review first. That might help us move further.

I'd incorporate the fixes from v1 into the PR after applying the results of the first review. Then, we can bring the changes from v1 one by one and place them into the correct places right away. In case of doubts, we can discuss what the best place for them would be. What do you think?

[sebastiaanspeck]

Let's try to apply the suggestions from the previous review first. That might help us move further.

I'd incorporate the fixes from v1 into the PR after applying the results of the first review. Then, we can bring the changes from v1 one by one and place them into the correct places right away. In case of doubts, we can discuss what the best place for them would be. What do you think?

Thank you for your corporation and plan. Will do!

[sebastiaanspeck]

Let's try to apply the suggestions from the previous review first. That might help us move further.

I'd incorporate the fixes from v1 into the PR after applying the results of the first review. Then, we can bring the changes from v1 one by one and place them into the correct places right away. In case of doubts, we can discuss what the best place for them would be. What do you think?

Did that now. If the PR looks okay, I will add the fixes from v1.

[DmitrySharabin]

LGTM! The checks seem to pass (ignore the lint check; I know that it fails). Let's move on to the next phase.

[sebastiaanspeck]

I just pushed a commit which contains all fixes/enhancements from v1. Some tests are still failing, but I hope those will work out after we fixed all issues.

[DmitrySharabin]

Operator compound assignment patterns: ordering fix

Per the extending docs, Prism scans all remaining text for each pattern in sequence. Since \^<< and << appear before the compound patterns (_and<<, **<<, etc.) in the array, << consumes the << portion from _and<< before the compound pattern gets a chance — splitting _and<< into keyword(_and) + operator(<<).

The fix is to reorder compound patterns before \^<< and <<.

Also, line 49 is a duplicate of the pattern on line 48 but missing _mod and _div.

See inline suggestion below.

[DmitrySharabin]

Thank you for your patience. Really appreciate it. It is the first time I write a new language for PrismJS

Don't mention it. We're almost there. Don't give up! 😀

[DmitrySharabin]

I also asked Claude if something is missing from the language. It pointed out the _recursive keyword. I don't know if it's correct. Just a heads-up.

[sebastiaanspeck]

I also asked Claude if something is missing from the language. It pointed out the _recursive keyword. I don't know if it's correct. Just a heads-up.

Claude doesn't know anything about Magik 😉 _recursive isn't a keyword.

[DmitrySharabin]

The last step is to verify that the code is highlighted correctly in a live environment. I've built a tiny playground for the Magik language: https://codepen.io/dmitrysharabin/full/OPRwGKz. It always works with the most recent changes in this PR. Please check if the code in Magik is correctly highlighted with Prism. Give it a test drive so we make sure that the first iteration of the language is solid before we merge it.

[sebastiaanspeck]

Looks good in the codepen 😄 Should we add the snippet thats there as a test case as well?

[DmitrySharabin]

Should we add the snippet thats there as a test case as well?

No need, I think. I just added it so that the playground works with at least some code when it loads. Tests should cover all that independently (on a token basis). Just check with your examples. And if everything's OK, and we have tests for all kinds of tokens, then we can merge it.

[sebastiaanspeck]

Then I believe this is GTG 🚀

[DmitrySharabin]

LGTM! I hope to merge it today.

[DmitrySharabin]

@sebastiaanspeck, thank you for adding a new language to Prism, and congratulations! 🎉