Markdown is a terrible language

I've been a huge fan and advocate of Markdown for as long as I can remember, even to the point where it hurt my career. I literally didn't apply to jobs at companies where I couldn't submit my Markdown resume as my official resume for the company. I had my entire resume on my GitHub, and if I had to or I was really interested, I would just submit a PDF with a single link to the Markdown file on my GitHub. That's how dedicated I was. I just didn't think all of these fancy ways of presenting information mattered when you had a good, simple, reliable language for it specifically. I will always be incredibly thankful to John Gruber and Aaron Swartz for creating something as magical as Markdown, having such a simple, reliable way to just write content and have it come out and render so well. It still has a bit of a magical feel to it. As someone who tried really hard to make LaTeX work in college, Markdown, once it clicked for me, just kind of became how I write everything to this day, which is why this article immediately grabbed my attention. Why the heck are we still using Markdown? As great as Markdown is, I am more than happy to agree we have went too far with it. Once we got to the point where we were making new file extensions to embed other types of content into our Markdown, we were probably over the line. And now that we're in a world where we're using LLMs and agents and all of these things, and Markdown is the method we communicate between them, it's getting even more questionable. I have a lot of different things I want to talk about with Markdown in the near future, things like how Markdown is not the right primitive for memory or how Markdown is actually pretty good as a programming scripting language in specific scenarios, which we'll definitely talk a lot about in the future. But today I want to question the fundamentals. Why are we still using Markdown? What makes it so great? And what would a better solution for the stuff we do today look like? I've barely even started and I can already imagine all of the Obsidian comments in my head, and if I know y'all love anything more than Obsidian, it's probably today's sponsor. I have a question for you. What's the most times you've committed code that didn't work just trying to fix CI? I've probably stacked up to 15 commits before just watching CI break again and again. And all I wanted was the ability to run it locally without having to make more commits, ideally in a way that doesn't even force me to leave my editor or my agent. Today's sponsor RWX. I can tell you about how fast they are, how great they are to work with agents, or how they run things in parallel so you don't have to wait for a bunch of stuff that's blocking when it doesn't have to be. There's a lot of reasons to use RWX, but I don't want to talk about any of that because I'm in love with the CLI. Once you write the YAML file for your CI, all you have to do is run their CLI to trigger a run without having to commit the code. So, instead of committing, pushing, and waiting for the whole thing to run, you can just run a CLI command locally, it will then spin up a cloud instance that can still take advantage of all of the caching infra and all the other things they do right, and immediately get your feedback as to whether or not it worked. This should be enough to sell you. But when you combine it with their agent skill, suddenly it's like, oh, yeah. Seriously though, Run Loops are such a great solution to so many different problems. If you were working on a hard feature that you can't test, go write the test, commit it, and then tell the agent, make sure CI passes by running the RWX run command, and then when it passes, commit it and push. This is such a good workflow, and I'm amazed nothing else has it. And this is just the tip of the iceberg. The more you dig in, the more you'll appreciate RWX's unique way of fixing CI so that you will move faster instead of slower. Sign up for the CI that makes your agents faster at swade.link/rwx. So, why the heck are we still using Markdown? There are a few things in life that bring me much joy and heat at the same time, like a chocolate that hurts when eaten, or Markdown. Seriously, why? Half of the time we aren't even using the full language. Speaking of which, this is a bold opening. HTML is the best programming language. I know you've heard people say that the only programming language they know is HTML, and I know we both roll their eyes in discontent trying to get our programming language papers out of assembled decks on of papers on how HTML is only a markup language and not a programming language. I mean, yes, we're on the right, but that guy probably has something that we don't have. A life. Okay, already owned and the article's barely started. I I will have you know that my girlfriend loves when I talk about HTML and Markdown. But to each their own. Note, when the author is talking about Markdown, he is specifically talking about Common Mark unless stated otherwise, because it is the unambiguous syntax specification. I love the project. I really appreciate their efforts on making this language a bit more grounded. It's not the specification that's broken, it's the language itself. Oh boy. The good. Markdown is a minimal language used for typesetting trivial documents. It needed to do one simple job, get a Markdown file and output an HTML file. Its index is as legible as it gets, and it's easy to write even with no assists. Like the C language, you can see the output that will be created. Bold is always open, closed B tags at the end, and the italics are the same. Learning curve is simply nonexistent if you're just a casual user. Just one look at the cheat sheet and you're ready. But now the bad. We don't know what we want. Do we want UI? Do we want a programming language? We don't know. The only reason feature creep exists is because of unclear specifications. You want a minimal, easily legible markup language, you have Markdown. Simple as that, right? Well, here are two different syntaxes to generate the exact same output. You have the single hash for title, hello, stars for doing an italics here for I am, and then unambiguous, which you want bold, and then grammar in a block quote. This is roughly how I would write this. I would have used double stars for unambiguous, but I am what I am. Actually, actually, I would have used double stars for unambiguous and underscores for that, so I guess I'm closer to here. But I would never have done the header syntax like this. God. Yeah. There is almost no syntax overlap between these other than the block quote at the end. Woof. Oh boy, we're about to have fun. Hope you have the two eyeballs enough to see that Markdown is not what you asked for. These produce two identical outputs, and this is just the tip of the iceberg. It has so many poor decisions baked in that if you try to use it, it will actively fight against you the moment you think you know what you're doing. Let's start with bold, italic, and bold italic. In Markdown, you can write a bold in different ways. You have the double star syntax, double underscores, or the B tag. And these are just some of the ways a valid bold can be written. And these are for Common Mark. If you're using something which isn't marketing itself as Common Mark compliant, you can very well encounter valid stuff that produces the same input. Like this, or this, or yeah, God, yeah. We have had to deal with so much [ __ ] like this for T3 Chat, you guys have no idea. It is so absurdly difficult to handle all the different weird cases for Markdown rendering, but I will also personally say the edge cases for LaTeX are even worse. So, yeah. [snorts] And don't even get him started on layered ones like this. Oh God, I don't even want to try to parse this. Okay, we got the two star here, then this one is an italic. So, this whole section is bold here, this one is italicized. This is italicized. A is bolded here, not italicized, and this is re-italicized. Of is italicized. This is italics, and this is doubled the italics for the led here, I think. That hurt. Or this. Oh. No. No, please no. I'm not even going to try on that one. This thing is actually so peak that we have a class of parser vulnerabilities called ReDoS, regular expression denial of service, that are affecting this. Like this 6.9 severity level CVE for Markdown it. This is a legit DoS vulnerability. You just put a bunch of stars in a link and it can't process it. Beautiful. Beautiful. Yeah, this string here takes 65,553 steps. That will [ __ ] over anything. Great, super cool. We love that. So, how much worse can it possibly get here? Exhibit B, ASM was a good idea, but this? In old languages where compilers were producing optimal code like a river in a desert, inline assembly helped them write performance-critical code with ease and the cost of the compiler engineers' blood, sweat, tears, and the birth of their firstborn son. Yeah, writing your inline assembly to get perfect code is hellish. It's one of the reasons that a project like FFmpeg is so tough, but also so important. Inline ASM allowed stuff like SIMD operations before the compiler supported them. You want an overview of early SIMD generation failures, you can take a look at those here. Oh God, that's the last thing I want to look at. Now, let's take this wonderful idea and bolt it directly into the most bloated, single-threaded, sandboxed environment expecting a simple and easy way to write documents. And that's how inline HTML inside Markdown was born. Oh boy. Inline HTML allows you to do stuff like this. I've had so many problems with inline HTML and Markdown personally that I am even more on this author's side now. Yeah. I am an insert simple programmer doing span class fancy text elegant close span programming. Div class animation. And here is my portfolio. Close div. Isn't this just simple? Isn't this just neat? The main reason why correct Markdown parsing is exceptionally hard isn't because Markdown syntax is so hard to comprehend, it's only 1/10 of the issue. The real issue is that to ship a Markdown parser, you also need to ship a friendly HTML parser. If you're using HTML inside of Markdown, why not use HTML from the start? Fair. I actually remember now, friend of the channel, who might even be here right now, Jacob Evans, tried really hard a while back, and it looks like he has given it up since, but he tried super hard to have a single Markdown file for his GitHub profile and for his homepage by serving the Markdown as HTML, because there's enough overlap that hypothetically, you could write a valid HTML homepage and have it render as your profile as well without code between the two. Literally the same exact bytes for both. And the results were hellish. He tried really hard to make it work. But man, it was very fun watching him try. Markdown in and of itself isn't powerful enough to satisfy the simple monkey brain developers like the author, who's only satisfied when the site looks good enough. Good enough in this case means it needs to have at least basic LaTeX support with TikZ support with the ability to install packages, PlantUML, mermaid, custom styling, custom shortcodes, tagging and taxonomy, proper footnotes, BibTeX support. I don't want a simple job from the simple tool, too. To nail a painting to the wall, I need a hammer. In this case, the hammer is markdown. But what if I wanted to paint too? I'll break the canvas the moment I try to paint it with a hammer. Breaking the canvas also means a whole lot of CVEs, primarily around cross-site scripting vulnerabilities. This has me thinking. As we think, I want to take a look at my blog and see how [ __ ] my markdown is. I haven't thought about this in a while. So we already have a helmet here. That's great. I have an iframe embedded cuz it's a YouTube video. So in the first 13 lines in the second paragraph here, I'm already doing weird HTML [ __ ] Then the rest is fine. Oh, here's a great one. I needed to have an image tag that was also a link. So I put an image HTML tag inside of brackets to attach the link. Isn't this wonderful? I had to check two posts to find some cursed [ __ ] More cursed HTML inlined. Man, I used to be so good at writing. It's a shame that the syntax is awful. Yeah. Get the idea. It As soon as you want to use markdown for more interesting stuff, you end up writing some cursed stuff. Every time we allow inline HTML, plugin hooks, or embedded execution engines, we expand the attack surface. The result is predictable. Recurring cross-site scripting vulnerabilities across major markdown implementations and much, much more. And this will continue to rise with the vibe [ __ ] parsers that are hitting the market. Yep. I've seen a lot of attempts to rethink how we parse markdown because all of the agents spit out markdown. We want to make things that are better, faster, more reliable, and all of them are just full of exploits, edge cases, vulnerabilities, and more. Apparently, Obsidian markdown is even more cursed. I'm sure I will love that. Speaking of things I love, we're now in exhibit C, the old and obscure syntax. Markdown, like all the good tech we have around the World Wide Web, was made in the good old 2000s. Markdown was inspired by pre-existing conventions for marking up plain text in email and Usenet posts. Mind you, before 2000, most serious mail looked like this. Do you see this quote syntax putting the colon at the start of every line? Oh god, this ending in the signature. Woo. Do you see the beauty of it? The quote syntax, the vertical separation with pipes. This is what markdown was needed for, which it would fail horribly at because in markdown you can't divide the screen into n parts without inline HTML black magic. But because of this legacy, you have two different ways of writing headings, the normal way and the ATX way. You have two different ways and more than two CVEs of writing bold and italic. You have two different ways of writing horizontal rules, which collide with one of them, colliding with the set text header syntax. You have two different ways of writing unordered lists. You have an ordered list, which doesn't care about how you ordered them. And you have a footnote syntax, which moves the entire grammar to context-dependent grammar. And I remember the first time I saw that like the right way to do lists in markdown, the correct thing is to put zeros because then your markdown parser can number them correctly and if you change the order of things, you don't have to go redo all the numbers. Ugh, so much [ __ ] Oh, oh, I didn't see this line. This language is literally the C++ of markup languages. Oh man. Now that I've seen this, I can't unsee this. That is entirely correct. This might be my favorite programming video ever other than maybe the OOP is garbage video. It's 2 hours long and I had to stop and pause and go back to it later because unlike most 2-hour-long videos, I couldn't watch that one in the background cuz it was too good. So I actually sat and like locked in and just stared at my phone for 2 hours. My roommate at the time asked if I was okay. I was like, "I'm more than okay. You're going to have to watch this after." This video is so good. Highly recommend it. And knowing that, having that still in my head, C++ in markdown, yeah, there's a lot of overlap. Nearly everything can be done in two different ways, which some of them might allow for cross-site scripting and somehow leak memory in HTML. It's used everywhere, but it also fails everywhere equally. And now we're at exhibit D. You can't simply just parse it, you also have to use it. Parsing is the easy thing. If you remember, I told you that the footnote syntax moves this grammar up to a context-sensitive one. Let me elucidate you. Okay, so if we have this syntax with the carrot for a citation, this becomes test bracket one there. But if we actually handle a citation at the bottom, now the markdown changes cuz this is now a link instead. Previously, it just put this in, now this thing further on changes how this part here is parsed. Oh boy. Actually though, footnotes aren't supported in CommonMark. They have links. The only notable difference is that link syntax means you can put only one word after the definition as opposed to a shameful explanation of why bananas on pizza is a good idea. Reference style links and footnotes require global definition resolution. A token's meaning depends on declarations elsewhere in the document. That breaks purely context-free parsing assumptions. Ergo, the updates to a CSG from a CFG. I had some intuition that markdown as as we use it was a context-sensitive grammar, but god damn, this is a really, really damning example of that. Woo. All this a lot of words for saying, "If you want a simple language, it should say simple." Rendering is the hard thing. I think this is the most controversial part about the author's rant because nobody wants to admit that what we use and need are two very different things. Vanilla markdown needs a transliterator, which is literally a one-to-one mapping function. You see bold and then HTML bold comes out. But modern markdown has to support stuff like footnotes, which takes this simple transliterator into a full-blown compiler. And after you made this step, here's the ladder that you'll climb. I need a personal knowledge management system. Oh boy, the Obsidian people are going to love this. Requirement, I'll need footnotes. Cool. We're now at a CSG grammar. Now we've built a compiler. Oh, you want custom callouts? Hooks that bind HTML with markdown? Now you have a dependency graph. I'll need math. Well, you need typesetting library integrations, too. So your dependency graph just got way more complex with execution engines as well. And if you want custom styling, custom CSS, you now have to deal with CSS injections with file-based scoping rules. This one is really fun. Having CSS in one markdown file and making sure it doesn't break things in others, good luck, have fun. This is a great section. I'm sure y'all will love. I swear what I have to do in Obsidian work to make my needs using markdown make it seem like Notion made a sane decision copying Word and Excel internally instead. Yeah. The author thinks that Notion seems sane because of how bad markdown is. Do you understand how absurd that is? Honestly, okay, here I'm going to drop my hot take here. I like Notion's use of markdown as hotkeys effectively, where you can export markdown from Notion, but the formatting of it's broken and garbage. It's so bad. So like don't do that. But when I'm typing in Notion, the fact that I can just use markdown syntax to change the element without having to learn a whole bunch of new hotkeys, that's really nice. Like if I'm looking at the Notion doc for this video, I say sup and I decide this should actually be a title. Hash space, now it's a title. Oh, I want a smaller one, two hash space, three hash space. That's really nice. But that's a different use case. That's not the syntax we're using. That is a syntax or a grammar that we're familiar with being used effectively as a hotkey layer. Fun. Uh Wabo just linked me async syntax in markdown. This is JS though, but still, this syntax for doing execution with awaits in blockers is so cursed. This is like view levels of cursed, but in a markup language. Async markdown, not even once. This This horrifies me. I feel sick. Oh god, there's more? Oh, this is DataView JS as text in Oh, no, I want nothing to do with any of this. Save me. Get me away. No, let's let's see how we solve this according to the author. You ask the guy with a majestic beard with 30-plus years of experience in 8086 assembly, the answer is plain text. You ask the guy with MacBook and OpenClaw running three different Mac minis making a startup a day, the answer is MDX. No, I Okay, I'm not an OpenClaw guy with three Mac minis. I do have three Mac minis in this apartment, but only one is on. I even know MDX is not the right answer here. You ask the guy who codes in C for a living, the answer is reStructuredText. You ask the author, the answer is none. All of these solutions are broken in their own ways. Plain text is beautiful, but I can't show it to somebody that doesn't know what a null pointer dereferences. reStructuredText is wonderful if you only read it and never write it. And MDX is so busy trying to be HTML, it forgets it needs to be legible. And the biggest problem in all of them is that they don't have a build system. We cut corners in the name of speed. If they had a build system, a sane, unambiguous, legible syntax purpose-built for what it's needed to be, I think all of the problems will be fixed. Don't allow inline HTML, allow for well-defined shortcodes and functions to manipulate the text. Allow for custom hooks to be executed before, during, and after the compilation. Oh, so you are in favor of async markdown. And most importantly, define everything. What we need is a custom-built tool, not a Frankenstein's monster that we dare to call a language. Not saying markdown or markup should be a programming language, I'm saying we are trying to use it like one and because it lacks a formal foundation, it's failing at both. I genuinely think that a proper build system around a saner markup language with compile-time hook support can fix a lot of the problems around this with the right constraints. At this point, we should just let go of markdown for good and look for our answers elsewhere, probably with a trivially parsable grammar. He has a whole formal language theory section here. Markup languages are a standard text encoding system consisting of a set of symbols inserted in a text document to control its structure, formatting, or the relationship between its parts. And programming languages are computer programming languages, any of various languages for expressing a set of detailed instructions for digital computer. As you can clearly see, these official definitions are not the way to go. So he's going to use another separate one. A language is a programming language if it's Turing complete. Oh god, so markdown is definitely a programming language. I write JavaScript for a living, so we're not going to get into EBNF and context-free languages. Articles in the description if you're the type that cares about these depths to Turing completeness things. God, none of these words are in the Bible. The thing separating a set of mathematical axioms from provability to non-provability is the want of recursion self-reference, which manifests itself as the common multiplication operation. A trivial observation can have non-trivial reactions. Yeah. Oh God, I missed this. Man, I forgot about this. This was in 2012. Jeff Atwood wrote an article begging Gruber to give up Markdown or at least bless these attempts to make a better standard. Oh man, this is actually really good. I wish that this happened. RIP. John Gruber's original markdown.pl is one of the worst small programs I've ever read, completely riddled with outright bugs and misfeatures that continually bite its users in the ass. It's awful even by the already low standards of handwritten many past regex-based spaghetti parsers. Nobody should be using this original script and unfortunately many of the other implementations out there are direct transliterations that replicate all of its absurd errors. Like where if you mention the MD5 hash of another token in the document, the hash will be replaced with the token because it uses that as inline escaping mechanisms. Reddit hit with a cross-site scripting virus that got through the filters because of this. Jesus Christ. And another Markdown is a disaster from last year. Wild. I did not know there was this quiet cult of people fighting the chaos that is Markdown, but it's really cool to see and I have a lot of respect for the author for taking the time to write this. Thank you to Brock for writing this. This was a fantastic read. I did not realize how much I hated Markdown and I use it every day. This article very much pushed me over the edge to more deeply appreciate how bad it is. Sadly, this author does not have a Twitter or anything linked so I can't throw you at anything other than his blog, but I would recommend checking it out if you're nerdy about these things and the original article has even more fun details if you want to check it out yourself. I got nothing else though. I can't help but notice the irony of me wearing this HTML shirt and I'm going to resist the urge to go rewrite all of my stuff in HTML or maybe a new markup language I make up myself. Please don't make me do this guys. T3 down is not the solution, but I am admitting [sighs] a little bit of temptation. Until next time. Peace nerds.