Is there a markdown flavor that allows for font size changes mid-document? It seems like you'd struggle to embed higher-fidelity diagrams that don't encounter issues with wrapping in markdown unless you can also reduce the font size for some diagrams.
What you can do is custom css for a code block or just insert html with a custom class and style that css seperately, many markdown implementations allow for injecting html into the markdown.