Extension priority is not adequately documented

[pekkaklarck]

I'm writing an extension for linkifying URLs like http://example.com so that users don't even need to know that they could use <http://example.com> instead. Matching URLs isn't too hard (it doesn't need to be perfect), but the challenge is not modifying URLs in cases like [example](http://example.com). My understanding is that extension priority affects this, but the priority concept isn't really documented in the API docs.

The docs first say that

You may register instances of your processors and patterns with an appropriate priority

but as a user it's hard for me to tell what would be an "appropriate priority" in my case. The examples then later use 175 as the priority without explaining why that number was chosen. Finally, Register.register documents priority argument like

An integer or float used to sort against all items.

which isn't very helpful.

Thinks I'd expect to find from the docs:

• Does a small number mean a higher or smaller priority? In other words, is a plugin with priority 1 run before a plugin with priority 1000?
• Are negative priorities allowed? How are they interpreted?
• If a content is processed with an extension with a higher priority, is it anymore processed with other extensions? Does this behavior vary between different kind of extensions? I'm mostly interested in inline processors.
• What are priorities of built-in extensions? Can they be considered to be stable?

---

Good news is that I was able to make my prototype work by using a low priority like 10. When I used 175 that the examples had, the plugin messed up the [example](http://example.com) syntax. Based on that my assumption is that a higher number means higher priority and that extensions with lower priority aren't run if a higher priority extension already matched something.

Below is my current prototype code in case someone is interested. I still need to test it more thoroughly and the somewhat hairy regex should probably get some comments.

import xml.etree.ElementTree as ET

from markdown.inlinepatterns import InlineProcessor
from markdown.extensions import Extension


class LinkifyInlineProcessor(InlineProcessor):
    def handleMatch(self, match, data):
        url, tail = match.groups()
        el = ET.Element("a", href=url)
        el.text = url
        el.tail = tail
        return el, match.start(0), match.end(0)


class LinkifyExtension(Extension):
    def extendMarkdown(self, md):
        pattern = r"([a-z][\w+-.]*://\S+?)([)\]}\"'.,!?:;|]*(?=\s|$))"
        md.inlinePatterns.register(LinkifyInlineProcessor(pattern, md), 'linkify', 10)

[waylan]

Thank you for providing your feedback. When the person who builds a feature documents it, sometimes it can be difficult to see it from another perspective. There are two things going on here.

First of all, when I named the option "priority," my thinking was that a higher priority would get precedence over a lower priority. In other words, a higher number would be run before a lower number. However, you are not the first person to stumble over this. And looking back at the documentation I wrote years ago (long enough ago that I have no relocation of what I actually documented), I can see that that is not actually fully explained. So, thank you, we will try to fix that.

As far as the actual numbers to use, that is dependent on the code. The code is always in flux (to a certain extent) and we could add or remove processors (or change a processor's priority) based upon new features or bug fixes. Therefore, in the API reference (which is generated from the code), we document all of the priorities used within the code (for example, see the inline patterns). I suppose we should point that out in the extension docs. Back when the docs were written, we didn't have any generated docs and we expected people to look at the code. Although, I see that isn't explained in the docs either.

[pekkaklarck]

Quick comments:

• Thanks for looking at this!
• I can understand your reasoning that higher priority number means, well, higher priority, but without this being explained it's easy to think that "priority 1" must be the highest priority.
• The tutorial talks a bit more about the priority, but also those docs could probably be enhanced a bit.
• API reference looks useful, but it doesn't seem to be linked from the extension API docs.
• I can understand that you don't want to set priorities to stone. That said, deciding and documenting that built-in extension priorities are on same range (e.g. 50-250) would mean that others could safely configure are their extensions used before or after the built-in ones.
• Python-Markdown getting an option to allow linkifying URLs would be convenient. I know the original Markdown implementation didn't support that and required using <http://example.com> style, but linkifying is so common (e.g. here on GitHub) that I suspect most users expect that. Probably this functionality should be off by default, though.

[facelessuser]

There are extensions that do this, I have one. In general, Python Markdown has deferred new non-standard stuff to 3rd party plugins and only retains the custom stuff implemented in the earlier days.

[waylan]

In general, Python Markdown has deferred new non-standard stuff to 3rd party plugins and only retains the custom stuff implemented in the earlier days.

This is something else we need to document. I think we need to make it clear that the included extensions are in "maintenance mode." The existing extensions will continue to receive bug fixes and be maintained (will continue to work with any updates to the core), but no new features or changes in behavior will be made. If users would like to address other needs, they should look to third-party extensions and/or fork the included extensions.