Skip to content

blockprocessors

CORE MARKDOWN BLOCKPARSER.

This parser handles basic parsing of Markdown blocks. It doesn't concern itself with inline elements such as bold or italics, but rather just catches blocks, lists, quotes, etc.

The BlockParser is made up of a bunch of BlockProssors, each handling a different type of block. Extensions may add/replace/remove BlockProcessors as they need to alter how markdown blocks are parsed.

Attributes

logger = logging.getLogger('MARKDOWN') module-attribute

Classes

BlockProcessor(parser)

Bases: object

Base class for block processors.

Each subclass will provide the methods below to work with the source and tree. Each processor will need to define it's own test and run methods. The test method should return True or False, to indicate whether the current block should be processed by this processor. If the test passes, the parser will call the processors run method.

Parameters:

Name Type Description Default
parser BlockParser

BlockParser instance

 required
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)

Test for block type. Must be overridden by subclasses.

As the parser loops through processors, it will call the test method on each to determine if the given block of text is of that type. This method must return a boolean True or False. The actual method of testing is left to the needs of that particular block type. It could be as simple as block.startswith(some_string) or a complex regular expression. As the block type may be different depending on the parent of the block (i.e. inside a list), the parent etree element is also provided and may be used as part of the test.

Parameters:

Name Type Description Default
parent Element

A etree element which will be the parent of the block.

 required
block list[Element]

A block of text from the source which has been split at blank lines.

 required
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
def test(self, parent, block):
    """Test for block type. Must be overridden by subclasses.

    As the parser loops through processors, it will call the ``test``
    method on each to determine if the given block of text is of that
    type. This method must return a boolean ``True`` or ``False``. The
    actual method of testing is left to the needs of that particular
    block type. It could be as simple as ``block.startswith(some_string)``
    or a complex regular expression. As the block type may be different
    depending on the parent of the block (i.e. inside a list), the parent
    etree element is also provided and may be used as part of the test.

    Args:
        parent (Element): A etree element which will be the parent of the block.
        block (list[Element]): A block of text from the source
            which has been split at blank lines.
    """
    pass  # pragma: no cover
run(parent, blocks)

Run processor. Must be overridden by subclasses.

When the parser determines the appropriate type of a block, the parser will call the corresponding processor's run method. This method should parse the individual lines of the block and append them to the etree.

Note that both the parent and etree keywords are pointers to instances of the objects which should be edited in place. Each processor must make changes to the existing objects as there is no mechanism to return new/different objects to replace them.

This means that this method should be adding SubElements or adding text to the parent, and should remove (pop) or add (insert) items to the list of blocks.

Other Parameters:

Name Type Description
parent Element

A etree element which is the parent of the current block.
blocks list[Element]

A list of all remaining blocks of the document.
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
def run(self, parent, blocks):
    """Run processor. Must be overridden by subclasses.

    When the parser determines the appropriate type of a block, the parser
    will call the corresponding processor's ``run`` method. This method
    should parse the individual lines of the block and append them to
    the etree.

    Note that both the ``parent`` and ``etree`` keywords are pointers
    to instances of the objects which should be edited in place. Each
    processor must make changes to the existing objects as there is no
    mechanism to return new/different objects to replace them.

    This means that this method should be adding SubElements or adding text
    to the parent, and should remove (``pop``) or add (``insert``) items to
    the list of blocks.

    Keyword Args:
        parent (Element): A etree element which is the parent of the current block.
        blocks (list[Element]): A list of all remaining blocks of the document.
    """
    pass  # pragma: no cover

ListIndentProcessor(*args)

Bases: BlockProcessor

Process children of list items.

Examples:

  • a list item process this part

    or this part

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
143
144
145
def __init__(self, *args):
    super(ListIndentProcessor, self).__init__(*args)
    self.INDENT_RE = re.compile(r'^(([ ]{%s})+)' % self.tab_length)

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute
ITEM_TYPES = ['li'] class-attribute instance-attribute
LIST_TYPES = ['ul', 'ol'] class-attribute instance-attribute
INDENT_RE = re.compile('^(([ ]{%s})+)' % self.tab_length) instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
147
148
149
150
151
152
def test(self, parent, block):
    return block.startswith(' '*self.tab_length) and \
        not self.parser.state.isstate('detabbed') and \
        (parent.tag in self.ITEM_TYPES or
            (len(parent) and parent[-1] is not None and
                (parent[-1].tag in self.LIST_TYPES)))
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
def run(self, parent, blocks):
    block = blocks.pop(0)
    level, sibling = self.get_level(parent, block)
    block = self.looseDetab(block, level)

    self.parser.state.set('detabbed')
    if parent.tag in self.ITEM_TYPES:
        # It's possible that this parent has a 'ul' or 'ol' child list
        # with a member.  If that is the case, then that should be the
        # parent.  This is intended to catch the edge case of an indented
        # list whose first member was parsed previous to this point
        # see OListProcessor
        if len(parent) and parent[-1].tag in self.LIST_TYPES:
            self.parser.parseBlocks(parent[-1], [block])
        else:
            # The parent is already a li. Just parse the child block.
            self.parser.parseBlocks(parent, [block])
    elif sibling.tag in self.ITEM_TYPES:
        # The sibling is a li. Use it as parent.
        self.parser.parseBlocks(sibling, [block])
    elif len(sibling) and sibling[-1].tag in self.ITEM_TYPES:
        # The parent is a list (``ol`` or ``ul``) which has children.
        # Assume the last child li is the parent of this block.
        if sibling[-1].text:
            # If the parent li has text, that text needs to be moved to a p
            # The p must be 'inserted' at beginning of list in the event
            # that other children already exist i.e.; a nested sublist.
            p = util.etree.Element('p')
            p.text = sibling[-1].text
            sibling[-1].text = ''
            sibling[-1].insert(0, p)
        self.parser.parseChunk(sibling[-1], block)
    else:
        self.create_item(sibling, block)
    self.parser.state.reset()
create_item(parent, block)

Create a new li and parse the block with it as the parent.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
190
191
192
193
def create_item(self, parent, block):
    """Create a new li and parse the block with it as the parent."""
    li = util.etree.SubElement(parent, 'li')
    self.parser.parseBlocks(li, [block])
get_level(parent, block)

Get level of indent based on list level.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
def get_level(self, parent, block):
    """Get level of indent based on list level."""
    # Get indent level
    m = self.INDENT_RE.match(block)
    if m:
        indent_level = len(m.group(1))/self.tab_length
    else:
        indent_level = 0
    if self.parser.state.isstate('list'):
        # We're in a tightlist - so we already are at correct parent.
        level = 1
    else:
        # We're in a looselist - so we need to find parent.
        level = 0
    # Step through children of tree to find matching indent level.
    while indent_level > level:
        child = self.lastChild(parent)
        if (child is not None and
           (child.tag in self.LIST_TYPES or child.tag in self.ITEM_TYPES)):
            if child.tag in self.LIST_TYPES:
                level += 1
            parent = child
        else:
            # No more child levels. If we're short of indent_level,
            # we have a code block. So we stop here.
            break
    return level, parent

CodeBlockProcessor(parser)

Bases: BlockProcessor

Process code blocks.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
227
228
def test(self, parent, block):
    return block.startswith(' '*self.tab_length)
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
def run(self, parent, blocks):
    sibling = self.lastChild(parent)
    block = blocks.pop(0)
    theRest = ''
    if (sibling is not None and sibling.tag == "pre" and
       len(sibling) and sibling[0].tag == "code"):
        # The previous block was a code block. As blank lines do not start
        # new code blocks, append this block to the previous, adding back
        # linebreaks removed from the split into a list.
        code = sibling[0]
        block, theRest = self.detab(block)
        code.text = util.AtomicString(
            '%s\n%s\n' % (code.text, block.rstrip())
        )
    else:
        # This is a new codeblock. Create the elements and insert text.
        pre = util.etree.SubElement(parent, 'pre')
        code = util.etree.SubElement(pre, 'code')
        block, theRest = self.detab(block)
        code.text = util.AtomicString('%s\n' % block.rstrip())
    if theRest:
        # This block contained unindented line(s) after the first indented
        # line. Insert these lines as the first block of the master blocks
        # list for future processing.
        blocks.insert(0, theRest)

BlockQuoteProcessor(parser)

Bases: BlockProcessor

Blockquote processor.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute
RE = re.compile('(^|\\n)[ ]{0,3}>[ ]?(.*)') class-attribute instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
262
263
def test(self, parent, block):
    return bool(self.RE.search(block))
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
def run(self, parent, blocks):
    block = blocks.pop(0)
    m = self.RE.search(block)
    if m:
        before = block[:m.start()]  # Lines before blockquote
        # Pass lines before blockquote in recursively for parsing forst.
        self.parser.parseBlocks(parent, [before])
        # Remove ``> `` from begining of each line.
        block = '\n'.join(
            [self.clean(line) for line in block[m.start():].split('\n')]
        )
    sibling = self.lastChild(parent)
    if sibling is not None and sibling.tag == "blockquote":
        # Previous block was a blockquote so set that as this blocks parent
        quote = sibling
    else:
        # This is a new blockquote. Create a new parent element.
        quote = util.etree.SubElement(parent, 'blockquote')
    # Recursively parse block with blockquote as parent.
    # change parser state so blockquotes embedded in lists use p tags
    self.parser.state.set('blockquote')
    self.parser.parseChunk(quote, block)
    self.parser.state.reset()
clean(line)

Remove > from beginning of a line.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
289
290
291
292
293
294
295
296
297
def clean(self, line):
    """Remove ``>`` from beginning of a line."""
    m = self.RE.match(line)
    if line.strip() == ">":
        return ""
    elif m:
        return m.group(2)
    else:
        return line

OListProcessor(parser)

Bases: BlockProcessor

Process ordered list blocks.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
312
313
314
315
316
317
318
319
320
321
def __init__(self, parser):
    super(OListProcessor, self).__init__(parser)
    # Detect an item (``1. item``). ``group(1)`` contains contents of item.
    self.RE = re.compile(r'^[ ]{0,%d}\d+\.[ ]+(.*)' % (self.tab_length - 1))
    # Detect items on secondary lines. they can be of either list type.
    self.CHILD_RE = re.compile(r'^[ ]{0,%d}((\d+\.)|[*+-])[ ]+(.*)' %
                               (self.tab_length - 1))
    # Detect indented (nested) items of either type
    self.INDENT_RE = re.compile(r'^[ ]{%d,%d}((\d+\.)|[*+-])[ ]+.*' %
                                (self.tab_length, self.tab_length * 2 - 1))

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute
TAG = 'ol' class-attribute instance-attribute
STARTSWITH = '1' class-attribute instance-attribute
SIBLING_TAGS = ['ol', 'ul'] class-attribute instance-attribute
RE = re.compile('^[ ]{0,%d}\\d+\\.[ ]+(.*)' % self.tab_length - 1) instance-attribute
CHILD_RE = re.compile('^[ ]{0,%d}((\\d+\\.)|[*+-])[ ]+(.*)' % self.tab_length - 1) instance-attribute
INDENT_RE = re.compile('^[ ]{%d,%d}((\\d+\\.)|[*+-])[ ]+.*' % (self.tab_length, self.tab_length * 2 - 1)) instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
323
324
def test(self, parent, block):
    return bool(self.RE.match(block))
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
def run(self, parent, blocks):
    # Check fr multiple items in one block.
    items = self.get_items(blocks.pop(0))
    sibling = self.lastChild(parent)

    if sibling is not None and sibling.tag in self.SIBLING_TAGS:
        # Previous block was a list item, so set that as parent
        lst = sibling
        # make sure previous item is in a p- if the item has text,
        # then it isn't in a p
        if lst[-1].text:
            # since it's possible there are other children for this
            # sibling, we can't just SubElement the p, we need to
            # insert it as the first item.
            p = util.etree.Element('p')
            p.text = lst[-1].text
            lst[-1].text = ''
            lst[-1].insert(0, p)
        # if the last item has a tail, then the tail needs to be put in a p
        # likely only when a header is not followed by a blank line
        lch = self.lastChild(lst[-1])
        if lch is not None and lch.tail:
            p = util.etree.SubElement(lst[-1], 'p')
            p.text = lch.tail.lstrip()
            lch.tail = ''

        # parse first block differently as it gets wrapped in a p.
        li = util.etree.SubElement(lst, 'li')
        self.parser.state.set('looselist')
        firstitem = items.pop(0)
        self.parser.parseBlocks(li, [firstitem])
        self.parser.state.reset()
    elif parent.tag in ['ol', 'ul']:
        # this catches the edge case of a multi-item indented list whose
        # first item is in a blank parent-list item:
        # * * subitem1
        #     * subitem2
        # see also ListIndentProcessor
        lst = parent
    else:
        # This is a new list so create parent with appropriate tag.
        lst = util.etree.SubElement(parent, self.TAG)
        # Check if a custom start integer is set
        if not self.parser.markdown.lazy_ol and self.STARTSWITH != '1':
            lst.attrib['start'] = self.STARTSWITH

    self.parser.state.set('list')
    # Loop through items in block, recursively parsing each with the
    # appropriate parent.
    for item in items:
        if item.startswith(' '*self.tab_length):
            # Item is indented. Parse with last item as parent
            self.parser.parseBlocks(lst[-1], [item])
        else:
            # New item. Create li and parse with it as parent
            li = util.etree.SubElement(lst, 'li')
            self.parser.parseBlocks(li, [item])
    self.parser.state.reset()
get_items(block)

Break a block into list items.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
def get_items(self, block):
    """Break a block into list items."""
    items = []
    for line in block.split('\n'):
        m = self.CHILD_RE.match(line)
        if m:
            # This is a new list item
            # Check first item for the start index
            if not items and self.TAG == 'ol':
                # Detect the integer value of first list item
                INTEGER_RE = re.compile(r'(\d+)')
                self.STARTSWITH = INTEGER_RE.match(m.group(1)).group()
            # Append to the list
            items.append(m.group(3))
        elif self.INDENT_RE.match(line):
            # This is an indented (possibly nested) item.
            if items[-1].startswith(' '*self.tab_length):
                # Previous item was indented. Append to that item.
                items[-1] = '%s\n%s' % (items[-1], line)
            else:
                items.append(line)
        else:
            # This is another line of previous item. Append to that item.
            items[-1] = '%s\n%s' % (items[-1], line)
    return items

UListProcessor(parser)

Bases: OListProcessor

Process unordered list blocks.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
417
418
419
420
def __init__(self, parser):
    super(UListProcessor, self).__init__(parser)
    # Detect an item (``1. item``). ``group(1)`` contains contents of item.
    self.RE = re.compile(r'^[ ]{0,%d}[*+-][ ]+(.*)' % (self.tab_length - 1))

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute
STARTSWITH = '1' class-attribute instance-attribute
SIBLING_TAGS = ['ol', 'ul'] class-attribute instance-attribute
CHILD_RE = re.compile('^[ ]{0,%d}((\\d+\\.)|[*+-])[ ]+(.*)' % self.tab_length - 1) instance-attribute
INDENT_RE = re.compile('^[ ]{%d,%d}((\\d+\\.)|[*+-])[ ]+.*' % (self.tab_length, self.tab_length * 2 - 1)) instance-attribute
TAG = 'ul' class-attribute instance-attribute
RE = re.compile('^[ ]{0,%d}[*+-][ ]+(.*)' % self.tab_length - 1) instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
323
324
def test(self, parent, block):
    return bool(self.RE.match(block))
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
def run(self, parent, blocks):
    # Check fr multiple items in one block.
    items = self.get_items(blocks.pop(0))
    sibling = self.lastChild(parent)

    if sibling is not None and sibling.tag in self.SIBLING_TAGS:
        # Previous block was a list item, so set that as parent
        lst = sibling
        # make sure previous item is in a p- if the item has text,
        # then it isn't in a p
        if lst[-1].text:
            # since it's possible there are other children for this
            # sibling, we can't just SubElement the p, we need to
            # insert it as the first item.
            p = util.etree.Element('p')
            p.text = lst[-1].text
            lst[-1].text = ''
            lst[-1].insert(0, p)
        # if the last item has a tail, then the tail needs to be put in a p
        # likely only when a header is not followed by a blank line
        lch = self.lastChild(lst[-1])
        if lch is not None and lch.tail:
            p = util.etree.SubElement(lst[-1], 'p')
            p.text = lch.tail.lstrip()
            lch.tail = ''

        # parse first block differently as it gets wrapped in a p.
        li = util.etree.SubElement(lst, 'li')
        self.parser.state.set('looselist')
        firstitem = items.pop(0)
        self.parser.parseBlocks(li, [firstitem])
        self.parser.state.reset()
    elif parent.tag in ['ol', 'ul']:
        # this catches the edge case of a multi-item indented list whose
        # first item is in a blank parent-list item:
        # * * subitem1
        #     * subitem2
        # see also ListIndentProcessor
        lst = parent
    else:
        # This is a new list so create parent with appropriate tag.
        lst = util.etree.SubElement(parent, self.TAG)
        # Check if a custom start integer is set
        if not self.parser.markdown.lazy_ol and self.STARTSWITH != '1':
            lst.attrib['start'] = self.STARTSWITH

    self.parser.state.set('list')
    # Loop through items in block, recursively parsing each with the
    # appropriate parent.
    for item in items:
        if item.startswith(' '*self.tab_length):
            # Item is indented. Parse with last item as parent
            self.parser.parseBlocks(lst[-1], [item])
        else:
            # New item. Create li and parse with it as parent
            li = util.etree.SubElement(lst, 'li')
            self.parser.parseBlocks(li, [item])
    self.parser.state.reset()
get_items(block)

Break a block into list items.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
def get_items(self, block):
    """Break a block into list items."""
    items = []
    for line in block.split('\n'):
        m = self.CHILD_RE.match(line)
        if m:
            # This is a new list item
            # Check first item for the start index
            if not items and self.TAG == 'ol':
                # Detect the integer value of first list item
                INTEGER_RE = re.compile(r'(\d+)')
                self.STARTSWITH = INTEGER_RE.match(m.group(1)).group()
            # Append to the list
            items.append(m.group(3))
        elif self.INDENT_RE.match(line):
            # This is an indented (possibly nested) item.
            if items[-1].startswith(' '*self.tab_length):
                # Previous item was indented. Append to that item.
                items[-1] = '%s\n%s' % (items[-1], line)
            else:
                items.append(line)
        else:
            # This is another line of previous item. Append to that item.
            items[-1] = '%s\n%s' % (items[-1], line)
    return items

HashHeaderProcessor(parser)

Bases: BlockProcessor

Process Hash Headers.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute
RE = re.compile('(^|\\n)(?P<level>#{1,6})(?P<header>.*?)#*(\\n|$)') class-attribute instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
429
430
def test(self, parent, block):
    return bool(self.RE.search(block))
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
def run(self, parent, blocks):
    block = blocks.pop(0)
    m = self.RE.search(block)
    if m:
        before = block[:m.start()]  # All lines before header
        after = block[m.end():]     # All lines after header
        if before:
            # As the header was not the first line of the block and the
            # lines before the header must be parsed first,
            # recursively parse this lines as a block.
            self.parser.parseBlocks(parent, [before])
        # Create header using named groups from RE
        h = util.etree.SubElement(parent, 'h%d' % len(m.group('level')))
        h.text = m.group('header').strip()
        if after:
            # Insert remaining lines as first block for future parsing.
            blocks.insert(0, after)
    else:  # pragma: no cover
        # This should never happen, but just in case...
        logger.warn("We've got a problem header: %r" % block)

SetextHeaderProcessor(parser)

Bases: BlockProcessor

Process Setext-style Headers.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute
RE = re.compile('^.*?\\n[=-]+[ ]*(\\n|$)', re.MULTILINE) class-attribute instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
460
461
def test(self, parent, block):
    return bool(self.RE.match(block))
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
463
464
465
466
467
468
469
470
471
472
473
474
def run(self, parent, blocks):
    lines = blocks.pop(0).split('\n')
    # Determine level. ``=`` is 1 and ``-`` is 2.
    if lines[1].startswith('='):
        level = 1
    else:
        level = 2
    h = util.etree.SubElement(parent, 'h%d' % level)
    h.text = lines[0].strip()
    if len(lines) > 2:
        # Block contains additional lines. Add to  master blocks for later.
        blocks.insert(0, '\n'.join(lines[2:]))

HRProcessor(parser)

Bases: BlockProcessor

Process Horizontal Rules.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute
RE = '^[ ]{0,3}((-+[ ]{0,2}){3,}|(_+[ ]{0,2}){3,}|(\\*+[ ]{0,2}){3,})[ ]*' class-attribute instance-attribute
SEARCH_RE = re.compile(RE, re.MULTILINE) class-attribute instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
484
485
486
487
488
489
490
491
492
493
def test(self, parent, block):
    m = self.SEARCH_RE.search(block)
    # No atomic grouping in python so we simulate it here for performance.
    # The regex only matches what would be in the atomic group - the HR.
    # Then check if we are at end of block or if next char is a newline.
    if m and (m.end() == len(block) or block[m.end()] == '\n'):
        # Save match object on class instance so we can use it later.
        self.match = m
        return True
    return False
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
def run(self, parent, blocks):
    block = blocks.pop(0)
    match = self.match
    # Check for lines in block before hr.
    prelines = block[:match.start()].rstrip('\n')
    if prelines:
        # Recursively parse lines before hr so they get parsed first.
        self.parser.parseBlocks(parent, [prelines])
    # create hr
    util.etree.SubElement(parent, 'hr')
    # check for lines in block after hr.
    postlines = block[match.end():].lstrip('\n')
    if postlines:
        # Add lines after hr to master blocks for later parsing.
        blocks.insert(0, postlines)

EmptyBlockProcessor(parser)

Bases: BlockProcessor

Process blocks that are empty or start with an empty line.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
515
516
def test(self, parent, block):
    return not block or block.startswith('\n')
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
def run(self, parent, blocks):
    block = blocks.pop(0)
    filler = '\n\n'
    if block:
        # Starts with empty line
        # Only replace a single line.
        filler = '\n'
        # Save the rest for later.
        theRest = block[1:]
        if theRest:
            # Add remaining lines to master blocks for later.
            blocks.insert(0, theRest)
    sibling = self.lastChild(parent)
    if (sibling is not None and sibling.tag == 'pre' and
       len(sibling) and sibling[0].tag == 'code'):
        # Last block is a codeblock. Append to preserve whitespace.
        sibling[0].text = util.AtomicString(
            '%s%s' % (sibling[0].text, filler)
        )

ParagraphProcessor(parser)

Bases: BlockProcessor

Process Paragraph blocks.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
54
55
56
def __init__(self, parser):
    self.parser = parser
    self.tab_length = parser.markdown.tab_length

Attributes

parser = parser instance-attribute
tab_length = parser.markdown.tab_length instance-attribute

Functions

lastChild(parent)

Return the last child of an etree element.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
58
59
60
61
62
63
def lastChild(self, parent):
    """Return the last child of an etree element."""
    if len(parent):
        return parent[-1]
    else:
        return None
detab(text)

Remove a tab from the front of each line of the given text.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
65
66
67
68
69
70
71
72
73
74
75
76
def detab(self, text):
    """Remove a tab from the front of each line of the given text."""
    newtext = []
    lines = text.split('\n')
    for line in lines:
        if line.startswith(' '*self.tab_length):
            newtext.append(line[self.tab_length:])
        elif not line.strip():
            newtext.append('')
        else:
            break
    return '\n'.join(newtext), '\n'.join(lines[len(newtext):])
looseDetab(text, level=1)

Remove a tab from front of lines but allowing dedented lines.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
78
79
80
81
82
83
84
def looseDetab(self, text, level=1):
    """Remove a tab from front of lines but allowing dedented lines."""
    lines = text.split('\n')
    for i in range(len(lines)):
        if lines[i].startswith(' '*self.tab_length*level):
            lines[i] = lines[i][self.tab_length*level:]
    return '\n'.join(lines)
test(parent, block)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
542
543
def test(self, parent, block):
    return True
run(parent, blocks)
Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
def run(self, parent, blocks):
    block = blocks.pop(0)
    if block.strip():
        # Not a blank block. Add to parent, otherwise throw it away.
        if self.parser.state.isstate('list'):
            # The parent is a tight-list.
            #
            # Check for any children. This will likely only happen in a
            # tight-list when a header isn't followed by a blank line.
            # For example:
            #
            #     * # Header
            #     Line 2 of list item - not part of header.
            sibling = self.lastChild(parent)
            if sibling is not None:
                # Insetrt after sibling.
                if sibling.tail:
                    sibling.tail = '%s\n%s' % (sibling.tail, block)
                else:
                    sibling.tail = '\n%s' % block
            else:
                # Append to parent.text
                if parent.text:
                    parent.text = '%s\n%s' % (parent.text, block)
                else:
                    parent.text = block.lstrip()
        else:
            # Create a regular paragraph
            p = util.etree.SubElement(parent, 'p')
            p.text = block.lstrip()

Functions

build_block_parser(md_instance, **kwargs)

Build the default block parser used by Markdown.

Source code in pyrevitlib/pyrevit/coreutils/markdown/blockprocessors.py 
25
26
27
28
29
30
31
32
33
34
35
36
37
38
def build_block_parser(md_instance, **kwargs):
    """Build the default block parser used by Markdown."""
    parser = BlockParser(md_instance)
    parser.blockprocessors['empty'] = EmptyBlockProcessor(parser)
    parser.blockprocessors['indent'] = ListIndentProcessor(parser)
    parser.blockprocessors['code'] = CodeBlockProcessor(parser)
    parser.blockprocessors['hashheader'] = HashHeaderProcessor(parser)
    parser.blockprocessors['setextheader'] = SetextHeaderProcessor(parser)
    parser.blockprocessors['hr'] = HRProcessor(parser)
    parser.blockprocessors['olist'] = OListProcessor(parser)
    parser.blockprocessors['ulist'] = UListProcessor(parser)
    parser.blockprocessors['quote'] = BlockQuoteProcessor(parser)
    parser.blockprocessors['paragraph'] = ParagraphProcessor(parser)
    return parser