recently fixed bugs

From the source of usage:

<a href="mailto:joey@ikiwiki.info">&#x6A;&#111;&#101;&#x79;&#64;i&#107;&#105;w&#105;&#107;&#x69;&#46;&#105;n&#x66;&#x6F;</a>

Text::Markdown obfuscates email addresses in the href= attribute and in the text. Apparently this can't be configured.

HTML::Scrubber doesn't set attr_encoded for its HTML::Parser, so the href= attribtute is decoded. Currently it seems it doesn't set attr_encoded for good reason: so attributes can be sanitized easily, e.g. as in htmlscrubber with $safe_url_regexp. This apparently can't be configured either.

So I can't see an obvious solution to this. Perhaps improvements to Text::Markdown or HTML::Scrubber can allow a fix.

One question is: how useful is email obfuscation? Don't spammers use HTML parsers?

I now see this was noted in the formatting discussion, and won't/can't be fixed. So I guess this is done. --Gabriel

I've patched mdwn.pm to prevent Text::Markdown from obfuscating the emails. The relevant commits are on the master branch of my "fork" of ikiwiki on Github:

  • 7d0970adbcf0b63e7e5532c239156f6967d10158
  • 52c241e723ced4d7c6a702dd08cda37feee75531

--Gabriel.

Thanks for coming up with a patch, but overriding Text::Markdown::_EncodeEmailAddress gets into its internals more than I'm comfortable with.

It would probably be best to add an option to Text::Markdown to let the email address munging be disabled. --Joey

Email obfuscation is very useful -- in practice, spammers apparently don't use HTML parsers -- according to the only published study I have read ( a 2003 study by the Center for Democracy and Technology cited by https://en.wikipedia.org/wiki/Address_munging ).

Posted Mon Jul 21 23:25:17 2008

The new way to confirm ownership of a domain on Flattr is to add a meta tag to the page head. For example:

 <meta name="flattr:id" content="4j6y0v">

However, the meta directive doesn't allow setting of names with colons.

If I do this:

[[!meta  flattr:id="4j6y0v"]]

it gets rendered as:

<meta name="flattr:id=&quot;4j6y0v&quot;" content="" />

I tried a number of possibilities and they all failed to produce the correct output.

Directive syntax doesn't allow named arguments containing colons, so we would have to add a different syntax for weird names. However, we already have that:

[[!meta  name="flattr:id" content="4j6y0v"]]

This was missing from the documentation, but I have now added it. This feature was broken until 2015, but we now have an automated test to make sure it keeps working; the test includes a check for twitter:card which is essentially equivalent to what you're doing here. done --smcv

Posted Thu May 18 13:33:44 2017

Bug Description

If color and toc plugins are enabled and you use colored headers, those headers are never colored but sometimes are prefixed with text artifacts like "color: red".

Example: The header

# [[!color   foreground=red text="Testing"]]

would sometimes be seen in the toc as

color: redTesting

Reason for this behaviour is:

  1. the color plugin uses a special syntax to preserve the color through sanitize and that syntax has a plain text component.
  2. the toc plugin removes everything except plain text from headers in the toc
  3. if the toc plugin is executed before the color plugin in the format hook it sees the special syntax and clobbers the toc, otherwise it just removes the color markup

The bug here is that the color plugin's special syntax does not gracefully degrade to "render nothing", which I have now fixed by putting the color bits through a value attribute instead of character data. --smcv

Solutions

There are a few possible solutions to this depending on how it should work:

  1. The easiest thing would be to just add a "last" parameter to the toc plugin format hook (or "first" to the color plugin). Result: No color in tocs at all
  2. Adding seven lines to toc.pm would make it preserve ALL markup in headers, color as well as html markup or markdown (emphasize for example). Execution order of the plugins would not matter at all
  3. A bit more code would be necessary to just specifically preserve the color, but nothing else

I would propose implementing the second option because visual markers in headers are useful to convey additional information very fast and this information should be preserved in the toc. Example: Bug or task/project tracker with color conveying status of the bug or task.

This is really a separate feature request: copy non-<a> markup in headings into the TOC. I don't think this necessarily makes sense in general. In particular, any id attributes on child elements must not be passed through because that would make the ID non-unique. --smcv

It seems you can stuff anything into ordered lists (according to w3.orgs doku), so apart from stylistic reasons and suboptimal display of links in headers (see below) I don't see any problems with markup in the toc.

Patch

This is the proposed patch to the second solution. Tested with the latest version. It works with all markup and markdown I could think of. The only case not handled optimal is if the header is just a link and nothing else, then there is no text left for the local link, the toc links directly to a different page. Is that acceptable or not?

diff --git a/IkiWiki/Plugin/toc.pm b/IkiWiki/Plugin/toc.pm
index ac07b9a..5c2b056 100644
--- a/IkiWiki/Plugin/toc.pm
+++ b/IkiWiki/Plugin/toc.pm
@@ -57,6 +57,7 @@ sub format (@) {
    my $startlevel=($params{startlevel} ? $params{startlevel} : 0);
    my $curlevel=$startlevel-1;
    my $liststarted=0;
+   my $headercollect=0;
    my $indent=sub { "\t" x $curlevel };
    $p->handler(start => sub {
        my $tagname=shift;
@@ -107,6 +108,7 @@ sub format (@) {
            $index.=&$indent."<li class=\"L$curlevel\">".
                "<a href=\"#$anchor\">";

+           $headercollect=1;
            $p->handler(text => sub {
                $page.=join("", @_);
                $index.=join("", @_);
@@ -117,12 +119,17 @@ sub format (@) {
                    $p->handler(text => undef);
                    $p->handler(end => undef);
                    $index.="</a>\n";
+                   $headercollect=0;
+               }
+               else {
+                   $index.=join("",@_);
                }
                $page.=join("", @_);
            }, "tagname, text");
        }
        else {
            $page.=$text;
+           $index.=$text if ($headercollect);
        }
    }, "tagname, text");
    $p->handler(default => sub { $page.=join("", @_) }, "text");
Posted Fri Mar 11 11:29:55 2016 Tags:

Hi! While working on Reproducible Builds for Tails, we noticed that the pagestats plugin's output is not deterministic: pages that have the same number of hits (counts) are sorted in hash order.

The pagestats-determinism branch in the https://git-tails.immerda.ch/ikiwiki.git Git repository has a fix for this problem.

--intrigeri

Merged in 3.20161219 --smcv

Posted Sun Nov 20 03:00:35 2016 Tags:

When IkiWiki uses discount to implement mdwn rendering, there is a workaround for https://rt.cpan.org/Ticket/Display.html?id=74016:

$t=~s/<style/<elyts/ig;
my $r=Text::Markdown::Discount::markdown($t);
$r=~s/<elyts/<style/ig;

However, this workaround also applies to indented text or text in backticks: if you write there is a bug involving the `<style>` tag, or use indentation like

you can use this markup:

    <style type="text/css">...</style>

then that gets turned into <elyts in the source before passing through markdown, comes out as &lt;elyts in the output HTML, and is rendered as <elyts by the browser. This makes it quite difficult to talk about HTML stylesheet markup on an IkiWiki instance (I had to use raw HTML in this bug report's source to avoid the bug).

I think the side-effect of the workaround is more damaging than the actual bug being worked around: I've never wanted to write inline style tags in the body of a Markdown page (which isn't even valid HTML) but I have certainly wanted to discuss style markup several times. The first couple of times I saw this happen, I thought it was some sort of misguided anti-cross-site-scripting filter...

--smcv

Fixed by passing the MKD_NOSTYLE flag to Discount instead. Unfortunately this isn't bound by Text::Markdown::Discount yet, so for now I'm hard-coding its numeric value. --smcv

Posted Sun Oct 5 08:40:26 2014
  1. We have a $srcdir/writable/page.mdwn source file in Git.
  2. ikiwiki is configured to allow edits via the CGI in writable/*, but nowhere else.
  3. Modify $srcdir/writable/page.mdwn, commit ⇒ commit $id.
  4. git mv $srcdir/writable/page.mdwn $srcdir/read-only/page.mdwn

⇒ The web interface allows reverting commit $id (presumably because it changes files only in $srcdir/writable). This operation effectively modifies $srcdir/read-only/page.mdwn, which feels wrong. My guess is that check_canchange does not take into account that Git will automatically detect that the file affected by the to-be-reverted commit has moved, and modify the file in its new location when reverting.

Working on it. In future please report non-public security vulnerabilities (such as authorization bypass) by private email to the maintainers, so that they are not visible to the general public until we have had a chance to fix the bug. --smcv

Sorry about that, I should clearly know better :/ --intrigeri

Fixed by using git revert --strategy=recursive --strategy-option=no-renames. I tried to do something more clever (doing the revert, and checking whether it made changes that aren't allowed) but couldn't get it to work in a reasonable time, so I'm going with the simpler fix. Fix committed, a release will follow later today.

CVE-2016-10026 has been assigned to this vulnerability. --smcv

You rock, thanks a lot! --intrigeri

Posted Sat Dec 17 07:12:01 2016

This may, strictly speaking, be a bug in the pandoc plugin, but I think it would be better to fix it in ikiwiki because of its kind (and maybe because I believe/hope pandoc will become the markdown dialect standard). For all I know it might not only affect pandoc tables.

When creating a simple table in pandoc-flavoured markdown,

1    2
---  ---
3    4

pandoc converts this to the html code

<table>
<thead>
<tr class="header">
<th align="left">1</th>
<th align="left">2</th>
</tr>
</thead>
<tbody>
<tr class="odd">
<td align="left">3</td>
<td align="left">4</td>
</tr>
</tbody>
</table>

<tr class="header"> causes it to be affected by style.css's

.header {
    margin: 0;
    font-size: 140%;
    font-weight: bold;
    line-height: 1em;
    display: block;
}

(more specifically by display: block;), which results in all header cells to cramp together in the first column.

The fix is easy: In style.css change .header { to .header tr:not(.header) {.

Alternatively, add the following code.

tr.header {
    display: table-row;
    }

I've added that last code snippet to my custom.css file. I admit .header tr:not(.header) is not especially elegant, but then again, I have almost no knowledge of CSS. There might be better solutions. (I don't even know why display: block; breaks the tables or why changing it to display: table-header; doesn't fix it but display: table-row; does :D )

This is essentially a conflict between ikiwiki's expectations for the definitions of CSS classes, and pandoc's expectations. The ikiwiki templates use class="header" to mean essentially the same thing as a HTML5 <header>, while Pandoc assumes a different meaning.

I think div.header, header.header { is probably a cleaner fix, and I have done that.

FYI, display: block breaks the tables because it makes the <tr> not be treated as a table row by the browser's layout engine. table-header is not a valid value for the CSS display attribute so that won't work.

--smcv

Posted Fri Sep 30 04:10:10 2016

In commits by Simon McVittie on Oct 5, 2014, the following was added to cgitemplate():

b0a35c81 (Simon McVittie   2014-10-05  61)  my $topurl = $config{url};
3b8da667 (Simon McVittie   2014-10-05  62)  if (defined $cgi && ! $config{w3mmode} && ! $config{reverse_proxy}) {
b0a35c81 (Simon McVittie   2014-10-05  63)      $topurl = $cgi->url;
b0a35c81 (Simon McVittie   2014-10-05  64)  }

I am trying to determine what was intended by this change. The variable $topurl is not used again in this function, so this is essentially dead code. --blipvert

If you look at git log -p IkiWiki/CGI.pm you'll see that at the time, $topurl was used further down the function. Later in the branch, commit 33f6026 "In html5 mode, generate a host- or protocol-relative for the CGI" made this conditional on ! $config{html5}.

Somewhat later, commit 490a1ec "Always produce HTML5 doctype and new attributes, but not new elements" repurposed $config{html5} from "use HTML5" to "use new HTML5 elements" - which meant that commit a052771 "Now that we're always using HTML5, can be relative" could remove the only code that used $topurl.

You are correct to say that computing $topurl is now dead code, and I have removed it. done --smcv

Posted Wed Dec 14 19:04:04 2016

When using inline with postform=yes, the user can click on the edit button without providing a title, and are allowed to save the page. This results in a file with a name like ".mdwn", which ikiwiki won't render. --Joey

done; made it error out in this case.

Posted Wed Sep 21 13:52:03 2016

The final </div> in recentchanges.tmpl gets wrapped in a <p> tag for some reason, resulting in the following invalid XHTML at the end of the RecentChanges page

<p></div></p>

I'll bet this is fixed if you use the markdown 1.2 prerelease, which has a much less buggy html parser. (Ah, I see below that was the case.) --Joey

Also, there is a problem with the <img> tags generated by the smiley plugin which end up wrapped in a <pre> tag in the inline diff output. <img> tags is not allowed within a <pre> block. Maybe the smiley plugin should be disabled on RecentChanges?

See Smileys in the block code, which is now fixed. --Joey

See the validator output for more details.


I'll add this here since it's related. I also noticed that the meta tags for redirected pages need to be closed in order to be valid XHTML:

<meta http-equiv="refresh" content="10; URL=../ikiwiki/pagespec/">

I'm noticing these problems because I'm serving ikiwiki-generated content as application/xhtml+xml (as opposed to text/html) in order to include inline MathML. Any invalid XHTML causes Firefox to halt all processing and throw an error. —Jason Blevins


Here is a simple patch for the refresh problem. I haven't figured out what's causing the recentchanges bug yet.

--JasonBlevins

Thanks, applied that patch. --Joey


It turns out that the invalid XHTML on the recent changes page is due to a bug in Markdown. I was using the packaged version of markdown in Ubuntu (Gutsy and markdown 1.0.1-6). Everything is fine after installing the most recent version of Text::Markdown from CPAN.

Note that the above patch for the redirect tag is still applicable and the smiley issue remains open. --JasonBlevins

This bug is done, all issues are fixed. --Joey

Posted Sun Mar 16 12:27:58 2008 Tags: