recently fixed bugs

There is an issue where an initial "inline" directive would be translated correctly but subsequent inlines of the same page would result in the raw contents of the ".po" file (ie. starting with the raw copyright headers!) being inserted into the page instead.

For example, given a "index.mdwn" containing:

[[!inline  pages="inline" raw="yes"]]
[[!inline  pages="inline" raw="yes"]]

… and an "index.de.po" of:

msgid "[[!inline  pages=\"inline\" raw=\"yes\"]]\n"
msgstr "[[!inline  pages=\"inline.de\" raw=\"yes\"]]\n"

… together with an "inline.mdwn" of:

This is inlined content.

… and an "inline.de.po" of:

msgid "This is inlined content."
msgstr "This is German inlined content."

§

This would result in the following translation:

This is the inlined content.
# SOME DESCRIPTIVE TITLE
# Copyright (C) YEAR Free Software Foundation, Inc.
# This file is distributed under the same license as the PACKAGE package.
# FIRST AUTHOR <EMAIL@ADDRESS>, YEAR.

… instead of (of course)

This is the inlined content.
This is the inlined content.

Initially proposed patch from Chris Lamb

Thank you Chris! I've reviewed the patch (with my "original author of the po plugin" hat on) and it looks good to me. I'm not 100% sure about alreadyfiltered being the best name for something that's not a predicated anymore but it's good enough. Then I wore my end-user hat and confirmed that with Chris' patch applied, the reproducer we had for this bug at Tails works fine. So IMO we're good to go and I recommend to apply this patch. Thanks in advance! -- intrigeri

Any update on getting this merged? — ?lamby, Fri, 24 Aug 2018 12:36:37 +0200

Indeed, would love to see this merged! What might be the next steps here? — ?lamby, Thu, 18 Oct 2018 17:57:37 -0400

I've filed this in Debian GNU/Linux at https://bugs.debian.org/911356?lamby, Thu, 18 Oct 2018 20:18:58 -0400

As I said on the Debian bug, I think we badly need test coverage for this sort of thing, otherwise it will regress. The po plugin is relatively complicated and hooks into lots of places in ikiwiki, and I don't think any of the active ikiwiki maintainers use it themselves, which means it can easily break (or have pre-existing bugs) without anyone realising.

For now I've added a failing test-case for this particular bug. --smcv


Review from smcv:

The patch attached to the Debian bug and the patch pasted here (which I've moved to an attachment) appear to be different, but I'm not going to do a line-by-line review of the patches and their differences for now because I'm not sure their approach is fully correct.

As we know, the two hardest things in computer science are naming, cache invalidation and off-by-one errors. Unfortunately this patch has issues with naming and cache invalidation. It's effectively turning the alreadyfiltered mechanism into a cache of memoized results of calling po_to_markup() on pages' content, keyed by the page name and the destpage, which is either the page name again or the name of a page into which the page is to be inlined (even though the result of po_to_markup() doesn't actually vary with the destpage).

This naturally raises the usual concerns about having a cache:

  • How large does it grow?
  • Do we invalidate it every time we need to?
  • Do we even need it?

The cache size is mainly a concern for large wikis being rebuilt. If you have a wiki with 1000 translated pages in 3 languages each, each of which is inlined into an average of one other page, then by the time you finish a rebuild you'll be holding 6000 translated pages in memory. If we change the alreadyfiltered mechanism to be keyed by the page name and not the (page, destpage) pair, that drops to 3000, but that's still O(pages * languages) which seems like a lot. As a general design principle, ikiwiki tries not to hold full content in RAM for more than the currently-processed page.

We invalidate the alreadyfiltered for a (page, page) pair in an editcontent hook, and we never invalidate (page, destpage) pairs for page != destpage at all. Are we sure there is no other circumstance in which the content of a page can change?

One of the things I tried doing for a simple solution was to remove the cache altogether, because I wasn't sure why we had this alreadyfiltered mechanism in the first place. This passes tests, which suggests that either the alreadyfiltered mechanism is unnecessary, or our regression test coverage for po is insufficient.

Looking back at the history of the po plugin, it seems that the alreadyfiltered mechanism was introduced (under a different name, with less abstraction) by intrigeri in commit 1e874b3f:

po plugin[filter]: avoid converting more than once per destfile

Only the first filter function call on a given {page,destpage} must convert it
from the PO file, subsequent calls must leave the passed $content unmodified.

Else, preprocessing loops are the rule.

I don't understand this. Under what circumstances would we pass content through the filter hooks, and then pass it back through the same filter hooks? Can we not do that, instead? If at all possible we should at least have test coverage for the situation where this happened (but I can't add this without knowing what it was).

I feel as though it should be an invariant that the output of a filter hook is never passed back through filter hooks: otherwise every filter hook would have to be able to be able to detect and skip processing its own output, which is not necessarily even possible. For instance, suppose you had a plugin with a filter that turned tab-separated text files into <table> markup: every HTML file that doesn't contain tabs is trivially a TSV file with one column, so you can't know whether a blob of text is TSV or HTML.

I wondered whether the loops referenced in 1e874b3f might have been fixed in 192ce7a2:

remove unnecessary and troublesome filter calls

This better defines what the filter hook is passed, to only be the raw,
complete text of a page. Not some snippet, or data read in from an
unrelated template.

Several plugins that filtered text that originates from an (already
filtered) page were modified not to do that. Note that this was not
done very consistently before; other plugins that receive text from a
page called preprocess on it w/o first calling filter.

The template plugin gets text from elsewhere, and was also changed not to
filter it. That leads to one known regression -- the embed plugin cannot
be used to embed stuff in templates now. But that plugin is deprecated
anyway.

Later we may want to increase the coverage of what is filtered. Perhaps
a good goal would be to allow writing a filter plugin that filters
out unwanted words, from any input. We're not there yet; not only
does the template plugin load unfiltered text from its templates now,
but so can the table plugin, and other plugins that use templates (like
inline!). I think we can cross that bridge when we come to it. If I wanted
such a censoring plugin, I'd probably make it use a sanitize hook instead,
for the better coverage.

For now I am concentrating on the needs of the two non-deprecated users
of filter. This should fix bugs/po_vs_templates, and it probably fixes
an obscure bug around txt's use of filter for robots.txt.

but I'm not sure that any of the redundant filtering removed in that commit was actually relevant to po users?

intrigeri, can you shed any light on this?

Naming is the easy part of this review: the alreadyfiltered family of functions are not named like cache getter/setter functions. This could be resolved by renaming.


Available in a git repository branch.
Branch: smcv/wip/po-filter-every-time
Author: Simon McVittie

If it's valid to remove the alreadyfiltered mechanism, my wip/po-filter-every-time branch does that. Please test?

intrigeri says this change works as intended on tails.org, so I've applied it. done --smcv

Posted Sat Dec 1 17:28:11 2018 Tags:

I use inline to create a blog like the below:

[[!inline  pages="./bugs/* and !./bugs/done and !./bugs/discussion and 
!link(patch) and !link(./bugs/done) and !./bugs/*/*"
actions=yes rootpage="./bugs" postform="yes" postformtext="请用一句话简述问题,然后点击 Edit 按钮添加具体细节" show=0]]

When I use posform to add a new page, it show:

Error: bad page name

Its url include a %2F, like below:

http://172.16.0.109/ikiwiki.cgi?do=blog&from=.%2Fbugs&subpage=1&title=aaa

I use ikiwiki 3.20180311


I have found that it is not "%2F"'s problem, it just that inline directive can not deal with Chinese char, the below link can work

http://172.16.0.109/ikiwiki.cgi?do=blog&from=aaa%2Fbugs&subpage=1&title=aaa

I don't think this is actually caused by the Chinese text. The problem is that you used rootpage="./bugs", which leads to the blog request handler generating an invalid page name. If you change it to rootpage="bugs" does that fix the error?

Ideally either the inline directive or the blog request handler would understand and remove ./, if it's something that makes sense in this context. --smcv


I have found the problem, it is inline plugin can not decode_utf8 "from", the below is patch:

From f79dde20b275707f70df2d481919a079abec6c19 Mon Sep 17 00:00:00 2001
From: Feng Shu <tumashu@163.com>
Date: Sun, 2 Dec 2018 08:38:34 +0800
Subject: [PATCH 1/2] Fix inline plugin can no set rootpage to a UTF-8 page

---
 IkiWiki/Plugin/inline.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/IkiWiki/Plugin/inline.pm b/IkiWiki/Plugin/inline.pm
index a85cd5d2f..f40956821 100644
--- a/IkiWiki/Plugin/inline.pm
+++ b/IkiWiki/Plugin/inline.pm
@@ -125,7 +125,7 @@ sub sessioncgi ($$) {
            error(gettext("please enter a page title"));
        }
        # if the page already exists, munge it to be unique
-       my $from=$q->param('from');
+       my $from=decode_utf8($q->param('from'));
        my $add="";
        while (exists $IkiWiki::pagecase{lc($from."/".$page.$add)}) {
            $add=1 unless length $add;
-- 
2.19.0

[Request for clarification removed]

I've now been able to reproduce this bug, and confirmed that your patch fixes it. Patch now applied.

(For other maintainers' reference: when testing Unicode bugs that relate to page titles, using Unicode that is considered to be punctuation, like ¬ or emoji, will probably not work; page titles treat [:alnum:] and ^[:alnum:] differently.)

In future bug reports it would be useful if you could provide a minimal example or test, perhaps on the sandbox on this wiki or as a unit test in t/*.t in the ikiwiki source code, that demonstrates this bug and would be fixed by the patch. If you've found multiple bugs, a separate example or test for each one would be easiest to deal with.

You can run all the tests with:

./Makefile.PL
make
make test

or a single test with something like:

./Makefile.PL
make
PERL5LIB=. ./t/git-cgi.t

--smcv

Posted Wed Nov 28 03:36:08 2018

Table directive can not deal with Chinese, when format csv

[[!table  format=csv data="""
a,b,c
1,2,你好
"""
]]

But the below example works well.

[[!table  format=csv data="""
a,b,c
1,2,3
"""
]]

The below example works well too

[[!table  format=dsv delimiter=, data="""
a,b,c
1,2,你好
"""
]]

You don't say what actually happens when you try this, but I hit something similar trying unicode symbols in a CSV-based table. (I wasn't aware of the DSV work-around. Thanks!) The specific error I get trying is

[\[!table Error: Cannot decode string with wide characters at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 243.]]

That file is owned by the libperl5 package, but I think I've seen an error mentioning Text::CSV i.e. libtext-csv-perl when I've encountered this before. -- Jon

A related problem, also fixed by using DSV, is messing up the encoding of non-ASCII, non-wide characters, e.g. £ (workaround was to use &pound; instead) -- Jon

Sorry, I have faced the same error: [[!table Error: Cannot decode string with wide characters at /usr/lib/x86_64-linux-gnu/perl/5.24/Encode.pm line 243.]] -- ?tumashu1


The below patch seem to deal with this problem:

From d6ed90331b31e4669222c6831f7a0f40f0677fe1 Mon Sep 17 00:00:00 2001
From: Feng Shu <tumashu@163.com>
Date: Sun, 2 Dec 2018 08:41:39 +0800
Subject: [PATCH 2/2] Fix table plugin can handle UTF-8 csv format

---
 IkiWiki/Plugin/table.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

From ad1a92c796d907ad293e572a168b6e9a8219623f Mon Sep 17 00:00:00 2001
From: Feng Shu <tumashu@163.com>
Date: Sun, 2 Dec 2018 08:41:39 +0800
Subject: [PATCH 2/2] Fix table plugin can handle UTF-8 csv format

---
 IkiWiki/Plugin/table.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/IkiWiki/Plugin/table.pm b/IkiWiki/Plugin/table.pm
index f3c425a37..7fea8ab1c 100644
--- a/IkiWiki/Plugin/table.pm
+++ b/IkiWiki/Plugin/table.pm
@@ -135,6 +135,7 @@ sub split_csv ($$) {
    my $csv = Text::CSV->new({ 
        sep_char    => $delimiter,
        binary      => 1,
+       decode_utf8 => 1,
        allow_loose_quotes => 1,
    }) || error("could not create a Text::CSV object");

@@ -143,7 +144,7 @@ sub split_csv ($$) {
    foreach my $line (@text_lines) {
        $l++;
        if ($csv->parse($line)) {
-           push(@data, [ map { decode_utf8 $_ } $csv->fields() ]);
+           push(@data, [ $csv->fields() ]);
        }
        else {
            debug(sprintf(gettext('parse fail at line %d: %s'), 
-- 
2.19.0

Thanks, I've applied that patch and added test coverage. done --smcv


I can confirm that the above patch fixes the issue for me. Thanks! I'm not an ikiwiki committer, but I would encourage them to consider the above. Whilst I'm at it, I would be really grateful for some input on support multi-row table headers which relates to the same plugin. Jon

Posted Mon Nov 26 04:44:34 2018

From the source of usage:

<a href="mailto:joey@ikiwiki.info">&#x6A;&#111;&#101;&#x79;&#64;i&#107;&#105;w&#105;&#107;&#x69;&#46;&#105;n&#x66;&#x6F;</a>

Text::Markdown obfuscates email addresses in the href= attribute and in the text. Apparently this can't be configured.

HTML::Scrubber doesn't set attr_encoded for its HTML::Parser, so the href= attribtute is decoded. Currently it seems it doesn't set attr_encoded for good reason: so attributes can be sanitized easily, e.g. as in htmlscrubber with $safe_url_regexp. This apparently can't be configured either.

So I can't see an obvious solution to this. Perhaps improvements to Text::Markdown or HTML::Scrubber can allow a fix.

One question is: how useful is email obfuscation? Don't spammers use HTML parsers?

I now see this was noted in the formatting discussion, and won't/can't be fixed. So I guess this is done. --Gabriel

I've patched mdwn.pm to prevent Text::Markdown from obfuscating the emails. The relevant commits are on the master branch of my "fork" of ikiwiki on Github:

  • 7d0970adbcf0b63e7e5532c239156f6967d10158
  • 52c241e723ced4d7c6a702dd08cda37feee75531

--Gabriel.

Thanks for coming up with a patch, but overriding Text::Markdown::_EncodeEmailAddress gets into its internals more than I'm comfortable with.

It would probably be best to add an option to Text::Markdown to let the email address munging be disabled. --Joey

Email obfuscation is very useful -- in practice, spammers apparently don't use HTML parsers -- according to the only published study I have read ( a 2003 study by the Center for Democracy and Technology cited by https://en.wikipedia.org/wiki/Address_munging ).

Posted Mon Jul 21 23:25:17 2008

Now that the rst plugin uses Python3, the test should test docutils existence also with Python3:

--- rst.t.orig  2018-02-28 10:41:06.000000000 +0000
+++ rst.t   2018-03-03 17:17:23.862702468 +0000
@@ -3,7 +3,7 @@
 use strict;

 BEGIN {
-   if (system("python -c 'import docutils.core'") != 0) {
+   if (system("python3 -c 'import docutils.core'") != 0) {
        eval 'use Test::More skip_all => "docutils not available"';
    }
 }

Applied, thanks. --smcv

Posted Sat Mar 3 13:21:45 2018

I'm having a hard time figuring out how the creation time, modification time, internal ctime and mtime fields (in indexdb) play along with the meta directive.

In some articles I write, I hardcode the creation and modification time, because they are imported from LWN.net, as such:

[[!meta  title="The cost of hosting in the cloud"]]
[[!meta  date="2018-02-281T12:00:00-0500"]]
[[!meta  updated="2018-03-12T14:22:45-0500"]]

But strangely, that article does not show up as "created" on "february 28th": it shows up as "Created 6 days and 20 hours ago", ie. "march 12th" (2018-03-12T18:29:12Z). That is strange, because that's the modification date (meta updated), not the creation date. Similarly, the "edited" date is 2018-03-19T14:47:45Z (40 minutes ago), which is more or less accurate: the page was modified some time ago, but shouldn't the meta tag override that? Note that the edited date matches the file's mtime field in the source directory:

w-anarcat@marcos:~$ LANG=C stat source/blog/2018-03-12-cost-of-hosting.mdwn 
  File: source/blog/2018-03-12-cost-of-hosting.mdwn
  Size: 14022           Blocks: 32         IO Block: 4096   regular file
Device: fd05h/64773d    Inode: 7905532     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1026/w-anarcat)   Gid: ( 1026/w-anarcat)
Access: 2018-03-19 11:19:21.237074935 -0400
Modify: 2018-03-19 10:47:45.000000000 -0400
Change: 2018-03-19 11:19:20.509065456 -0400
 Birth: -

This wouldn't be so much of a problem if that stuff was consistent: but it's not really. What's supposed to be the following article actually shows up before in the blog index which is rather annoying. It's also backwards in the RSS feed, which will possibly break some feed readers who will miss the new article.

That newer article shows up as Created 12 days and 15 hours ago (2018-03-07T00:00:00Z) and also "edited 40 minutes ago" (2018-03-19T14:51:29Z). It has the following meta:

[[!meta title="Easy photo galleries with Sigal"]] [[!meta date="2018-03-07T00:00:00+0000"]] [[!meta updated="2018-03-19T10:26:12-0400"]]

So there the date meta tag works: the creation date is correct, but obviously, it means it comes before the other article, because that one doesn't get loaded correctly.

By now, clever folks will have noticed the problem: it's with the first timestamp:

[[!meta  date="2018-02-281T12:00:00-0500"]]

There's an extra one in there! Obviously, february 281 is not a valid date... What happened is that I sometimes modify those dates by hand, and I sometimes mess it up. I actually messed it up twice there, the original timestamps were:

[[!meta  date="2018-02-281T12:00:00-0500"]]
[[!meta  updated="2018-14-22T14:22:45-0500"]]

The error, in the second one, is that I put the time instead of the date (!). I must have been very distracted, but still it's kind of hard to edit those timestamps correctly. I think the fundamental problem here is that Ikiwiki doesn't say anything when those timestamps can't be parsed properly. It seems to me there should be an error somewhere, if not directly in the page, at least in the rendering logs or somewhere, if the date cannot be parsed correctly.

So, long story short: shouldn't invalid dates in meta tags yield an error of some sort instead of being silently ignored? I spent half an hour figuring this one out, going as far as regenerating the whole wiki to try and see if it was a caching issue in indexdb...

Thanks!

-- anarcat

If you're reporting a bug, it would be helpful to lead with the actual bug and save the account of how you tried to debug it for later (or omit it). I've moved this from a forum post into a bug report.

The meta plugin uses Date::Parse::str2time from the TimeDate Perl library, so it has whatever error handling that has. However, historically any error has essentially been ignored, which I think is a bug.

[[!meta date]] and [[!meta updated]] have two purposes:

  • they create <meta name="date" content="xxx"> and <meta name="updated" content="xxx">
  • they change the ctime/mtime used by ikiwiki for sorting, etc.

I think the historical assumption was that even if the date can't be parsed for the second purpose, you still want the first purpose. However, you're right that this is really fragile, and the first purpose seems fairly niche anyway. In ikiwiki git master (to be released as 3.20180321 or later) I've made [[!meta date=...]] and [[!meta updated=...]] produce an error message if you don't have Date::Parse or if the date/time is malformed.

In the unlikely event that someone really wants <meta name="date" content="xxx"> without parsing the date, they can still use [[!meta name="date" content="xxx"]].

--smcv

To my defense, when I wrote this, I didn't consider this a bug: I was assuming the problem I was seeing was just some dumb mistake that I made and, indeed, there was one such formatting mistake.

But yeah, I could have re-edited this whole thing to make it look better. I'm sorry, but I was at the end of an already long yak-shaving session...

I wasn't sure if doing an error was the right way to go, as this might break rendering for existing sites... But I'm glad you fixed this anyways!

Thank you for the super-fast-response! :) I also tried updating the meta directive documentation so that it's a little more detailed about that stuff. I hope that's alright... -- anarcat

Posted Mon Mar 19 11:39:53 2018 Tags: done

we've recently updated Imagemagick in NixOS from version 6.9.7-6 to 6.9.8-4, and this change causes the Ikiwiki test suite to fail in t/img.t, like so:

#   Failed test at t/img.t line 119.
#          got: 'no image'
#     expected: '10x10'

#   Failed test at t/img.t line 129.
#          got: 'no image'
#     expected: '12x12'

#   Failed test at t/img.t line 130.
#          got: 'no image'
#     expected: '16x2'

#   Failed test at t/img.t line 134.
#          got: 'no image'
#     expected: '8x8'

#   Failed test at t/img.t line 135.
#          got: 'no image'
#     expected: '4x4'

#   Failed test at t/img.t line 136.
#          got: 'no image'
#     expected: '6x6'

#   Failed test at t/img.t line 138.
#          got: 'no image'
#     expected: '11x11'

#   Failed test at t/img.t line 139.
#          got: 'no image'
#     expected: '12x12'

#   Failed test at t/img.t line 140.
#          got: 'no image'
#     expected: '13x13'
# Looks like you failed 9 tests of 62.
t/img.t ........................
Dubious, test returned 9 (wstat 2304, 0x900)
Failed 9/62 subtests

Is this is a known problem and is there maybe a fix for this issue?

This was not a known bug before your report. It looks as though every time we use Image::Magick->Read(":foo.png"), which is (or was) ImageMagick's syntax for opening a file of unknown type without interpreting a prefix containing : as a special directive instead of part of the filename, it fails.

Please try re-running the test with better diagnostics using commit 4ace7dbb7 and report what it says. --smcv

I see the same issue on Fedora, with ImageMagic 6.9.9-19:

#   Failed test at t/img.t line 119.
#          got: 'no image: Exception 435: unable to open image `:t/tmp/out/imgconversions/10x-redsquare.png': No such file or directory @ error/blob.c/OpenBlob/2701'
#     expected: '10x10'
[...]

So it seems, that an empty coder prefix is not accepted anymore. To me it seems that this commit changed the behavior. Unfortunately, the commit message doens't tell us about the reasons behind. The commit is included from version 6.9.8-3 on.

This should now be fixed in git and in the next release. The test failure does not indicate a loss of functionality, unless you are using uncommon image formats enabled with img_allowed_formats: [everything], which is a potential security vulnerability because it exposes the attack surface of all ImageMagick decoder modules. --smcv

Posted Thu Jun 22 09:55:52 2017

For around 2 weeks, I've been getting an increasing quantity of nonspecific reports from users of login problems on ikiwiki sites, mostly joeyh.name and git-annex.branchable.com. A few users are still logging in successfully, but it seems to be hitting many users; post volume has gone down more than holidays would explain.

It doesn't seem limited to any login method; email and password have both been said not to work. (Openid too, but could be openid provider problem there.)

I have not managed to reproduce the problem myself, using firefox, firefox-esr, or chromium. --Joey

Otto Kekäläinen described to me a problem where email login to post a comment seemed to work; it displayed the comment edit form; but posting the form went back to the login page. Cookie problem?

Ok, to reproduce the problem: Log into joeyh.name using https. The email login link is a http link. The session cookie was set https-only. --Joey

The reason the edit form is able to be displayed is that emailauth sets up a session, in getsession(), and that $session is used for the remainder of that cgi call. But, a cookie for that session is not stored in the browser in this case. Ikiwiki does send a session cookie, but the browser seems to not let an existing https-only session cookie be replaced by a new session cookie that can be used with http. (If the emailed link, generated on https is opened in a different browser, this problem doesn't happen.) There may have been a browser behavior change here?

So what to do about this? Sites with the problem have redirect_to_https: 0 and the cgiurl is http not https. So when emailauth generates the url with cgiurl_abs, it's a http url, even if the user got to that point using https.

I suppose that emailauth could look at $ENV{HTTPS} same as printheader() does, to detect this case, and rewrite the cgiurl as a https url. Or, printheader() could just not set "-secure" on the cookie, but that does degrade security as MITM can then steal the cookie you're using on a https site.

Of course, the easy workaround, increasingly a good idea anyway, is to enable redirect_to_https.. --Joey

One of the users also reported a problem with password reset, and indeed, passwordauth is another caller of cgiurl_abs. (The only other caller, notifyemail, is probably fine.) The emailed password reset link also should be https if the user was using https. So, let's add a cgiurl_abs_samescheme that both can use. --Joey

fixed.. At least I hope that was the thing actually preventing most of the people from logging in. --Joey

Posted Thu Jan 4 19:00:45 2018

What I did

Upgraded from 3.20180105 to 3.20180228 (from pkgsrc). No change to installed Text::Markdown::Discount (0.11nb4 from pkgsrc, using upstream's bundled Discount library).

What I expected to happen

Markdown-style links to continue being rendered as before.

What actually happened

Markdown-style links stopped being links. Instead, they render the part in square brackets as ordinary text.

Proximate cause

In f46e429, if I comment out the MKD_GITHUBTAGS if block, the problem goes away.

Further causes and possible solutions

Some guesses:

  • Sufficiently old versions of the Discount library may break when passed unrecognized flags, in which case ikiwiki might want to version-check before passing flags
  • The version of the Discount library bundled with upstream Text::Markdown::Discount may be extremely old, in which case pkgsrc might want to make it depend instead on an external Discount package

This appears to be because MKD_GITHUBTAGS and MKD_LATEX both have numeric values that were previously used for the internal flag IS_LABEL, which strips HTML (its value has changed a couple of times).

Having thought about this a bit, I realise I can probe for the values of these flags by processing HTML that should have different results with the flag set or unset, which would be safer than just blindly using them.

Orthogonally, pkgsrc should probably use an up-to-date version of Discount, and we already know that Text::Markdown::Discount needs updating. --smcv

This should be fixed in current git. The mdwn module now detects what your version of Discount supports by trying several short HTML fragments that render differently under the different flags. --smcv

Posted Thu Mar 8 13:36:29 2018

Hi!

While working on Reproducible Builds for Tails, we noticed that the img plugin's output is not deterministic: PNG images embed timestamps.

The img-determinism branch in the https://git-tails.immerda.ch/ikiwiki.git Git repository has a fix for this problem + a new test (that fails without this change, and succeeds with the branch merged).

--intrigeri

Thanks, merged --smcv

Posted Fri Sep 1 15:38:29 2017 Tags: