When adding a new english page, the po files are created, committed, and then committed again. The second commit makes this change:

-"Content-Type: text/plain; charset=utf-8\n"
-"Content-Transfer-Encoding: ENCODING"
+"Content-Type: text/plain; charset=UTF-8\n"
+"Content-Transfer-Encoding: ENCODING\n"

Same thing happens when a change to an existing page triggers a po file update. --Joey

• The s/utf-8/UTF-8 part has been fixed.
• The ENCODING\n part is due to an inconsistency in po4a, which I've just send a patch for. --intrigeri

I resubmitted the patch to po4a upstream, sending it this time to their mailing-list: post archive. --intrigeri

If comments_allowdirectives is set, previewing a comment can run directives that create files. (Eg, img.) Unlike editpage, it does not keep track of those files and expire them. So the files will linger in destdir forever.

Probably when the user then tries to save the comment, ikiwiki will refuse to overwrite the unknown file, and will crash. --Joey

I'd like a way to always ask the RCS (Git) to update a file's mtime in refresh mode. This is currently only done on the first build, and later for --gettime --rebuild. But always rebuilding is too heavy-weight for this use-case. My options are to either manually set the mtime before refreshing, or to have ikiwiki do it at command. I used to do the former, but would now like the latter, as ikiwiki now generally does this timestamp handling.

From a quick look, the code in IkiWiki/Render.pm:find_new_files is relevant: if (! $pagemtime{$page}) { [...].

How would you like to tackle this?

--tschwinge

This could be done via a needsbuild hook. The hook is passed the list of changed files, and it should be safe to call rcs_getmtime and update the pagemtime for each.

That lets the feature be done by a plugin, which seems good, since rcs_getmtime varies between very slow and not very fast, depending on VCS.

AFAICS, the only use case for doing this is if you commit changes and then delay pushing them to a DVCS repo. Since then the file mtime will be when the change was pushed, not when it was committed. But I've generally felt that recording when a change was published to the repo of a wiki as its mtime is good enough. --Joey

I'm attempting a merge with the SVN plugin via the web interface with ikiwiki-3.20100403 and subversion 1.6.11.

The web interface says

Your changes conflict with other changes made to the page.

Conflict markers have been inserted into the page content. Reconcile the conflict and commit again to save your changes.

However there are no merge conflict markers in the page. My apache error log says:

[Fri Apr 30 16:43:57 2010] [error] [client 10.64.64.42] svn: Commit failed (details follow):, referer: https://unixwiki.ncl.ac.uk/ikiwiki.cgi
[Fri Apr 30 16:43:57 2010] [error] [client 10.64.64.42] svn: Authorization failed, referer: https://unixwiki.ncl.ac.uk/ikiwiki.cgi

-- Jon

Only way for this to be improved would be for the svn plugin to explicitly check the file for conflict markers. I guess it could change the error message then, but the actual behavior of putting the changed file back in the editor so the user can recommit is about right as far as error recovery goes. --Joey

Lighttpd apparently sets REDIRECT_STATUS=200 for the server.error-handler-404 page. This breaks the 404 plugin which checks this variable for 404 before processing the URI. It also doesn't seem to set REDIRECT_URL.

I was able to fix my server to check the REQUEST_URI for ikiwiki.cgi and to continue processing if it was not found, passing $ENV{SEVER_NAME} . $ENV{REQUEST_URI} as the first parameter to cgi_page_from_404. However, my perl is terrible and I just made it work rather than figuring out exactly what to do to get it to work on both lighttpd and apache.

This is with lighttpd 1.4.19 on Debian.

/cgi-bin/ikiwiki.cgi?do=goto also provides redirection in the same way, if that's any help? You might need to set the lighttpd 404 handler to that, then compose REDIRECT_URL from other variables if necessary.

I originally wrote the plugin for Apache; weakish contributed the lighttpd docs and might know more about how to make it work there. --smcv

As I said, I got it working for me, but somebody who knows perl should probably look at it with the aim of making it work for everyone. I considered having lighttpd construct a proper url for the 404 redirect itself, but I don't know if it can do something like that or not. For what it's worth, here's the change I made to the module:

sub cgi ($) {
        my $cgi=shift;
        if ($ENV{REQUEST_URI} !~ /ikiwiki\.cgi/) {
                my $page = cgi_page_from_404(
                        Encode::decode_utf8($ENV{SERVER_NAME} . $ENV{REQUEST_URI}),
                        $config{url}, $config{usedirs});
                IkiWiki::Plugin::goto::cgi_goto($cgi, $page);
        }

#       if (exists $ENV{REDIRECT_STATUS} &&
#           $ENV{REDIRECT_STATUS} eq '404') {
#               my $page = cgi_page_from_404(
#                       Encode::decode_utf8($ENV{REDIRECT_URL}),
#                       $config{url}, $config{usedirs});
#               IkiWiki::Plugin::goto::cgi_goto($cgi, $page);
#       }
}

When using Creole for markup, creating an external link appears to be impossible. Neither [[Outside URL|http://example.com]] nor <http://example.com> nor [Outside URL](http://example.com) work. The first gets rendered as a broken WikiLink, the second get eaten and the last is not parsed in anyway so you end up with that exact text in your page.

I'd have made this as a Creole page as a practical demonstration, but that doesn't seem possible here. Here's a page with an example: https://www.icanttype.org//demo/CreoleExternalLinks

It seems that rebuild a wiki (ikiwiki --rebuild) after changing the underlaydir config option doesn't remove the pages coming from the previous underlaydir.

I've noticed this with the debian package version 3.20100102.3~bpo50+1.

Perhaps it is possible to improve this or mention it in the manual page?

--prosper

--rebuild causes ikiwiki to throw away all its info about what it built before, so it will never clean up pages that have been removed, by any means. Suggest you do a --refresh, possibly followed by a --rebuild if that is really necessary. --Joey

I have a page with the name "umläute". When I try to remove it, ikiwiki says:

Error: ?umläute does not exist

I'm curious about the '?' in the "?umläute" message. Suggests that the filename starts with another strange character. Can I get a copy of a git repository or tarball containing this file? --Joey

I wrote the following patch, which seems to work on my machine. I'm running on FreeBSD 6.3-RELEASE with ikiwiki-3.20100102.3 and perl-5.8.9_3.

--- remove.pm.orig      2009-12-14 23:26:20.000000000 +0100
+++ remove.pm   2010-01-18 17:49:39.000000000 +0100
@@ -193,6 +193,7 @@
                        # and that the user is allowed to edit(/remove) it.
                        my @files;
                        foreach my $page (@pages) {
+                               $page = Encode::decode_utf8($page);
                                check_canremove($page, $q, $session);

                                # This untaint is safe because of the

The problem with this patch is that, in a recent fix to the same plugin, I made @pages come from $form->field("page"), and that, in turn is already run through decode_form_utf8 just above the code you patched. So I need to understand why that is apparently not working for you. (It works fine for me, even when deleting a file named "umläute" --Joey

Update, having looked at the file in the src of the wiki that is causing trouble for remove, it is: uml\303\203\302\244ute.mdwn
And that is not utf-8 encoded, which, represented the same would be: uml\303\244ute.mdwn

I think it's doubly-utf-8 encoded, which perhaps explains why the above patch works around the problem (since the page name gets doubly-decoded with it). The patch doesn't fix related problems when using remove, etc.

Apparently, on apoca's system, perl encodes filenames differently depending on locale settings. On mine, it does not. Ie, this perl program always creates a file named uml\303\244ute, no matter whether I run it with LANG="" or LANG="en_US.UTF-8":

perl -e 'use IkiWiki; writefile("umläute", "./", "baz")'

Remains to be seen if this is due to the older version of perl used there, or perhaps FreeBSD itself. --Joey

Update: Perl 5.10 fixed the problem. --Joey

To reproduce:

1. Add the backlinkbug plugin below to ikiwiki.
2. Create a page named test.mdwn somewhere in the wiki.
3. Refresh ikiwiki in verbose mode. Pages whose bestlink is the test.mwdn page will be printed to the terminal.
4. Delete test.mdwn.
5. Refresh ikiwiki in verbose mode again. The same pages will be printed to the terminal again.
6. Refresh ikiwiki in verbose mode another time. Now no pages will be printed.

bestlink() checks %links (and %pagecase) to confirm the existance of the page. However, find_del_files() does not remove the deleted page from %links (and %pagecase).

Since find_del_files removes the deleted page from %pagesources and %destsources, won't it make sense for bestlink() to check %pagesources first? --harishcm

This same problem turned out to also be the root of half of ikiwiki's second-oldest bug, bestlink change update issue.

Fixing it is really a bit involved, see commit f1ddf4bd98821a597d8fa1532092f09d3d9b5483. The fix I committed fixes bestlink to not return deleted pages, but only after the needsbuild and scan hooks are called. So I was able to fix it for every case except the one you gave! Sorry for that. To fix it during beedsbuild and scan, a much more involved approach would be needed. AFAICS, no existing plugin in ikiwiki uses bestlink in needsbuild or scan though.

If the other half of bestlink change update issue is fixed, maybe by keeping a copy of the old backlinks info, then that fix could be applied here too. --Joey

Cool that was fast! Well at least half the bug is solved For now I'll probably try using a workaround if using bestlink within the needsbuild or scan hooks. Maybe by testing if pagemtime equals zero. --harishcm

Yeah, and bestlink could also do that. However, it feels nasty to have it need to look at pagemtime. --Joey

#!/usr/bin/perl
# Plugin to reproduce bestlink returning deleted pages. 
# Run with ikiwiki in verbose mode.

package IkiWiki::Plugin::bestlinkbug;

use warnings;
use strict;
use IkiWiki 3.00;

sub import {
    hook(type => "getsetup", id => "bestlinkbug", call => \&getsetup);
    hook(type => "needsbuild", id => "bestlinkbug", call => \&needsbuild);
}

sub getsetup () {
    return
        plugin => {
            safe => 1,
            rebuild => 0,
        },
}

sub needsbuild (@) {
    my $needsbuild=shift;

    foreach my $page (keys %pagestate) {
        my $testpage=bestlink($page, "test") || next;

        debug("$page");
    }
}   

1

The map directive sort by pagename. That looks kind of odd, when used together with show=title. I would expect it to sort by title then.

This would be quite hard to fix. Map sorts the pages it displays by page name, which has the happy effect of making "foo/bar" come after "foo"; which it has to do, so that it can be displayed as a child of the page it's located in. If sorting by title, that wouldn't hold. So, map would have to be effectively totally rewritten, to build up each group of child pages, and then re-sort those. --Joey

Ok, you are right, that does would break the tree. This made me think that I do not need to generate a tree for my particular use case just a list, so i thought i could use inline instead. This created two new issues:

1. inline also does sort by pagename even when explicitly told to sort by title.

2. I cannot get inline to create a list when the htmltidy plugin is switched on. I have a template which is enclosed in an li tag, and i put the ul tag around the inline manually, but htmltidy breaks this. --martin

You might want to check if the report plugin solves your problem. It can sort by title, among other things. --KathrynAndersen

See also: sort parameter for map plugin and directive --smcv

[[!inline ]] is rendered with a space in front of the first closing bracket. --tschwinge

I don't think that complicating the directive parser is warrented by the minorness of this bug. The result that it outputs is still valid. --Joey

I'm trying to make a pretty theme for ikiwiki and I'm making progress (or at least I think I am :-). However I've noticed an issue when it comes to theming. On the front page the wiki name is put inside the "title" span and on all the other pages, it's put in the "parentlinks" span. See here:

From my dev home page:

<div class="header">
<span>
<span class="parentlinks">

</div>


From a sub-page of my dev home page:

<div class="header">
<span>
<span class="parentlinks">

</div>


I understand the logic behind doing this (on the front page it is the title as well as the name of the wiki) however if you want to do something different with the title of a page vs. the name of the wiki it makes things pretty tricky.

I'll just modify the templates for my own site but I thought I'd report it as a bug in the hopes that it will be useful to others.

Cheers,
Adam.

I just noticed that it's also different on the comments, preferences and edit pages. I'll come up with a diff and see what you guys think. -- Adam.

Example (from here):

[[`\[[!taglink TAG\]\]`|plugins/tag]]

gives:

[[\[[!taglink TAG\]\]|plugins/tag]]

Expected: there is a wikilink with the complex text as the displayed text. --Ivan Z.

aggregate takes a name parameter that specifies a global name for a feed. This causes some problems:

• If a site has multiple pages that aggregate, and they use the same name, one will win and get the global name, the other will claim it's working, but it's really showing what the other aggregated.
• If an aggregate directive is moved from page A to page B, and the wiki refreshed, aggregate does not realize the feed moved, and so it will keep aggregated pages under A/feed_name/*. To work around this bug, you have to delete A, refresh (maybe with --aggregate?), and then add B.

Need to find a way to not make the name be global. Perhaps it needs to include the name of the page that contains the directive?

Escaping pipe-symbol in ?taglink targets doesn't work as I wanted:

?smth with a pipe|about the \ ?smth with a pipe|about the

as opposed to simple wikilinks:

?a link to smth with a pipe ?a link to smth with a pipe

And it seems to work in pagespecs:

tagged:

link:

The remove plugin does not report an error if git rm fails. (It probably doesn't if other VCS backends fail too). This can happen for example if a page in your source directory is not a tracked file for whatever reason (in my case, due to renaming the files and forgetting to commit that change).

-- Jon

I was just hovering over the '...' next to the backlinks on a page on http://ikiwiki.info/. In terms of the size of my browser window, this was towards the bottom-right of the screen.

When I hovered over the '...', the additional backlinks float appeared. This caused the page length to grow down, meaning a horizontal scrollbar was added to the page. This meant the text reflowed, and the '...' moved outside of my mouse pointer region.

This caused an infinite loop of box appears... text moves, box disappears... box re-appears.. which was not very visually pleasant.

In general I think that the onhover float is a bit of bad UI. Even a truncated list of backlinks looks cluttered due to there being no delimiters. I moved to having an always-complete list of backlinks and having them as LI elements inside a UL to make it look neater, although I appreciate that would make some pages very long indeed.

How about doing something a little like toggle for the excess items instead?

-- Jon

An additional, related issue: if the box expands beyond the bottom of the page, you might move your mouse pointer to the scrollbar in order to move further down the list, but of course then you are outside the hover region.

-- Jon

I agree, browser handling of this CSS is often not good.

A toggle would be the perfect UI, but the heaviness of needing to include 30 lines of javascript to do it, plus then it only working with javascript enabled, is also not optimal.

Another idea would be to make the "..." a link to the ikiwiki cgi. The cgi could then have a mode that displays all the backlinks of a page in a list.

Yet another idea: Find some more refined CSS for handling a variable size popup.. --Joey

Error received when clicking on the "edit" link:

Error: [CGI::FormBuilder::AUTOLOAD] Fatal: Attempt to address non-existent field 'text' by name at /home/tealart/bin/share/perl/5.8.4/IkiWiki/CGI.pm line 112

Error received when following a "Create New Page" (eg. ?) link:

Error: [CGI::FormBuilder::AUTOLOAD] Fatal: Attempt to address non-existent field 'param' by name at /home/tealart/bin/share/perl/5.8.4/IkiWiki/Plugin/editpage.pm line 122

I could probably find several other flavors of this error if I went looking, but I trust you get the idea.

The CGI starts to render (this isn't the "you forgot to set the permissions/turn on the CGI" error) and then fails.

Further details:

• Running on shared hosting (dreamhost; but everything compiles, dependencies installed, the site generates perfectly, other CGIs work, the file permissions work).

• It's running perl 5.8.4, but I did upgrade gettext to 0.17

• the server is running gcc v3.3.5 (at this point, this is the main difference between the working system and my box.)

• I've removed the locale declarations from both the config file and the environment variable.

• I've also modified the page template and have my templates in a non standard location. The wiki compiles fine, with the template, but might this be an issue? The CGI script doesn't (seem) to load under the new template, but I'm not sure how to address this issue.

• All of the required/suggested module dependencies are installed (finally) to the latest version including (relevantly) CGI::FormBuilder 3.0501.

• I'm running ikiwiki v3.08. Did I mention that it works perfectly in nearly every other way that I've managed to test thusfar?

I suspect that your perl is too old and is incompatible with the version of CGI::FormBuilder you have installed.

Is so, it seems likely that the same error message can be reproduced by running a simple command like this at the command line:

perl -e 'use warnings; use strict; use CGI::FormBuilder; my $form=CGI::FormBuilder->new; $form->text("boo")'

--Joey

nope, that command produces no output.

I considered downgrading CGI::FormBuilder but I saw evidence of previous versions being incompatible with ikiwiki so I decided against that.

-- tychoish

In ikiwiki 2.66, SVG images are not recognized as images. In ikiwiki.pm, the hardcoded list of image file extensions does not include ".svg", which it probably should unless there's some other issue about rendering SVGs?

The 'img' plugin also seems to not support SVGs.

SVG images can only be included via an <object>, <embed>, or <iframe> tag. Or, perhaps as inline SVG. The htmlscrubber strips all three tags since they can easily be used maliciously. If doing inline SVG, I'd worry that the svg file could be malformed and mess up the html, or even inject javascript. So, the only options seem to be only supporting svgs on wikis that do not sanitize their html, or assuming that svgs are trusted content and embedding them inline. None of which seem particularly palatable.

I suppose the other option would be converting the svg file to a static image (png). The img plugin could probably do that fairly simply. --Joey

I'm working on inline SVG and MathML support in ikiwiki and I've modified my htmlscrubber to sanitize SVG and MathML using the whitelists from html5lib. Here's a patch. I've also made some notes about this here: svg.

I suspect that this bug may have caught the eye of anyone interested in this sort of thing. I'll elaborate a bit on my user page to avoid getting off-topic here. --JasonBlevins, October 21, 2008

A lot of strings in ikiwiki are hardcoded and not taken for locales resources through gettext. This is bad because ikiwiki is thus difficult to spread for non-english users.

I mean that, for instance in CGI.pm, line like:

my @buttons=("Save Page", "Preview", "Cancel");

should be written as

my @buttons=(gettext("Save Page"), gettext("Preview"), gettext("Cancel"));

Yes, these need to be fixed. But note that the localised texts come back into ikiwiki and are used in various places, including plugins. Including, possibly, third-party plugins. So localising the buttons would seem to require converting from the translations back into the C locale when the form is posted. --Joey

Wouldn't it be more easy to change all calls to the corrects ones (including in plugins) ? For instance in the same file (CGI.pm): elsif ($form->submitted eq gettext("Save Page")) {. That way no conversion to the C locale is needed. gettext use should just be publicized in documentation (at least in write). --bbb

It would be easy, but it could break third-party plugins that hardcode the english strings. It's also probably less efficient to run gettext over and over. --Joey

In standards templates things seems wrongly written too. For instance in page.tmpl line like:

<li><a href="<TMPL_VAR EDITURL>" rel="nofollow">Edit</a></li>

should be written as

<li><a href="<TMPL_VAR EDITURL>" rel="nofollow"><TMPL_VAR EDITURL_TEXT</a></li>

with EDITURL_TEXT variable initialized in Render.pm through a gettext call.

Am I wrong ?

No, that's not a sane way to localise the templates. The templates can be translated by making a copy and modifying it, or by using a tool to generate .mo files from the templates, and generate translated templates from .po files. (See l10n for one attempt.) But pushing the localisation of random strings in the templates through the ikiwiki program defeats the purpose of having templates at all. --Joey

If not I can spend some time preparing patches for such corrections if it can help.

-- bbb

A ?PageSpec that is entirely negated terminals, such as "!foo and !bar" matches all other pages, including all internal pages. This can lead to unexpected results, since it will match a bunch of recentchanges pages, etc.

Recall that internal-use pages are not matched by a glob. So "*" doesn't match them. So if the pagespec is "* and !foo and !bar", it won't match them. This is the much more common style.

There's an odd inconsistency with entirely negated pagespecs. If "!foo" matches page bar, shouldn't "" also match bar? But, the empty pagespec is actually special-cased to not match anything.

Indeed, it seems what would be best would be for "!foo" to not match any pages, unless it's combined with a terminal that positively matches pages ("* and !foo"). Although this would be a behavior change, with transition issues.

Another approach would be to try to detect the case of an entirely negated pagespec, and implicitly add "and !internal()" to it.

Either approach would require fully parsing the pagespec. And consider cases like "!(foo and !bar)". Doesn't seem at all easy to solve. --Joey

It occurs to me that at least one place in ikiwiki optimizes by assuming that pagespecs not mentioning the word "internal" never match internal pages. I wonder whether this bug could be solved by making that part of the definition of a pagespec, rather than a risky optimization like it is now? That seems strange, though - having this special case would make pagespecs significantly harder to understand. --smcv

The Atom and RSS templates use ESCAPE=HTML in the title elements. However, HTML-escaped characters aren't valid according to http://feedvalidator.org/.

Removing ESCAPE=HTML works fine, but I haven't checked to see if there are any characters it won't work for.

For Atom, at least, I believe adding type="xhtml" to the title element will work. I don't think there's an equivalent for RSS.

Removing the ESCAPE=HTML will not work, feed validator hates that just as much. It wants rss feeds to use a specific style of escaping that happens to work in some large percentage of all rss consumers. (Most of which are broken). http://www.rssboard.org/rss-profile#data-types-characterdata There's also no actual spec about how this should work.

This will be a total beast to fix. The current design is very clean in that all (well, nearly all) xml/html escaping is pushed back to the templates. This allows plugins to substitute fields in the templates without worrying about getting escaping right in the plugins -- and a plugin doesn't even know what kind of template is being filled out when it changes a field's value, so it can't do different types of escaping for different templates.

The only reasonable approach seems to be extending HTML::Template with an ESCAPE=RSS and using that. Unfortunately its design does not allow doing so without hacking its code in several places. I've contacted its author to see if he'd accept such a patch.

(A secondary bug is that using meta title currently results in unnecessry escaping of the title value before it reaches the template. This makes the escaping issues show up much more than they need to, since lots more characters are currently being double-escaped in the rss.)

--Joey

Update: Ok, I've fixed this for titles, as a special case, but the underlying problem remains for other fields in rss feeds (such as author), so I'm leaving this bug report open. --Joey

When committing a page like this one, with an escaped toc directive in it:

[[!toc  ]]

The recentchangesdiff comes back with it unescaped. Which can be confusing.

When you click on a broken link to create a new page, Ikiwiki lower-cases the new page's filename. I wish it wouldn't.

If I click on "Czars in Russia", I'd like Ikiwiki to create "Czars_in_Russia.mdwn", not "czars_in_russia.mdwn". Is this possible? --sabr

There's a simple patch that can do this:

-- a/IkiWiki.pm
+++ b/IkiWiki.pm
@@ -584,7 +584,7 @@ sub htmllink ($$$;@) {
            return "<span class=\"createlink\"><a href=\"".
                cgiurl(
                    do => "create",
-                   page => pagetitle(lc($link), 1),
+                   page => pagetitle($link, 1),
                    from => $lpage
                ).
                "\">?</a>$linktext</span>"

This is fine if you don't mind mixed or randomly cased filenames getting created. Otoh, if the link happened to start a sentence and so had its first letter upper-cased, that might not be desired.

Of course ikiwiki's case insensative, and there are other ways of creating pages that don't lower case them, including using the create a page form on a blog (as was done for this page..).

I'm undecided about making the above change by default though, or about making it a config option. Maybe it would be better to include both capitalisations in the select list that is used to pick the name for the newly created page. Then, which one is the default wouldn't much matter. (The non-lower cased one would probably be the best choice.) --Joey

Either of your proposed solutions (make it the default or include both in the pop-up menu) sounds fine to me. Which one is easier? --sabr

If you create a foo.rst containing only a number, such as "11", rendering results in the following error being thrown. (Now that I've fixed the error throwing code..):

exceptions.TypeError:coercing to Unicode: need string or buffer, int found

--Joey

Does this patch against proxy.py help?

index 5136b3c..545e226 100755
--- a/plugins/proxy.py
+++ b/plugins/proxy.py
@@ -88,7 +101,7 @@ class _IkiWikiExtPluginXMLRPCHandler(object):

    @staticmethod
    def _write(out_fd, data):
-        out_fd.write(data)
+        out_fd.write(str(data))
        out_fd.flush()

    @staticmethod

No, still the same failure. I think it's failing parsing the input data, (which perl probably transmitted as an int due to perl internals) not writing out its response. --Joey

It would be nice if the aggregate plugin would try to extract the m/ctime out of each post and touch the files on the filesystem appropriately, so that ikiwiki reflects the actual time of the post via the inline plugin, rather than the time when the aggregation ran to pull the post in. --madduck

Like this? (Existing code in aggregate.pm...) --Joey

# Set the mtime, this lets the build process get the right creation
# time on record for the new page.
utime $mtime, $mtime, pagefile($guid->{page})
    if defined $mtime && $mtime <= time;

I'll have to debug this, it's not working here... and this is an ikiwiki aggregator scraping another ikiwiki site.

Any news about this? --Joey

I am using mercurial as RCS backend and ikiwiki 2.40.

It seems that, when adding a blog post, it is not immediately commited to the mercurial repo. I have a page with this directive:

[[!inline  pages="journal/blog2008/* and !*/Discussion" show="0" feeds="no" actions="yes" rootpage="journal/blog2008"]]

When I add a blog post, I see it on the wiki but it doesn't appear on History or RecentChanges. If I run hg status on the wiki source dir, I see the new file has been marked as A (ie, a new file that has not been commited).

If I then edit the blog post, then the file gets commited and I can see the edit on History and RecentChanges. The creation of the file remains unrecorded. --?buo

Ikiwiki calls rcs_add() if the page is new, followed by rcs_commit(). For mercurial, these run respectively hg add and hg commit. If the add or commit fails, it will print a warning to stderr, you might check apache's error.log to see if there's anything there. --Joey

The problem was using accented characters (é, í) on the change comments. I didn't have an UTF-8 locale enabled in my setup file. By coincidence this happened for the first time in a couple of consecutive blog posts, so I was mistaken about the root of the problem. I don't know if you will consider this behavior a bug, since it's strictly speaking a misconfiguration but it still causes ikiwiki's mercurial backend to fail. A quick note in the docs might be a good idea. For my part, please close this bug, and thanks for the help. --?buo

So, in a non-utf8 locale, mercurial fails to commit if the commit message contains utf8? --Joey

(Sorry for the delay, I was AFK for a while.) What I am seeing is this: in a non-utf8 locale, using mercurial "stand-alone" (no ikiwiki involved), mercurial fails to commit if the commit message has characters such as á. If the locale is utf8, mercurial works fine (this is with mercurial 1.0).

However, the part that seems a bit wrong to me, is this: even if my locale is utf8, I have to explicitly set a utf8 locale in the wiki's setup file, or the commit fails. It looks like ikiwiki is not using this machine's default locale, which is utf8. Also, I'm not getting any errors on apache's error log.

Wouldn't it make sense to use the machine's default locale if 'locale' is commented out in the setup file?

Ikiwiki wrappers only allow whitelisted environment variables through, and the locale environment variables are not included currently.

But that's not the whole story, because "machine's default locale" is not very well defined. For example, my laptop is a Debian system. It has a locale setting in /etc/environment (LANG="en_US.UTF-8"). But even if I start apache, making sure that LANG is set and exported in the environment, CGI scripts apache runs do not see LANG in their environment. (I notice that /etc/init.d/apache explocitly forces LANG=C. But CGI scripts don't see the C value either.) Apache simply does not propigate its runtime environment to CGI scripts, and this is probably to comply with the CGI specification (although it doesn't seem to completly rule out CGI's being passed other variables).

If mercurial needs a utf-8 locale, I guess the mercurial plugin needs to check if it's not in one, and do something sane (either fail earlier, or complain, or strip utf-8 out of comments). --Joey

As far as I can tell, ikiwiki is not checking the SSL certificate of the remote host when using openid authentication. If so, this would allow for man-in-the-middle type attacks. Alternatively, maybe I am getting myself confused.

Test #1: Enter URL as openid server that cannot be verified (either because the certificate is self signed or signed by an unknown CA). I get no SSL errors.

Test #2: Download net_ssl_test from dodgy source (it uses the same SSL perl library, and test again. It seems to complain (on same site ikiwiki worked with) when it can't verify the signature. Although there is other breakage with the version I managed to download (eg. argument parsing is broken; also if I try to connect to a proxy server, it instructs the proxy server to connect to itself for some weird reason).

For now, I want to try and resolve the issues with net_ssl_test, and run more tests. However, in the meantime, I thought I would document the issue here.

-- Brian May

Openid's security model does not rely on the openid consumer (ie, ikiwiki) performing any sanity checking of the openid server. All the security authentication goes on between your web browser and the openid server. This may involve ssl, or not.

Note that I'm not an openid expert, and the above may need to be taken with a grain of salt. I also can make no general statements about openid being secure. --Joey

For example, my openid is "http://joey.kitenet.net/". If I log in with this openid, ikiwiki connects to that http url to determine what openid server it uses, and then redirects my browser to the server (https://www.myopenid.com/server), which validates the user and redirects the browser back to ikiwiki with a flag set indicating that the openid was validated. At no point does ikiwiki need to verify that the https url is good. --Joey

Ok, so I guess the worst that could happen when ikiwiki talks to the http address is that it gets intercepted, and ikiwiki gets the wrong address. ikiwiki will then redirect the browser to the wrong address. An attacker could trick ikiwiki to redirect to their site which always validates the user and then redirects back to ikiwiki. The legitimate user may not even notice. That doesn't so seem secure to me...

All the attacker needs is access to the network somewhere between ikiwiki and http://joey.kitenet.net/ or the ability to inject false DNS host names for use by ikiwiki and the rest is simple.

-- Brian May

I guess that the place to add SSL cert checking would be in either LWPx::ParanoidAgent or Net::OpenID::Consumer. Adding it to ikiwiki itself, which is just a user of those libraries, doesn't seem right.

It's not particularly clear to me how a SSL cert can usefully be checked at this level, where there is no way to do anything but succeed, or fail; and where the extent of the check that can be done is that the SSL cert is issued by a trusted party and matches the domain name of the site being connected to. I also don't personally think that SSL certs are the right fix for DNS poisoning issues. --Joey

I was a bit vague myself on the details on openid. So I looked up the standard. I was surprised to note that they have already considered these issues, in section 15.1.2, http://openid.net/specs/openid-authentication-2_0.html#anchor41.

It says:

"Using SSL with certificates signed by a trusted authority prevents these kinds of attacks by verifying the results of the DNS look-up against the certificate. Once the validity of the certificate has been established, tampering is not possible. Impersonating an SSL server requires forging or stealing a certificate, which is significantly harder than the network based attacks."

With regards to implementation, I am surprised that the libraries don't seem to do this checking, already, and by default. Unfortunately, I am not sure how to test this adequately, see Debian bug #466055. -- Brian May

I think Crypt::SSLeay already supports checking the certificate. The trick is to get LWP::UserAgent, which is used by LWPx::ParanoidAgent to enable this checking.

I think the trick is to set one of the the following environment variables before retrieving the data:

$ENV{HTTPS_CA_DIR} = "/etc/ssl/certs/";
$ENV{HTTPS_CA_FILE} = "/etc/ssl/certs/file.pem";

Unfortunately I get weird results if the certificate verification fails, see Debian bug #503440. It still seems to work though, regardless.

-- Brian May

Pages under templates/ are invalid (in fact, not only invalid, but also not well-formed) xhtml pages.

This problem is especially serious when you change extension from .html to .xhtml in ikiwiki.setup and use Firefox. Since Firefox will display a error message only for not well-formed application/xhtml+xml pages.

It seems that HTML::Template also support <!--Variable--> syntax instead of <Variable>. Chaning to this syntax will solve this problem, I guess.

Even if changed to <!-- TMPL_VAR --> style, the problem may still exist if the template contains if else block.

Maybe just encode all < and > when compling pages within the templates folder will solve this problem.

I never noticed this bug, since it only happens if the htmlscrubber is disabled. --Joey

After installing IkiWiki 2.16 on Mac OS X 10.4 server I attempted to use "/Library/Application\ Support/IkiWiki/Working\ Copies" as the parent of my $SRCPATH and get "skipping bad filename" errors for any .mdwn file in that directory:

skipping bad filename /Library/Application Support/IkiWiki/Working Copies/ikiwikinewt/index.mdwn

Tthe .ikiwiki directory is correctly created in that directory. I switched to using a path with no spaces and it works correctly.

In markdown syntax, none of the other special characters get processed inside a code block. However, in ikiwiki, wiki links and preprocessor directives still get processed inside a code block, requiring additional escaping. For example, [links don't work](#here), but a <a href="../ikiwiki/wikilink/">wikilink</a> becomes HTML. --JoshTriplett

Indented lines provide a good way to escape a block of text containing markdown syntax, but ikiwiki links like [[this]] are still interpreted within such a block. I think that intepretation should not be happening. That is I should be able to write:

<span class="createlink"><a href="http://ikiwiki.info/ikiwiki.cgi?page=this&amp;from=bugs%2Fwiki_links_still_processed_inside_code_blocks&amp;do=create" rel="nofollow">?</a>this</span>

and have it render like:

[[this]]

--?cworth

Has there been any progress or ideas on this bug recently? I use an expanded CamelCase regexp, and without much escaping in freelink text, or url links, or in codeblocks I get IkiWiki's attempt at creating a "link within a link".

I have no ideas other than perhaps once IkiWiki encounters [[ or the position is reset with a backreference from a CamelCased word, further processing of wikilinks is disabled until the position is reset and a "no not makelinks" flag or variable is cleared.

I've come up with some really ugly workarounds to handle case specific stuff like codeblocks but the problem creeps up again and again in unexpected places. I'd be happy to come up with a patch if anyone has a bright idea on a nice clean way (in theroy) to fix this. I'm out of ideas.

--CharlesMauch

I've moved the above comment here because it seems to be talking about this bug, not the similar Smileys bug.

In the case of either bug, no, I don't have an idea of a solution yet. --Joey

I've now solved a similar bug involving the smiley plugin. The code used there should give some strong hints how to fix this bug, though I haven't tried to apply the method yet. --Joey

Debian bug #487397

The brokenlinks plugin falsely complains that formatting has a broken link to smileys, if the smiley plgin is disabled. While the page links to it inside a conditional, and so doesn't show the link in this case, ikiwiki scans for links w/o looking at conditionals and so still thinks the page contains the link.

This bug is described here:

http://kitenet.net/~joey/blog/entry/OpenID/discussion/

If sandbox/page.mdwn has been generated and sandbox/sidebar.mdwn is created, the sidebar is only added to sandbox and none of the subpages. --TaylorKillian

Yes, a known bug. As noted in the code: --Joey

# FIXME: This isn't quite right; it won't take into account
# adding a new sidebar page. So adding such a page
# currently requires a wiki rebuild.
add_depends($page, $sidebar_page);

RSS output contains relative links. Ie. http://kitenet.net/~joey/blog/index.rss contains a link to http://kitenet.net/~joey/blog/../blog.html

Web browsers don't word-wrap lines in submitted text, which makes editing a page that someone wrote in a web browser annoying (gqip is vim user's friend here). Is there any way to improve this?

See "using the web interface with a real text editor" on the tips page. --JoshTriplett

Would it be useful to allow a "max width" plugin, which would force on commit the split of long lines ?

Please, no. That would wreak havoc on code blocks and arguments to preprocessor directives, and it would make bulleted lists and quoted blocks look bogus (because the subsequent lines would not match), among other problems. On the other hand, if you want to propose a piece of client-side JavaScript that looks at the active selection in a text area and word-wraps it, and have a plugin that adds a "Word-Wrap Selection" button to the editor, that seems fine. --JoshTriplett

The header of subpages always links to its "superpage", even if it doesn't exist. I'm not sure if this is a feature or a bug, but I would certainly prefer that superpages weren't mandatory.

For example, if you are in 'example/page.html', the header will be something like 'wiki / example / page'. Now, if 'example.html' doesn't exist, you'll have a dead link for every subpage.

This is a bug, but fixing it is very tricky. Consider what would happen if example.mdwn were created: example/page.html and the rest of example/* would need to be updated to change the parentlink from a bare word to a link to the new page. Now if example.mdwn were removed again, they'd need to be updated again. So example/* depends on example. But it's even more tricky, because if example.mdwn is modified, we don't want to rebuild example/*!

ikiwiki doesn't have a way to represent this dependency and can't get one without a lot of new complex code being added.

Note that this code has now been added. In new terms, example/* has a presence dependency on example. So this bug is theoretically fixable now. --Joey

For now the best thing to do is to make sure that you always create example if you create example/foo. Which is probably a good idea anyway..

Note that this bug does not exist if the wiki is built with the "usedirs" option, since in that case, the parent link will link to a subdirectory, that will just be missing the index.html file, but still nicely usable. --Joey

http://www.gnu.org/software/hurd/hurd/translator/writing.html does not exist. Then, on http://www.gnu.org/software/hurd/hurd/translator/writing/example.html, in the parentlinks line, writing links to the top-level index file. It should rather not link anywhere at all. --tschwinge

So, the bug has changed behavior a bit. Rather than a broken link, we get a link to the toplevel page. This, FWIW, is because the template now uses this for each parentlink:

<a href="<TMPL_VAR URL>"><TMPL_VAR PAGE></a>/

Best workaround is still to enable usedirs. --Joey

• Has bugs updating things if the bestlink of a page changes due to adding/removing a page. For example, if Foo/Bar links to "Baz", which is Foo/Baz, and Foo/Bar/Baz gets added, it will update the links in Foo/Bar to point to it, but will forget to update the backlinks in Foo/Baz.

  The buggy code is in refresh(), when it determines what links, on what pages, have changed. It only looks at changed/added/deleted pages when doing this. But when Foo/Bar/Baz is added, Foo/Bar is not changed -- so the change it its backlinks is not noticed.

  To fix this, it needs to consider, when rebuilding Foo/Bar for the changed links, what oldlinks Foo/Bar had. If one of the oldlinks linked to Foo/Baz, and not links to Foo/Bar/Baz, it could then rebuild Foo/Baz.

  Problem is that in order to do that, it needs to be able to tell that the oldlinks linked to Foo/Baz. Which would mean either calculating all links before the scan phase, or keeping a copy of the backlinks from the last build, and using that. The first option would be a lot of work for this minor issue.. it might be less expensive to just rebuild all pages that Foo/Bar links to.

  Keeping a copy of the backlinks has some merit. It could also be incrementally updated.

  This old bug still exists as of 031d1bf5046ab77c796477a19967e7c0c512c417.

• And if Foo/Bar/Baz is then removed, Foo/Bar gets a broken link, instead of changing back to linking to Foo/Baz.

  This part was finally fixed by commit f1ddf4bd98821a597d8fa1532092f09d3d9b5483.

If a file in the srcdir is removed, exposing a file in the underlaydir, ikiwiki will not notice the removal, and the page from the underlay will not be built. (However, it will be if the wiki gets rebuilt.)

This problem is caused by ikiwiki storing only filenames relative to the srcdir or underlay, and mtime comparison not handling this case.

A related problem occurs if changing a site's theme with the theme plugin. The style.css of the old and new theme often has the same mtime, so ikiwiki does not update it w/o a rebuild. This is worked around in theme.pm with a special-purpose needsbuild hook. --Joey