Suppose a wiki has a source page a.mdwn, which is then moved to a.wiki. (Suppose both the mdwn and wikitext plugins are enabled, so this changes how "a" is rendered.) Currently, when the wiki is refreshed, ikiwiki doesn't notice the change and the page is not rebuilt.

I have a patch that fixes this. The relevant commit on my Github fork of ikiwiki is:

b6a3b8a683fed7a7f6d77a5b3f2dfbd14c849843

The patch (ab)uses%forcerebuild, which is meant for use by plugins. If, for some reason, a plugin deletes the page's entry in %forcerebuild, it won't be rebuilt.

This patch uncovers another problem. Suppose a wiki has a source page "a" (no extension) which is then moved to "a.mdwn" (or vice versa). ikiwiki fails when trying to create a directory "a" where there is a file "a" (or vice versa).

The same problem occurs if both "a" and "a.mdwn" exist in the wiki.

Thank you for looking into it!

On the use of forcerebuild, I think it's acceptable; plugins that unset it would break other plugins that set it, too.

cherry-picked --Joey

From the source of usage:

<a href="mailto:joey@ikiwiki.info">&#x6A;&#111;&#101;&#x79;&#64;i&#107;&#105;w&#105;&#107;&#x69;&#46;&#105;n&#x66;&#x6F;</a>

Text::Markdown obfuscates email addresses in the href= attribute and in the text. Apparently this can't be configured.

HTML::Scrubber doesn't set attr_encoded for its HTML::Parser, so the href= attribtute is decoded. Currently it seems it doesn't set attr_encoded for good reason: so attributes can be sanitized easily, e.g. as in htmlscrubber with $safe_url_regexp. This apparently can't be configured either.

So I can't see an obvious solution to this. Perhaps improvements to Text::Markdown or HTML::Scrubber can allow a fix.

One question is: how useful is email obfuscation? Don't spammers use HTML parsers?

I now see this was noted in the formatting discussion, and won't/can't be fixed. So I guess this is done. --Gabriel

I've patched mdwn.pm to prevent Text::Markdown from obfuscating the emails. The relevant commits are on the master branch of my "fork" of ikiwiki on Github:

• 7d0970adbcf0b63e7e5532c239156f6967d10158
• 52c241e723ced4d7c6a702dd08cda37feee75531

--Gabriel.

Thanks for coming up with a patch, but overriding Text::Markdown::_EncodeEmailAddress gets into its internals more than I'm comfortable with.

It would probably be best to add an option to Text;:Markdown to let the email address munging be disabled. --Joey

I've just noticed that attachment plugin escapes the underscore characters in attached filenames. For example, when I wanted to add foo_bar_baz.txt file, then Ikiwiki added file foo__95__bar__95__baz.txt to my Subversion repo. I hope that the filename is terribly ugly not only for me

Is it a bug or security feature? --Paweł

Update: It's not only problem with attached filenames. I have mysql/myisam_vs_ndb.mdwn page in my wiki and attached two images (myisam_vs_ndb_sql.png and myisam_vs_ndb_cpu.png) and one OpenDocument file (myisam_vs_ndb.ods). Ikiwiki placed them into myisam__95__vs__95__ndb subdirectory as myisam__95__vs__95__ndb__95__sql.png, myisam__95__vs__95__ndb__95__cpu.png and myisam__95__vs__95__ndb.ods files. When I click "Attachments" link, I can't see my uploaded files, because there are in another subdirectory (myisam__95__vs__95__ndb instead of myisam_vs_ndb). --Paweł

done, uses linkpage now.

There is currently a restriction in ikiwiki that there cannot be any symlinks in the source path. This is to deal with a security issue discussed here. The issue, as I understand it, is that someone might change a symlink and so cause things on the server to be published when the server admin doesn't want them to be.

I think there are two issues here:

• Symlinks with the source dir path itself, and
• Symlinks inside the source directory.

The first appears to me to be less of a security issue. If there is a way for a malicious person to change where that path points, then you have problems this check isn't going to solve. The second is quite clearly a security issue - if someone were to commit a symlink into the source dir they could cause lots of stuff to be published that shouldn't be.

Correct. However, where does the revision controlled source directory end? Ikiwiki has no way of knowing. It cannot assume that srcdir is in revision control, and everything outside is not. For example, ikiwiki's own source tree has the doc wiki source inside ikiwiki/doc. So to fully close the source dir symlink issue, it's best to, by default, assume that the revision controlled directories could go down arbitrarily deep, down to the root of the filesystem. --Joey

Fair point.

The current code seems to check this constraint at the top of IkiWiki/Render.pm at the start of refresh(). It seems to only check the source dir itself, not the subdirs. Then it uses File::Find to recuse which doesn't follow symlinks.

Now my problem: I have a hosted server where I cannot avoid having a symlink in the source path. I've made a patch to optionally turn off the symlink checking in the source path itself. The patch would still not follow symlinks inside the source dir. This would seem to be ok security-wise for me as I know that path is ok and it isn't going to change on me.

BTW, if you have a problem, please file it in todo or bugs in the future. Especially if you also have a patch. --Joey

Well, I was unsure I wasn't missing something. I wanted to discuss the concept of the patch as much as submit the patch. But, ok

Is there a huge objection to this patch?

patch updated.

diff --git a/IkiWiki/Render.pm b/IkiWiki/Render.pm
index 990fcaa..0fb78ba 100644
--- a/IkiWiki/Render.pm
+++ b/IkiWiki/Render.pm
@@ -260,13 +260,15 @@ sub prune ($) { #{{{

 sub refresh () { #{{{
    # security check, avoid following symlinks in the srcdir path
-   my $test=$config{srcdir};
-   while (length $test) {
-       if (-l $test) {
-           error("symlink found in srcdir path ($test)");
-       }
-       unless ($test=~s/\/+$//) {
-           $test=dirname($test);
+   if (! $config{allow_insecure_symlinks_in_path_to_srcdir}) {
+       my $test=$config{srcdir};
+       while (length $test) {
+           if (-l $test) {
+               error("symlink found in srcdir path ($test)");
+           }
+           unless ($test=~s/\/+$//) {
+               $test=dirname($test);
+           }
        }
    }

diff --git a/doc/ikiwiki.setup b/doc/ikiwiki.setup
index 10cb3da..eb86e49 100644
--- a/doc/ikiwiki.setup
+++ b/doc/ikiwiki.setup
@@ -203,4 +203,10 @@ use IkiWiki::Setup::Standard {
    # For use with the attachment plugin, a program that returns
    # nonzero if its standard input contains an virus.
    #virus_checker => "clamdscan -",
+   
+   # The following setting allows symlinks in the path to your
+   # srcdir.  Symlinks are still not followed within srcdir.
+   # Allowing symlinks to be followed, even in the path to srcdir,
+   # will make some setups insecure.
+   #allow_insecure_symlinks_in_path_to_srcdir => 0,
 }

No, I don't have a big objection to such an option, as long as it's extremely well documented that it will make many setups insecure. It would be nice to come up with an option name that makes clear that it's allowing symlinks in the path to the srcdir, but still not inside the srcdir. --Joey

Slightly modified version of patch applied. --Joey

Ok, I'll try to get it cleaned up and documented.

There is a second location where this can be an issue. That is in the front of the wrapper. There the issue is that the path to the source dir as seen on the cgi server and on the git server are different - each has symlinks in place to support the other. The current wrapper gets the absolute path to the source dir, and that breaks things for me. This is a slightly different, albeit related, issue to the one above. The following patch fixes things. Again, patch inline. Again, this patch could be cleaned up :). I just wanted to see if there was any chance of a patch like this being accepted before I bothered.

Patch updated:

index 79b9eb3..ce1c395 100644
--- a/IkiWiki/Wrapper.pm
+++ b/IkiWiki/Wrapper.pm
@@ -4,14 +4,14 @@ package IkiWiki;

 use warnings;
 use strict;
-use Cwd q{abs_path};
 use Data::Dumper;
 use IkiWiki;
+use File::Spec;

 sub gen_wrapper () { #{{{
-       $config{srcdir}=abs_path($config{srcdir});
-       $config{destdir}=abs_path($config{destdir});
-       my $this=abs_path($0);
+       $config{srcdir}=File::Spec->rel2abs($config{srcdir});
+       $config{destdir}=File::Spec->rel2abs($config{destdir});
+       my $this=File::Spec->rel2abs($0);
        if (! -x $this) {
                error(sprintf(gettext("%s doesn't seem to be executable"), $this
        }

ikiwiki uses absolute paths for srcdir, destdir and this because the wrapper could be run from any location, and if any of them happen to be a relative path, it would crash and burn.

Which makes perfect sense. It is annoying that abs_path() is also expanding symlinks.

I think the thing to do might be to make it check if srcdir and destdir look like an absolute path (ie, start with "/"). If so, it can skip running abs_path on them.

I'll do that. I assume something like File::Spec->filenameis_absolute( $path ); would have more cross-platformy goodness. hrm. I might see if File::Spec->rel2abs( $path ) ; will give absolute an path without expanding symlinks.

Patch using rel2abs() works well - it no longer expands symlinks.

That patch is applied now. --Joey

Since upgrading from Ikiwiki 2.20 to 2.32.3 (from Debian Lenny), I don't get hyperlinks to items which have a colon in their name anymore. This applies to both the normal and the archive view. Before the update, the links have been created as relative links, so they weren't usable either as the browser tried to take the part before the colon as protocol specification (as in e.g. http:). Iirc, this applied to the archive view only, links in the normal view were ok (will check this out again if it's important). Is there a way to quote each colon as %xy in the hyperlinks? Perhaps it's only a problem with my config, and not an actual bug...?

EDIT: I just found that in this wiki under http://ikiwiki.info/bugs/done/ the entry "mailto: links not properly generated in rss/atom feeds" also doesn't have a hyperlink - at least it's not a problem with my config only

madduck: I traced this down to htmlscrubber. If disabled, it works. If enabled, then $safe_url_regexp determines the URL unsafe because of the colon and hence removes the src attribute.

Digging into this, I find that RFC 3986 pretty much discourages colons in filenames:

A path segment that contains a colon character (e.g., "this:that") cannot be used as the first segment of a relative-path reference, as it would be mistaken for a scheme name. Such a segment must be preceded by a dot-segment (e.g., "./this:that") to make a relative- path reference.

The solution seems not to use colons.

In any case, htmlscrubber should get a new regexp, courtesy of dato: [^:]+($|\/). I have tested and verified this.

Commit/patch be0b4f60 fixes this.

July 21 2008: I update this bug report as it still seems to be an issue: E.g. when creating a subpage whose name contains a colon by inserting an appropriate wikilink in the parent page: the new page can be created using that link, but afterwards there won't be a link to this page. Like madduck said above it seems to be htmlscrubber removing this link. However everything works fine if the same page is being linked to from another subpage because in that case the resulting link starts with ../.

At the moment I see two possible solutions:

1. let all relative links at least start with ./. I haven't tested this.

2. Escape the colon in page titles. I created the following patch which worked for me:

   --- IkiWiki.pm.2.53-save        2008-07-08 15:56:38.000000000 +0200
   +++ IkiWiki.pm  2008-07-21 20:41:35.000000000 +0200
   @@ -477,13 +477,13 @@
   
   
    sub titlepage ($) { #{{{
           my $title=shift;
   -       $title=~s/([^-[:alnum:]:+\/.])/$1 eq ' ' ? '_' : "__".ord($1)."__"/eg;
   +       $title=~s/([^-[:alnum:]+\/.])/$1 eq ' ' ? '_' : "__".ord($1)."__"/eg;
           return $title;
    } #}}}
   
   
    sub linkpage ($) { #{{{
           my $link=shift;
   -       $link=~s/([^-[:alnum:]:+\/._])/$1 eq ' ' ? '_' : "__".ord($1)."__"/eg;
   +       $link=~s/([^-[:alnum:]+\/._])/$1 eq ' ' ? '_' : "__".ord($1)."__"/eg;
           return $link;
    } #}}}
   

What do you think about that? Does the patch have any side-effects I didn't see?

I almost really fixed this in 2.53, but missed one case. All fixed now AFAICS. --Joey

Using ikiwiki version 2.5gpa1 (the backport to Debian 3.1), I suddenly started getting the following error when rebuilding the wiki:

successfully generated /home/ikiwiki/cgi-bin/ikiwiki.cgi
Insecure dependency in rmdir while running with -T switch at /usr/share/perl5/IkiWiki/Render.pm line 242.
BEGIN failed--compilation aborted at (eval 5) line 130.

I've no idea what's happening (hey, I'm a C programmer), but I've hacked prune() to workaround this as follows:

use Scalar::Util qw(tainted);

sub prune ($) { #{{{
        my $file=shift;

        unlink($file);
        my $dir=dirname($file);
        if (!tainted($file) && $dir =~ /^(.*)$/) {
                $dir = $1;
        }
        while (rmdir($dir)) {
                $dir=dirname($dir);
                if (!tainted($file) && $dir =~ /^(.*)$/) {
                        $dir = $1;
                }
        }
} #}}}

Old versions of perl are known to have bugs with taint checking. I don't really support using ikiwiki with the perl 5.8.4 in debian oldstable, and would recommend upgrading. --Joey

I try to create wikilink in table. But it does not work. Here is example:

[[!table class=table1 data="""
[[wikilink_test|index]]
[[wikilink_test\|index]]
[wikilink test](/servers/webmail1)
"""]]

First two wikilink entries do not work.

The last one is url link and it works but it is not a wikilink. Or maybe it does not matter if I use URL links in stead of wikilinks for local wiki content?

fixed --Joey

works !! Great!

What exactly is a difference between wikilink and URL reference to the same page ?

ikiwiki will not be able to track pages linked using urls as having a link.

Trying to report this I found something weird. I changed in the example [[ with || because wiki renders something wrongly. You can see what I tried originally here:

[[!table class=table1 data="""
[[wikilink_test|servers/webmail1]]
[[wikilink_test|servers/webmail1]]
[wikilink test](/servers/webmail1)
"""]]

Please click edit to see unrendered text. First, it is not monospace-d (I have 4 spaces) and second, some wierd html is shown... Am I doing something wrong ?

See above for the right way to do it. Note that I also fixed a minor bug in ikiwiki to allow this. --Joey

Just curious ... if I wanted to have that block in monospace (four spaces in front of each line), how can I do that ?

The img plugin causes a taint failure if one tries to link a scaled image, e.g.

[[!img foo.png size=64x64]]

.ikiwiki.setup: Insecure dependency in mkdir while running with -T switch at /usr/lib/perl5/vendor_perl/5.8.8/IkiWiki.pm line 360.
BEGIN failed--compilation aborted at (eval 5) line 109.

If one omits the size argument it works. And if it worked once the taint failure will not happen again unless one rm -r's the destdir.

Seen with ikiwiki 2.30

And what version of perl? See Insecure dependency in mkdir et al. Also, the debian build of ikiwiki has taint checking disabled to avoid this perl bug. Did you build your own? Set NOTAINT=1 when building.. --Joey

perl-5.8.8, I've created a package for openSUSE. Will file perl bug there as well then.

I've gone ahead and turned off taint checking by default now. done

After editing a page pagename, ikiwiki redirects to pagename/index.html?updated. Ignoring for the moment that ?updated seems like a bad idea to begin with, this should at least not introduce /index.html into the URL.

The "?updated" works around caching issues with certain broken browsers, web proxys, and/or webservers. These assume that since the "?" is there, the page is not static, or is a different page, thus forcing the page to be reloaded and the edited version seen. So no, not a bad idea, really.

Removing the index.html would probably break this workaround. http://foo/bar/?updated will redirect to http://foo/bar/index.html, and said broken software will then display its old out of date cached version.

So, not changing this.

--Joey

The Monotone module still lacks support for setting up a post-commit hook, so commits made via monotone will not automatically update the wiki.

Here for future reference is the most recent version of support for that I've been sent. It's not yet working; there are path issues. --Joey

I think this was fixed in version 2.40. --Joey

diff --git a/IkiWiki/Rcs/monotone.pm b/IkiWiki/Rcs/monotone.pm
index cde6029..34f8f96 100644
--- a/IkiWiki/Rcs/monotone.pm
+++ b/IkiWiki/Rcs/monotone.pm
@@ -186,8 +186,9 @@ sub rcs_update () { #{{{
    check_config();

    if (defined($config{mtnsync}) && $config{mtnsync}) {
+       check_mergerc();
        if (system("mtn", "--root=$config{mtnrootdir}", "sync",
-                  "--quiet", "--ticker=none", 
+                  "--quiet", "--ticker=none", "--rcfile", $config{mtnmergerc},
                   "--key", $config{mtnkey}) != 0) {
            debug("monotone sync failed before update");
        }
@@ -604,4 +605,9 @@ __DATA__
               return true
          end
    }
+   function note_netsync_revision_received(new_id, revision, certs, session_id)
+       if (program_exists_in_path("ikiwiki-netsync-hook")) then
+           execute("ikiwiki-netsync-hook", new_id)
+       end
+   end
 EOF
diff --git a/IkiWiki/Wrapper.pm b/IkiWiki/Wrapper.pm
index 2103ea5..cff718c 100644
diff --git a/doc/ikiwiki.setup b/doc/ikiwiki.setup
index 1377315..0cbe27e 100644
--- a/doc/ikiwiki.setup
+++ b/doc/ikiwiki.setup
@@ -88,6 +88,16 @@ use IkiWiki::Setup::Standard {
        #   # Enable mail notifications of commits.
        #   notify => 1,
        #},
+       #{
+       #   # The monotone netsync revision received wrapper.
+       #   # Note that you also need to install a lua
+       #   # hook into monotone to make this work
+       #   # see: http://ikiwiki.info/rcs/monotone/
+       #   wrapper => "/usr/local/bin/ikiwiki-netsync-hook",
+       #   wrappermode => "04755",
+       #   # Enable mail notifications of commits.
+       #   notify => 1,
+       #},
    ],

    # Generate rss feeds for blogs?