If you've found a bug in ikiwiki, post about it here. TODO items go elsewhere. Link items to done when done.

Also see the Debian bugs.

There are 89 "open" bugs:

RSS Add a new bug titled:
< thm> joeyh: ping
< thm> can you update the embedded jquery-ui? (for cve 
       2010-5312, and/or 2012-6662)

I'll do this next time I spend some time on ikiwiki unless Joey or Amitai gets there first.

It doesn't look as though we actually use the vulnerable functionality.

--smcv

Posted Wed Nov 26 08:46:28 2014

We have a weird spam problem on our site - must be something via CGI.

see http://www.dk0tu.de/recentchanges/discussion/

The content is changing frequently without being checked into the git repository. Any ideas?

--?bastla

Please check your web server logs for any error messages from the CGI. It seems likely that the spammer is editing that page but the changes are somehow not getting committed or pushed.

I can't tell you much without knowing details of your setup. For instance, are you using the suggested git repository setup shown in the diagram on the git page, or something different? Can you publish a (possibly censored) setup file somewhere?

It would probably also be worthwhile to compare the git history of srcdir/.git with the git history of the bare repository, if you have one.

To recover, you could undo the spam in the srcdir (as the user ID that owns the wiki), commit that, and merge with the bare repository if necessary.


When I tried editing the spammed page to clear it, I got this error:

Error: /srv/www/Kurse/AFu-Lizenz/e09.tex independently created, not overwriting with version from Kurse/AFu-Lizenz/e09.tex

Your srcdir and destdir seem to have got out of sync. You might need to rebuild the wiki.

(I think I'd have received the same error for any edit right now.)

If you're going to enable completely anonymous editing, I recommend monitoring the wiki more carefully. It might be useful to enable the syslog option so that wiki errors go to the system log.

--smcv

Posted Thu Nov 20 19:40:04 2014

I have here a site that uses the po plugin, and recently had this change committed to its setup:

 po_slave_languages:
 - de|Deutsch
 - fr|Français
-- ja|日本語
-- tr|Türkçe

The change was made by the web UI, so it must have involved a site rebuild at the time, as that configuration item has rebuild => 1.

Some days after that config change, a push caused ikiwiki refresh to fail:

remote: /home/b-udm/public_html/Discussion/index.ja.html independently created, not overwriting with version from Discussion.ja

Rebuilding the wiki cleared that up, but it seems that po plugin config changes can lead to follow-on problems of this sort.

The site still has a source/index.ja.po. And it has public_html/index.ja.html, as well as public_html/index.ja/index.html.

--Joey

Posted Thu Oct 23 13:58:00 2014

http://developers.google.com/speed/pagespeed/insights/

indicates the viewport on mobile needs to be configured, e.g. <meta name=viewport content="width=device-width, initial-scale=1"> in the header of

http://source.ikiwiki.branchable.com/?p=source.git;a=blob;f=templates/page.tmpl;

This seems a lot like an "unbreak my application" option but OK... presumably the motivation for this being opt-in is that "most" websites have some sort of hard-coded fixed-width layout suitable for a proportion of desktop browsers, rather than being responsive to window size like they should have been all along. --smcv

Further more:

  • fonts need to be tweaked

    Suggestions?

    (Note that Joey has generally rejected stylistic changes to the default anti-theme; enhancing the other themes would be OK though.) --smcv

  • XHTML should be dropped !

    Already in the to-do list: generate HTML5 by default. --smcv

I'm practicing this on http://dabase.com/ with http://source.dabase.branchable.com/?p=source.git;a=blob;f=templates/page.tmpl;

-<TMPL_IF FORCEBASEURL><base href="<TMPL_VAR FORCEBASEURL>" /><TMPL_ELSE>
-<TMPL_IF BASEURL><base href="<TMPL_VAR BASEURL>" /></TMPL_IF>
You probably don't want to delete those. It breaks the CGI. --smcv

Posted Wed Oct 15 00:18:09 2014

When IkiWiki uses discount to implement mdwn rendering, there is a workaround for https://rt.cpan.org/Ticket/Display.html?id=74016:

$t=~s/<style/<elyts/ig;
my $r=Text::Markdown::Discount::markdown($t);
$r=~s/<elyts/<style/ig;

However, this workaround also applies to indented text or text in backticks: if you write there is a bug involving the `<style>` tag, or use indentation like

you can use this markup:

    <style type="text/css">...</style>

then that gets turned into <elyts in the source before passing through markdown, comes out as &lt;elyts in the output HTML, and is rendered as <elyts by the browser. This makes it quite difficult to talk about HTML stylesheet markup on an IkiWiki instance (I had to use raw HTML in this bug report's source to avoid the bug).

I think the side-effect of the workaround is more damaging than the actual bug being worked around: I've never wanted to write inline style tags in the body of a Markdown page (which isn't even valid HTML) but I have certainly wanted to discuss style markup several times. The first couple of times I saw this happen, I thought it was some sort of misguided anti-cross-site-scripting filter...

--smcv

Posted Sun Oct 5 08:40:26 2014

What I did

A friend reported this, and I'm seeing it too. With 3.20140916, on a system with Python 2.7 and 3.4 (and little else) installed, I tried to run the auto.setup:

:; ikiwiki --setup /etc/pkg/ikiwiki/auto.setup
What will the wiki be named? Import Errors
What revision control system to use? git
Which user (wiki account or openid) will be admin? schmonz


Setting up Import Errors ...
Importing /Users/schmonz/ImportErrors into git
Initialized empty shared Git repository in /Users/schmonz/ImportErrors.git/
Initialized empty Git repository in /Users/schmonz/ImportErrors/.git/
[master (root-commit) 20b1128] initial commit
 1 file changed, 1 insertion(+)
 create mode 100644 .gitignore
Counting objects: 3, done.
Writing objects: 100% (3/3), 230 bytes | 0 bytes/s, done.
Total 3 (delta 0), reused 0 (delta 0)
To /Users/schmonz/ImportErrors.git
 * [new branch]      master -> master
Directory /Users/schmonz/ImportErrors is now a clone of git repository /Users/schmonz/ImportErrors.git
Traceback (most recent call last):
  File "/usr/pkg/lib/ikiwiki/plugins/rst", line 45, in <module>
    from proxy import IkiWikiProcedureProxy
  File "/usr/pkg/lib/ikiwiki/plugins/proxy.py", line 41, in <module>
    import xml.parsers.expat
  File "/usr/pkg/lib/python3.4/xml/parsers/expat.py", line 4, in <module>
    from pyexpat import *
ImportError: No module named 'pyexpat'


Creating wiki admin schmonz ...
Choose a password:
[...]

What I expected

I expected to get a basic site.

What happened instead

I got a basic site with some Python error messages.

Likely fix

Looks like proxy.py needs the trick from Debian bug #637604 so that it can defer a few imports (at least xml.parsers.expat and the XML-RPC libs) until the methods using them are called. --schmonz


It's more complicated than I thought. Findings and questions so far:

Failing to load an external plugin should be an error

When a typical Perl plugin fails to load (say, by failing to compile), IkiWiki::loadplugin() throws an exception. For XML-RPC plugins written in any language, ikiwiki assumes loading succeeded.

Let's take plugins/rst as an example. It's written in Python and uses proxy.py to handle XML-RPC communication with ikiwiki. Let's say that proxy.py compiles, but rst itself doesn't. We'd like ikiwiki to know the plugin isn't loaded, and we'd like an error message about it (not just the Python errors).

Now let's say rst would be fine by itself, but proxy.py doesn't compile because some of the Python modules it needs are missing from the system. (This can't currently happen on Debian, where libpython2.7 includes pyexpat.so, but pkgsrc's python27 doesn't; it's in a separate py-expat package.) As before, we'd like ikiwiki to know rst didn't load, but that's trickier when the problem lies with the communication mechanism itself.

For the tricky case, what to do? Some ideas:

  • Figure out where in auto.setup we're enabling rst by default, and stop doing that
  • In pkgsrc's ikiwiki package, add a dependency on Python and py-expat just in case someone wants to enable rst or other Python plugins

For the simple case, I've tried the following:

Available in a git repository branch.
Branch: schmonz/external-plugin-loading
Author: schmonz
  • In IkiWiki::Plugin::external::import(), capture stderr
  • Before falling off the end of IkiWiki::Plugin::external::rpc_call(), if the command had been 'import' and stderr is non-empty, throw an exception
  • In IkiWiki::loadplugin(), try/catch/throw just like we do with regular non-external plugins

With these changes, we have a test that fails when an external plugin can't be loaded (and passes, less trivially, when it can). Huzzah! (I haven't tested yet whether I've otherwise completely broken the interface for external plugins. Not-huzzah!) --schmonz

Posted Fri Sep 19 08:41:10 2014 Tags:

I'm trying to put a list of tags in a table, so I carefully make a newline-free taglist.tmpl and then do:

| [[!inline  pages="link(/category/env)" feeds=no archive=yes sort=title template=taglist]] |

but there's a line in inline.pm that does:

return "&lt;div class=\"inline\" id=\"$#inline\"&gt;&lt;/div&gt;\n\n";

And the extra newlines break the table. Can they be safely removed?

If you want an HTML table, I would suggest using an HTML table, which should pass through Markdown without being interpreted further. To avoid getting the <div> inside the <table> you can use:

[[!inline  pages="link(/category/env)" feeds=no archive=yes sort=title template=tagtable]]

where tagtable.tmpl looks like

<TMPL_IF FIRST>
<table><tr>
</TMPL_IF>

<td>your tag here</td>

<TMPL_IF LAST>
</tr></table>
</TMPL_IF>

I don't think you're deriving much benefit from Markdown's table syntax if you have to mix it with HTML::Template and ikiwiki directives, and be pathologically careful with whitespace. "Right tool for the job" and all that :-)

When I edited this page I was amused to find that you used HTML, not Markdown, as its format. It seems oddly appropriate to my answer, but I've converted it to Markdown and adjusted the formatting, for easier commenting. --smcv

Posted Tue Sep 9 11:04:10 2014

templates expose odd behavior when it comes to composing links and directives:

  • the parameters are passed through the preprocessor twice, once on per-parameter basis and once for the final result (which usually contains the preprocessed parameters).

    one of the results it that you have to write:

    [[!template  id="infobox" body="""
        Just use the \\\[[!template]] directive!
    """]]
    

    (that'd be three backslashes in front of the opening [.)

    this also means that parts which are not used by the template at all still have their side effects without showing.

    furthermore, the evaluation sequence is hard to predict. this might or might not be a problem, depending on whether someone comes up with a less contrived example (this one assumes a [[!literal value]] directive that just returns value but protects it from the preprocessor):

    we can use [[!literal """[[!invalid example]]"""]], but we can't use [[!template id=literalator value="""[[!invalid example]]"""]] with a 'literalator' template <span class="literal">[[!literal """<TMPL_VAR value>"""]]</span> because then the invalid directive comes to action in the first (per-argument) preprocessor run

  • links in templates are not stored at all; they appear, but the backlinks don't work unless the link is explicit in one of the arguments.

    [[!template  id="linker" destination="foo"]]
    

    with a 'linker' template like

    Go to [[<TMPL_VAR destination>]]!
    

    would result in a link to 'destination', but would not be registered in the scan phase and thus not show a backlink from 'foo'.

    (a [[!link to=...]] directive, as suggested in flexible relationships between pages, does get evaluated properly though.)

    this seems to be due to linkification being called before preprocess rather than as a part of it, or (if that is on purpose) by the template plugin not running linkification as an extra step (not even once).

(nb: there is a way to include the raw_ value of a directive, but that only refers to htmlification, not directive evaluation.)

both those behaviors are non-intuitive and afaict undocumented. personally, i'd swap them out for passing the parameters as-is to the template, then running the linkifier and preprocessor on the final result. that would be as if all parameters were queried raw_ -- then again, i don't see where raw_ makes anything not work that worked originally, so obviously i'm missing something.

i think it boils down to one question: are those behaviors necessary for compatibility reasons, and if yes, why?

--chrysn

Posted Fri Jul 11 13:06:48 2014

The FormattingHelp link in the edit form of any page points to the same ikiwiki/formatting help text for Markdown, regardless of page type (which could be HTML, reStructuredText, etc.) On the wiki I run, this is confusing users.

What I would like is that either the FormattingHelp link changes with page type (requires Javascript, if one is going to change the page type for new pages), or that the ikiwiki/formatting page is an index of supported page types with a further link to help text for each one (less user-friendly but likely easier to implement).

Posted Tue Jun 10 05:14:54 2014

If you have a page like

[[!if  test="enabled(smileys)" then=":-P"]]

then enabling or disabling the smileys plugin will not rebuild it.

Unfortunately, I can't think of a good way to solve this without introducing a special case for enabled() in Render.pm, either a new dependency type "enabled(smileys)" => $DEPENDS_ENABLED or a special case that treats "enabled(smileys)" => $DEPENDS_PRESENCE differently. --smcv

Posted Wed Mar 5 05:03:37 2014

Similar to syslog fails with non-ASCII wikinames, this bug happens when the wiki name has non-ascii characters in the site name. In my case, it has the "CⒶTS" string.

We get the following error in a password reset:

Error: Wide character in subroutine entry at /usr/share/perl5/Mail/Sendmail.pm line 308.

Help! :) --anarcat

I assume this means Mail::Sendmail doesn't know how to send Unicode strings, so any string passed to it (or any message body, or something?) will need to be passed through encode_utf8(). It looks as though Mail::Sendmail also defaults to

Content-Type: 'text/plain; charset="iso-8859-1"'

so it'll need a 'Content-Type' => 'text/plain; charset="utf-8"' too.

I'm disappointed to see how many of the library modules used by ikiwiki are not Unicode-clean... but then again, Mail::Sendmail was last released in 2003 so it's hardly surprising. I wonder whether Email::Sender is any better?

(If you know Python 2, the analogous situation would be "doesn't know how to send unicode objects, so you have to get a str object with a_unicode_object.encode('utf-8')".) --smcv

Shameless plug: passwordauth: sendmail interface. Though, I have no idea whether that is UTF-8-safe. --tschwinge

Posted Mon Mar 3 11:49:52 2014

For some more flexibility in creating a stylesheet for ikiwiki, it would be nice if there were a few unused elements on the page that one can move around and assign content to using CSS.

For instance, something like this:

<div class='aux' id='aux1'></div>
<div class='aux' id='aux2'></div>

etc. For bonus points, the number could be configurable. To avoid empty content, style.css should have something like this:

.aux {
    display: none;
}

This can then be used to move things around. For instance, I have on my website's CSS stylesheet the following:

#aux1 {
    position: fixed;
    width: 150px;
    height: 150px;
    bottom: 0px;
    left: 0px;
    background-image: url("wouter3.png");
    background-position: top right;
    background-repeat: no-repeat;
    background-origin: content-box;
    display-block;
}

which adds my hackergochi to the bottom left of the webpage, with some margin.

I tried looking for something like this, but I couldn't find it. Perhaps I just didn't look in the right places, though; apologies if that is the case.

This can easily be achieved by modifying templates. Simply copy the default page template to the template directory of your wiki, and modify it to add your empty divs.

-- Louis

Posted Sat Mar 1 10:43:02 2014

Hunting down what was generating

utf8 "\xEB" does not map to Unicode at /usr/share/perl5/IkiWiki.pm line 873, <$in> chunk 1.

lead me to a call to utf8::valid, which lead to http://perldoc.perl.org/utf8.html which says this is an "INTERNAL" function:

Main reason for this routine is to allow Perl's testsuite to check that operations have left strings in a consistent state. You most probably want to use utf8::is_utf8() instead.

Apparently the main point of the function is to emit the warning in unit tests - problem is, in the ikiwiki context, the only useful thing to warn about would be the name of the file you're trying to parse, not the name of the source code. Alternatively, since the code does continue on with the data, not whining about it might be an option :-) but an actionable message would be better.

Posted Sat Jan 11 18:36:45 2014

Put something like this in the setup file:

conversion:
  - from: odt
    to: pdf
    command: [unoconv, -f, pdf, -o, $OUTPUTDIR, $INPUTFILE]
  - from: ditaa
    to: png
    command: [ditaa, $INPUTFILE, $OUTPUTFILE, -s, 0.7]

However Dumper($config{conversion}) shows:

$VAR1 = [
          'HASH(0x164e1a0)',
          'HASH(0x164e3c8)'
        ];

I think it is getting mangled in sub merge in IkiWiki/Setup.pm and its calls to possibly_foolish_untaint

Workaround: force the array values to be strings, and then re-parse them using YAML::XS::Load:

conversion:
  - |
    from: [odt, odp]
    to: pdf
    command: [unoconv, -f, pdf, -o, $OUTPUTDIR, $INPUTFILE]
  - |
    from: ditaa
    to: png
    command: [ditaa, $INPUTFILE, $OUTPUTFILE, -s, 0.7]

...

sub checkconfig {
    if (!defined $config{conversion} || ref $config{conversion} ne "ARRAY") {
        error(sprintf(gettext("Must specify '%s' and it must be a list"), "conversion"));
    }
    for (my $i=0; $i < @{$config{conversion}}; $i++) {
      $config{conversion}->[$i] = YAML::XS::Load($config{conversion}->[$i]) if
          ref $config{conversion}->[$i] ne 'HASH';
    }
}

getsetup defines config options to be one of: boolean, string, integer, pagespec, "internal" (non-user-visible string), ref to an array of one of those scalar types, or ref to a hash { string => one of those scalar types }. IkiWiki::Setup also appears to support regexps (qr//), although that's not documented (presumably they're treated the same as strings).

Supporting arbitrary arrays/hashes as values would require some way to untaint the values recursively.

Complex config data also can't be used with the websetup plugin, which currently supports everything that IkiWiki::Setup does, except for hashes. --smcv

Posted Fri Nov 29 10:04:06 2013
Available in a git repository branch.
Branch: anarcat/dev/syslog_utf8
Author: anarcat

this feature made it so syslog doesn't work anymore if the site being logged has non-ASCII characters it in.

Specifically, my wiki was named "CⒶTS", and nothing was showing up in syslog. When I changed that to "C@TS", it worked again.

My guess is this sits somewhere here:

        return eval {
            Sys::Syslog::syslog($type, "[$config{wikiname}] %s", join(" ", @_));
        };

Yet I am not sure how to fix that kind of problem in Perl... --anarcat

If I remove the "eval" above, I get:

Error: Wide character in syswrite at /usr/lib/perl/5.14/Sys/Syslog.pm line 485.

I have improved a little the error handling in log_message() so that we see something when syslog fails, see the branch documented above. I can also confirm that reverting syslog should show wiki name fixes the bug. Finally, I have a unit test that reproduces the problem in git, and a working patch for the bug, again in git.

One last note: I noticed that this problem also happens elsewhere in ikiwiki. For example, the notifyemail plugin will silently fail to send notifications if the pages contain unicode. The ?notifychanges plugin I am working on (in option to send only the diff in notifyemail) seems to be working around the issue so far, but there's no telling which similar problem are out there.

I'd merge it. --smcv

I've merged it, but I don't feel it fixes this bug. --Joey

(I removed the patch tag to take it off the patches list.)

What else is needed? Systematic classification of outputs into those that do and don't cope with Unicode? --smcv

Posted Sat Sep 7 18:04:57 2013 Tags:

I just got this message trying to post to this wiki:

Error: Sorry, but that looks like spam to blogspam: No reverse DNS entry for 2001:1928:1:9::1

So yeah, it seems I have no reverse DNS for my IPv6 address, which may be quite common for emerging IPv6 deployments...

This may be related to ?blogspam options whitelist vs. IPv6?.

Posted Wed Jun 19 09:56:39 2013

Given an uploaded image via: [[!img NAME.svg alt="image"]]

Viewing the generated page shows the following error:

"[[!img Error: failed to read name.svg: Exception 420: no decode delegate for this image format `/home/user/path/name.svg' @ error/svg.c/ReadSVGImage/2815]]"

The caps in the image title were somehow converted to small letters and then the image is saved as a directory. Very puzzling.
I get the same error when image names are small letters.

The error also occurs with png images.

How do I fix this?

Later investigation ... I got around the problem by creating the mark-up in a new directory. However, if I try to create a new directory with the same name as the directory containing the problem code, the problem re-emerges -- the old directory is apparently not overwritten. Perhaps this is an issue with the git storage.

Posted Wed May 1 15:59:56 2013

I turned on the sidebar plugin, with global_sidebars on (in the web setup page), created a sidebar page in the root, and edited the sidebar a few times.

I then noticed that all pages on the root had been updated with a sidebar, but no subpages (i.e. a/b). Only after editing a subpage did it get a sidebar. Editing sidebar itself only updated subpages with sidebars, the other subpages had not been refreshed (proven by their unchanged filesystem date)

After calling ikiwiki --setup on the command line all pages were updated. So this seems to be a difference between web-started --setup and command-line --setup. Or it just doesn't work the first time --setup is called after sidebars are enabled.

Posted Fri Feb 22 13:10:52 2013

A site got stuck like this:

/home/b-fusioninventory/public_html/documentation/index.es.html independently created, not overwriting with version from documentation.es

I tried rebuilding it, and the rebuild failed like this:

building recentchanges/change_ef4b9f92821335d96732c4b2c93ed96bc84c2f0d._change, which depends on templates/page.tmpl
removing recentchanges/change_9ca1de878ea654566ce4a8a031d1ad8ed135ea1c/index.html, no longer built by recentchanges/change_9ca1de878ea654566ce4a8a031d1ad8ed135ea1c
internal error: recentchanges/change_9ca1de878ea654566ce4a8a031d1ad8ed135ea1c._change cannot be found in /home/b-fusioninventory/source or underlay

This internal error seems like the root cause of the original failure. ikiwiki crashed and did not record that it wrote the index.es.html file.

Deleting the indexdb and rebuilding cleaned up the problem.

This needs more investigation. --Joey

Posted Mon Jan 28 17:10:37 2013

The toc directive scrapes all headings from the page, including those in the sidebar. So, if the sidebar includes navigational headers, every page with a table of contents will display those navigational headers before the headers in that page's content.

I'd like some way to exclude the sidebar from the table of contents. As discussed via Jabber, perhaps toc could have a config option to ignore headers inside a nav tag or a tag with id="sidebar".

Posted Thu Jan 3 14:56:06 2013

I accidentally made a typo spelling "surprises" and changed my URL from

http://natalian.org/archives/2012/12/04/Singapore_banking_suprises/ to http://natalian.org/archives/2012/12/04/Singapore_banking_suprises/

Using the meta redir. However the meta redir now appears in the index of http://natalian.org/

Any ideas how to handle this situation?

Well, you can adjust the inline's pagespec to exclude it, or even tag it with a tag that the pagespec is adjusted to exclude. --Joey

Posted Wed Dec 5 00:26:21 2012 Tags: bugs

For some time now, in circumstances that I've had enormous troubles trying to track, I've seen feeds getting removed by ikiwiki when apparently unrelated pages got changed, with the message:

removing somepath/somepage/somefeed, no longer built by some/unrelated/page

I've finally been able to find how and why it happens. The situation is the following:

  • page A has an inline directive that (directly) generates a feed F
  • page B inlines A, thus (indirectly) generating F again
  • page B is rendered after page A

The feed removal happens when changes are made to prevent B from inlining A; for example, because B is a tag page and A is untagged B, or because B includes A through a pagespec that no longer matches A. In this case, this happens:

  • page A is built, rendering F
  • page B is built, not rendering F, which it used to render
  • F is removed because it is not built by B anymore

Note that although this issue is triggered (for me) from the changes I proposed last year to allow feed generation from nested inlines coalescing it to be page-based instead of destpage-based (bb8f76a4a04686def8cc6f21bcca80cb2cc3b2c9 and 72c8f01b36c841b0e83a2ad7ad1365b9116075c5) there is potential for it popping up in other cases.

Specifically, the logic for the removal of dependent pages currently relies on the assumption that each output has a single generator. My changes caused this assumption to be violated, hence the error, but other cases may pop up for other plugins in the future.

I have a [patch] fixing this issue (for feeds specifically, i.e. only the problem I am actually having) on top of my mystuff branch, but since that also has heaps of other unrelated stuff, you may want to just pick it from my gitweb.

The patch changes the will_render() for feeds to be based on the page rather than on the destpage, matching the fact that for nested inlines it's the inner page that is ultimately responsible for generating the feed.

I've noticed that it requires at least two full rebuilds before the index is again in a sensible state. (On the first rebuild, all feeds from nested inlines are actually removed.)

While the patch is needed because there are legitimate cases in which nested feeds are needed (for example, I have an index page that inlines index pages for subsection of my site, and I want those feed from being visible), there are other cases when one may want to skip feed generation from nested inlines.

--GiuseppeBilotta

Posted Mon Aug 27 02:44:17 2012

Say you are commenting on this report. The Navbar on top will look like

ikiwiki/ bugs/ commenting on Navbar does not link to page being commented on while commenting

while either of those two options would be better:

ikiwiki/ bugs/ commenting on Navbar does not link to page being commented on while commenting

ikiwiki/ bugs/ Navbar does not link to page being commented on while commenting / New comment

-- RichiH

Posted Sun Aug 5 19:57:05 2012

Hi folks,

This is a fairly fresh wiki. I recently noticed the Links: section the the bottom looked like this:

Links: index recentchanges/change 0b2f03d3d21a3bb21f6de75d8711c73df227e17c recentchanges/change 1c5b830b15c4f2f0cc97ecc0adfd60a1f1578918 recentchanges/change 20b20b91b90b28cdf2563eb959a733c6dfebea7a recentchanges/change 3377cedd66380ed416f59076d69f546bf12ae1e4 recentchanges/change 4c53d778870ea368931e7df2a40ea67d00130202 recentchanges/change 7a9f3c441a9ec7e189c9df322851afa21fd8b00c recentchanges/change 7dcaea1be47308ee27a18f893ff232a8370e348a recentchanges/change 963245d4e127159e12da436dea30941ec371c6be recentchanges/change cd489ff4abde8dd611f7e42596b93953b38b9e1c ...

All of those "recentchanges/ change xxxxxxx" links are clickable, but all yield 404 when clicked.

When I disable the CamelCase plugin and rebuild the wiki, all the Links other than index disappear, as they should. Re-enable CamelCase, and they're back.

This is a very simple wiki. Just fresh, only one page other than index (this one), and nothing at all fancy/weird about it.

Posted Wed Aug 1 15:03:28 2012

If I use the linkmap directive twice on a single page, I get the same image appearing in both locations, even though the parameters for the two directives may have been different.

-- Martin

Posted Tue Jul 24 12:20:36 2012

If you look at org mode, the link to the Discussion page is not there (has a question mark), as if it didn't exist. But--through the search--I discovered that the Discussion page does exist actually: Discussion.

So, there is a bug that prevents a link to the existing Discussion page from appearing in the correct way on the corresponding main page. --Ivan Z.

Perhaps, this has something to do with the same piece of code/logic (concerning case-sensitivity) as the fixed unwanted discussion links on discussion pages? --Ivan Z.

Posted Fri Jun 29 12:57:02 2012

I have heard repeated reports on http://mesh.openisp.ca/ that editing a page that has a waypoint in it will sometimes make that waypoint disappear from the main map. I have yet to understand why that happens or how, but multiple users have reported that.

A workaround is to rebuild the whole wiki, although sometimes re-editing the same page will bring the waypoint back on the map.

I have been able to reproduce this by simply creating a new node. It will not show up on the map until the wiki is rebuilt or the node is resaved. -- anarcat

Posted Sat May 12 11:04:07 2012

The listdirectives` directive doesn't register a link between the page and the subpages. This is a problem because then the orphans directive then marks the directives as orphans... Maybe it is a but with the orphans directive however... A simple workaround is to exclude those files from the orphans call... --anarcat

There's a distinction between wikilinks (matched by link(), backlink() etc.) and other constructs that produce a hyperlink. Some directives count as a wikilink (like tag) but many don't (notably inline, map, listdirectives, and orphans itself). As documented in orphans, orphans will tend to list pages that are only matched by inlines/maps, too.

The rule of thumb seems to be that a link to a particular page counts as a wikilink, but a directive that lists pages matching some pattern does not; so I think listdirectives is working as intended here. orphans itself obviously shouldn't count as a wikilink, because that would defeat the point of it :-)

Anything that uses a pagespec to generate links, like inline and map, can't generate wikilinks, because wikilinks are gathered during the scan phase, and pagespecs can't be matched until after the scan phase has finished (otherwise, it'd be non-deterministic whether all wikilinks had been seen yet, and link() in pagespecs wouldn't work predictably).

I suggest just using something like:

[[!orphans  pages="* and !blog/* and !ikiwiki/directive/*"]]

This wiki's example of listing orphans has a more elaborate pagespec, which avoids bugs, todo items etc. as well.

--smcv

No follow-up or objection for a while, so considering this to be working as designed. --smcv

Seems I'm a bit late to butt in, but would it be possible to have two further phases after the scan phase, the first running map and inline and the second orphan? Then map and inline could log or register their links (obviously somewhere were it won't change the result of the link function) and orphan could take them into account. This logging could be turned on by parameter to not waste time for users not needing this and make it tunable (i.e. so that the user can decide which map directives count and which don't)

For someone using map and especially autoindex the output of the orphans directive is simply wrong/useless (at least it is for me). And there is no easy workaround like for listdirectives -- holger

Hmm. I think this can be done without introducing any "phases", even, but it would require each plugin that generates links according to a pagespec to have either a conditional call into the orphans plugin, or a call to a new core function in ikiwiki that exists solely to support the orphans plugin. Something like this, maybe:

# in map.pm, inline.pm, pagestats.pm etc., at scan time
if (IkiWiki::Plugin::orphans->can("add_reachable")) {
    IkiWiki::Plugin::orphans::add_reachable($page, $pagespec);
}

# in orphans.pm (pseudocode; note that this does not *evaluate*
# $pagespec, only stores it, so it's OK to do this at scan time)
sub needsbuild ($pages)
    for each page in $pages
        clear $pagestate{location}{orphans}{reachable}
sub reachable ($location, $pagespec)
    add $pagespec to @{$pagestate{location}{orphans}{reachable}}

# in preprocess function in orphans.pm (pseudocode)
# executed at build time, not at scan time, so pagespecs work

for each maybe_orphan with no links to it
    for each location with a list of reachable pagespecs
        make the page with the orphans directive depend on \
            the page that is the location
        for each of those pagespecs
            if pagespec matches orphan
                take orphan off the list
                go to next orphan
output list of orphans

(Maybe parentlinks should also annotate the parent/ancestors of each page as reachable from that page.)

Do other people (mainly Joey) think that'd be acceptable, or too intrusive?

Taking this off the list of resolved bugs again while we think about it.

I suspect that in the presence of autoindex, what you really want might be less "there's a link to it" and more "there's a path to it from the root of the wiki", which is why I called the proposed function "add_reachable". On the other hand, maybe that's too computationally intensive to actually do; I haven't tried it. --smcv

(I'll interpet Joeys silence as a good sign ;-). Is there a difference between "link to it" and "path to it"? If we assume autoindex produces bonafide "first class" links there shouldn't be one!?

So far your idea sounds great, says me without any knowledge of the source. I'll try to grok it. Is there a medium for silly questions, a wiki seems not the right fit for that? -- holger

Yes, there has to be a difference between a first class wikilink and the thing to which map and inline can contribute. map and inline use a pagespec to decide what they include, and pagespecs can't be evaluated and get a correct answer until the set of links has been collected, because their results often depend on the set of links. Otherwise, suppose you had a page foo whose only contents were this:

[[!inline  pages="!backlink(foo)"]]

If inline generated links, it would inline exactly those pages that it doesn't inline. That's never going to end well :-) --smcv

We have to differentiate between what users of ikiwiki consider first class links and what internally is happening. For the user any link contributing to the structured access tree is first class. The code on the other hand has to differentiate between the static links, then generated links, then orphan links. Three "passes", even your proposed solution could be seen as adding another pass since the orphan plugin has to run after all the plugins generating (first class user) links. -- holger

I think the difference between your point of view, and what ikiwiki currently implements / what its design is geared towards, is this: ikiwiki says A links to B if the source code of A contains an explicit link to B. You say A links to B if the compiled HTML of A contains a link to B.

Would you agree with that characterization?

I suspect that "link in the source code" may be the more useful concept when using links for backlinks (I think the original implementation is http://c2.com/cgi/wiki?BackLink) and as pseudo-tags (http://c2.com/cgi/wiki?WikiCategories). The fact that this is what link() and backlink() mean could be better-documented: it's entirely possible that the author of their documentation (Joey?) thought it was obvious that that's what they mean, because they were coming from a compiler/source-code mindset.

Also, backlinks become rather all-engulfing if their presence in the compiled output counts as a link, since after a render pass, they would all become bidirectional; and as I noted previously, if pagespecs can match by linkedness (which we want) and plugins can generate lists of links according to pagespecs (which we also want), then links in the compiled output can certainly get into Russell's paradox-like situations, such as the page that links to every page to which it does not link.

For the special case of deciding what is orphaned, sure, it's the compiled HTML that is the more relevant thing; that's why I talked about "reachability" rather than "links".

--smcv

Posted Tue Apr 17 12:04:13 2012

Definition lists do not look great here...

Here is an example.

this is a term
and this is its definition.

(This wiki doesn't support Markdown's extended definition lists, but still, this is valid markup.)

I believe <dt> should be made bold. I have added this to my local.css, and I would hate to add this all the time forever:

/* definition lists look better with the term in bold */
dt
{
    font-weight: bold;
}

:) How does that look? I can provide a patch for the base wiki if you guys really want... ;) -- anarcat

What you dislike seems to be the default rendering of definition lists by browsers. I don't think it's ikiwiki's place to override browser defaults for standard markup in the document body, at least not in the default antitheme. --Joey

How about in the actiontab theme then? :)

Posted Fri Apr 6 19:57:01 2012

Observed behavior:

When I create a link like [[cmd_test]] , the link appears as 'cmd test'.

Expected behavior:

I would like to be able to create links with underscores. I realize this is a feature, and I searched for ways to escape the underscore so it would appear, but I didn't find any.

as a workaround, you can use [[cmd__95__test|cmd_test]] (which will link to a page named "cmd test" at the url location "cmd_test") or [[cmd__95__test]] (which will link to a page named "cmd_test" at the url location "cmd__95__test"). i would, from my limited understanding of ikiwiki internals, consider the bug valid, and suggest that

  • explicit link text be not subject to de-escaping (why should it; this would be the short term solution)
  • escaped page names never be used in user visible parts of ikiwiki (in my opinion, a user should not need to know about those internals, especially as they are configuration dependant (wiki_file_regexp))

note that in wikilink, that very behavior is documented; it says that "[[foo_bar|Sandbox]]" will show as "foo bar". (although you can't tell that apart from "foo_bar" easily because it's a hyperlink).

i assume that this behavior stems from times when wikilinks and directives were not distinguished by [[ vs [[! but by the use of whitespace in directives, so whitespace had to be avoided in wikilinks.

--chrysn

having hacked around in the link plugin, i can confirm that the link texts are explicitly de-escaped, and that when no pipe is inside the link (ie links like [[cmd_test]]), the string "cmd_test" is regarded as a link (that will subsequently be converted to a readable text) rather than as a readable text (for which a suitable link target is found automatically). --chrysn

Posted Fri Mar 23 19:53:07 2012

When an ikiwiki instance is holding a lock, a web user clicking on "add comment" (for example) will have to wait for the lock to be released. However, all they are then presented with is a web form. Perhaps CGI requests that are read-only (such as generating a comment form, or perhaps certain types of edits) should ignore locks? Of course, I'd understand that the submission would need to wait for a lock. — Jon

Ikiwiki has what I think of as the Big Wiki Lock (remembering the "Big Kernel Lock"). It takes the exclusive lock before loading any state, to ensure that any changes to that state are made safely.

A few CGI actions that don't need that info loaded do avoid taking the lock.

In the case of showing the comment form, the comments plugin needs CGI session information to be loaded, so it can check if the user is logged in, and so it can add XSRF prevention tokens based on the session ID. (Actually, it might be possible to rely on CGI::Session's own locking of the sessions file, and have a hook that runs with a session but before the indexdb is loaded.)

But, the comment form also needs to load the indexdb, in order to call check_canedit, which matches a pagespec, which can need to look things up in the indexdb. (Though the pagespecs that can do that are unlikely to be relevant when posting a comment.)

I've thought about trying to get rid of the Big Wiki Lock from time to time. It's difficult though; if two ikiwikis are both making changes to the stored state, it's hard to see a way to reconcile them. (There could be a daemon that all changes are fed thru using a protocol, but that's really complicated, and it'd almost be better to have a single daemon that just runs ikiwiki; a major architectural change.)

One way that almost seems it could work is to have a entry path that loads everything read-only, without a lock. And then in read-only mode, saveindex would be an error to run. However, both the commenting code and the page edit code currently have the same entry path for drawing the form as is used for handling the posted form, so they would need to be adapted to separate that into two code paths. --Joey

Posted Tue Jan 17 08:13:47 2012

This is possibly/probably due to my weird setup, which is that I have apache behind nginx, with the result that apache sees the client's IPv4 address as having been mapped to IPv6. i.e. ::ffff:10.11.12.13. That being the case, I currently need to specify that (with the ::ffff: prepended) if I want to whitelist (or more importantly blacklist) and IPv4 address.

It strikes me that this is liable to become more of a problem as people finally start using IPv6, so it might be worth ensuring that the code that compares IP addresses be able to treat the two formats (with and without the ffff's) as equivalent. --fil

Posted Mon Dec 12 19:33:57 2011

Can't appear to get 'wiki' functions (i.e. editing) running when ikiwiki is running on a port other than the default (port 80). Somewhere in the processing it considers the base URL to exclude the port number and the websever throws back an error finding the page.

For example if you run on 'http://my.gear.xxx:8080/' then after clicking login (using default password auth) it will process and try to redirect you to 'http://my.gear.xxx/cgi-bin/ikiwiki.cgi'. I'm assuming that somewhere we've used the 'path' and the 'host' and dropped the remainder. I can figure out where this is yet but I'll post back if I get lucky.

-- fergus

NB: both the 'url' and the 'cgiurl' include the port and removing the port element provides the expected functionality.


I tried to reproduce this by making my laptop's web server use port 8080. Set up ikiwiki to use that in cgiurl and url, and had no problem with either openid or password auth login.

Ikiwiki has had some changes in this area in the past year; you don't say what version you were using. It could also be a problem with your web server, conceviably, if didn't correctly communicate the port to the cgi program. --Joey


I did think of that so threw a 'printenv' script to check the port was arriving right.

SERVER_PORT=8181  
HTTP_HOST=zippy0.ie0.cobbled.net  

[ ... ]

In apache, HTTP_HOST includes the port. This is not part of the CGI spec it seems, but perl's CGI module seems to rely on it, in virtual_port:

my $vh = $self->http('x_forwarded_host') || $self->http('host');
my $protocol = $self->protocol;
if ($vh) {
   return ($vh =~ /:(\d+)$/)[0] || ($protocol eq 'https' ? 443 : 80);

The CGI module only looks at SERVER_PORT when there's no HTTP_HOST. So this is either a bug in perl's CGI or thttpd. --Joey

[ ... ]


This is interesting. If HTTP_HOST is wrong then

  1. the client header must be wrong (i.e. not including the PORT)
  2. perl's doing something bad[tm] (or at least lazy)
  3. apache is adding it
  4. thttpd is stripping it

Quick hack shows that thttpd must be stripping the port number from the Host: header. That can be fixed.

Thanks for the assist. -- fergus


Patch for thttpd-2.25b for posterity and completeness

diff --git a/libhttpd.c b/libhttpd.c
index 73689be..039b7e3 100644
--- a/libhttpd.c
+++ b/libhttpd.c
@@ -2074,9 +2074,6 @@ httpd_parse_request( httpd_conn* hc )
        cp = &buf[5];
        cp += strspn( cp, " \t" );
        hc->hdrhost = cp;
-      cp = strchr( hc->hdrhost, ':' );
-      if ( cp != (char*) 0 )
-          *cp = '\0';
        if ( strchr( hc->hdrhost, '/' ) != (char*) 0 || hc->hdrhost[0] == '.' )
            {
            httpd_send_err( hc, 400, httpd_err400title, "", httpd_err400form, "" );

-- fergus


I've gone ahead and filed a bug on CGI.pm too: https://rt.cpan.org/Ticket/Display.html?id=72678 --Joey


That'll be an interesting discussion as I'd suggest that HTTP headers are defined in the CGI specification as client headers and thus what thttpd is doing is wrong (i.e. mangling the client's own representation). Whether a CGI client should trust HTTP header over the server is probably already settled by convention.

-- fergus

Posted Wed Nov 23 15:44:53 2011

I originally set up ikiwiki by using the debian package, but had some odd issues, so i figured i'd try installing from git. To do that i uninstalled the debian package and then did the Makefile dance from the git dir. In that process the original dirs configured in templatedir underlaydir in my wiki were deleted; HOWEVER when rebuilding the script just went ahead and did not even note the lack of those dirs. It would be nice if it threw errors if the dirs were configured, but non-existant.

Hmm. This behavior was explicitly coded into ikiwiki for underlay dirs: commit. Pity I didn't say why, but presumably there are cases where one of the underlaydirs is expected to be missing, or where this robustness of not crashing is needed.

The situation with missing templatedirs is more clear: When it's looking for a given template file it just tries to open it in each directory in turn, and uses the first file found; checking that a directory exists would be extra work and there's a nice error message if a template cannot be found. --Joey

I'd agree with the thought behind that ... if it actually had thrown an error. However it did not. How about just checking the config variables when the template and/or config is set up? --Mithaldu

Posted Fri Oct 28 10:01:21 2011

I just tried to clone the git repo onto a windows machine to test things out a bit and it turns out i cannot even successfully checkout the code because of those colons. Would a patch changing those to underscores be accepted? --Mithaldu

Well, this is a difficult thing. Ikiwiki has a configuration setting to prevent it writing filenames with colons, but for backwards compatability that is not enabled by default. Also nothing would stop people from making commits that added filenames with colons even if it were disabled in ikiwiki. I don't know that trying to work around obscure limitations in OSs that I've never heard of ikiwiki being used on is worth the bother TBH, but have not really made up my mind. --Joey

I'm not trying to run it there. Ikiwiki is way too friggin' weird to try that. I just want to be able to check out the main repo so i can work in a native editor. Right now your core repository is downright hostile to cross-platform development in any way, shape or form. (Just plain splitting the docs from the code would work too.) --Mithaldu

Does(n't) cygwin handle the filename limitation/translations? If so, can you check out via git inside a cygwin environment? — Jon

That actually allows me to check things out, but the resulting repo isn't compatible with most of the rest of my system, so it's extremely painful. --Mithaldu

Posted Thu Oct 27 09:15:00 2011

I'm using the most recent release of ikiwiki (3.20110905), the Perl shipped with SuSE 11.4 (v5.12.3), and built and installed xapian 1.2.7 from source, as it seems the current stable version that's encouraged for use by xapian.

After enabling the search plugin and pointing ikiwiki to the omega program, rerunning ikiwiki --setup, and attempting a search, all searches return 0 results. No errors are reported by omindex or ikiwiki while producing the indexes in .ikiwiki/xapian/*, and the files appear to contain the indexed data. I don't think it's a problem in indexing.

When running omega by hand in the .ikiwiki/xapian directory, providing queries on the command-line, runs correctly but again provides no results.

I found that Debian stable is currently shipping 1.2.3, and on a hunch, I built that version, and searching now works fine. This looks like the usage of xapian's query template has changed somewhere between 1.2.3 and 1.2.7. Someone more familiar with xapian's query template language should be able to figure out what needs to be changed more specifically.

Debian has 1.2.7 now, and I have it installed and searching is working fine with it. --Joey

I have this same issue. I tried xapian version 1.2.5. 1.2.8, 1.2.13. I will try and see if installing 1.2.3 fixes this issue. --Ramsey

1.2.3 didn't fix the issue either --Ramsey

Posted Wed Sep 28 18:09:25 2011

When I create a new page and upload an attachment all is fine.

If I try to upload a second attachment (or remove the previously uploaded attachment), no upload happens. Instead the page gets created. No matter what I typed in, I just get a map to show the attachment. Now I can edit this page and everything is fine again.

Another workaround is to first save the text and then edit and upload the rest.

Is this a problem on my site or does anyone else see this?

(If it's my fault feel free to move this to forum.)

I don't see a behavior like that. I don't know what you mean when you say "I just get a map to show the attachment" A map?

What version of ikiwiki? What browser? Is javascript enabled? --Joey

I mean the map directive. It was ikiwiki 3.20110430. Tried Firefox and uzbl (webkit) with or without javascript.

Just updated to 3.20110905. Now the problem has changed. Instead of saving the page with the second upload and leading me to it, it leaves me in the editform but creates the page anyway. When saving I get informed, that someone else created the page. Obviously it was ikiwiki itself with the mentioned map: [[!map pages="path/to/page/* and ! ...

This told me that autoindex is the bad guy. Deactivating this plugin helps out. Don't know if this is worth fixing... I can live without that plugin. --bacuh

The right fix would probably be for do=create to allow replacing a page in the transient underlay without complaining (like the behaviour that do=edit normally has).

... which it turns out it already does. --smcv

That wouldn't help you unless autoindex defaulted to making transient pages (autoindex_commit => 0), but if we can fix removal of transient pages then maybe that default can change? --smcv

It turns out that with autoindex_commit => 0, the failure mode is different. The transient map is created when you attach the attachment. When you save the page, it's written into the srcdir, the map is deleted from the transientdir, and the ctime/mtime in the indexdb are those of the file in the srcdir, but for some reason the HTML output isn't re-generated (despite a refresh happening). --smcv

Posted Wed Sep 7 14:00:36 2011

When the aggregate plugin was used for a feed and this is removed (or the same feed name given a different rss feed), the old entries don't automatically vanish.

I think that if it was just removed, they are never GC'd, because the expiry code works on the basis of existing feeds. And if it was replaced, old items won't go away until expirecount or expireage is met.

To fix it probably needs an explicit check for items aggregated by feeds that no longer provide them, Catching old items for feeds that were changed to a different url may be harder yet. --Joey

Posted Wed Aug 24 16:26:57 2011

Wikis are great tools for collaborative content of all types, but the choice for website creators who want a level of collaboration seem to have to choose between a static website, a wiki that anyone (or all members) can edit, or an overkill customized web app.

A simple innovation that needs to propagate through wiki software is adding the ability to suggest edits and accept those edits. Perhaps you want a wiki that anyone can suggest and edit, but only registered users can edit freely or accept edits. Or you want anyone, including members, to only be able to suggest edits, and only have moderators able to approve edits and edit freely. Etc, etc.

Ikiwiki always has some work in this area; there is the moderatedcomments plugin and the checkcontent hook. The hook allows, for example a plugin to reject changes with spam links or swear words. A plugin could also use it to save the diff for later moderation.

I think the difficulty is in the moderation interface, which would need to apply the diff and show the resulting page with the changes somehow evident (for users who can't just read diffs), and would have to deal with conflicting edits, etc. --Joey

Posted Tue Aug 2 05:26:04 2011

Two examples of encoding breakage observed in the wild. In both cases the ampersand needs to be escaped. --Joey

<link href="http://www.youtube.com/watch?v=Z9hP9lhBDsI&feature=youtube_gdata"/>

<category term="vicky&alice" />
Posted Wed Jul 20 19:39:10 2011

Hi, I created [[sandbox/subpage]] then I deleted it with the "remove" button. After confirmation there was a message about a xapian error (My bad, I did not write down the exact error message). Now, accessing ?sandbox/subpage leads my browser complains about a redirect loop. JeanPrivat

Uh. Now the bug of redirect loop seems to have solved itself. However, I don't know if the xapian error need to be investigated. But I found another bug. JeanPrivat

Posted Wed Jun 15 08:56:39 2011

Apache will return 403 (Forbidden) instead of 404 (Not Found) if the Indexes option is turned off. This is because with Indexes turned on, it considers it something it might be able to serve in the future. With Indexes off, it will never serve that page in the future (unless Indexes is turned back on).

The 404 plugin code only checks for 404, not 403. It should check for both.

There are plenty of reasons a web server might 403. In most of those cases, trying to create a page where the forbidden content is is not the right thing for ikiwiki to do. --Joey

See Also:

Posted Wed Jun 8 15:27:51 2011

The table plugin seems to be unable to read a CSV file that uses \r\n for line delimiters. The same file with \r works fine. The error message is "Empty data". --liw

I was seeing this as well on an Ubuntu 11.04 system with Ubuntu 11.04, Perl 5.10.1, IkiWiki 3.20110124ubuntu1, and libtext-csv-perl 1.21-1, all installed from APT. However, when I removed libtext-csv-perl from APT and installed it from CPAN, the problem went away. FWIW, what CPAN grabbed was MAKAMAKA/Text-CSV-1.21.tar.gz. --micahrl

Posted Thu Apr 21 14:18:56 2011

When I add a comment to a page, its title should be a hyperlink. This would make re-opening it to re-read parts of it, either.

I.e. when adding a comment to this page, the last part should be a hyperlink, as well:

ikiwiki/ bugs/ creating Site title not clickable while adding a comment

Richard

Posted Sun Apr 17 20:59:28 2011

I noticed this a few times in Google Chrome 12 (dev channel) a few times, already:

I added a comment to

http://git-annex.branchable.com/forum/performance_improvement:_git_on_ssd__44___annex_on_spindle_disk/

and left the page. Later, I revisited

http://git-annex.branchable.com/forum/

and clicked on

http://git-annex.branchable.com/forum/performance_improvement:_git_on_ssd__44___annex_on_spindle_disk/

My own comment did not appear. I pressed F5 and eh presto.

My assumption is that ikiwiki does not tell Chrome to reload the page as the cache is stale.

Richard

There is some lurking bug with certian web browsers, web servers, or combination of the two that makes modifications to html files not always be noticed by web browsers. See firefox doesn't want to load updated pages at ikiwiki.info see also http://bugs.debian.org/588623.

On Branchable, we work around this problem with an apache configuration: «ExpiresByType text/html "access plus 0 seconds"»

There seems to be no way to work around it in ikiwiki's generated html, aside from using the cache-control setting that is not allowed in html5.

And, which browsers/web servers have the problem, and where the bug is, seems very hard to pin down. --Joey

Posted Sat Apr 2 17:51:54 2011

Similarly to po: apache config serves index.rss for index, the po apache config has another bug.

The use of "DirectoryIndex index", when combined with multiviews, is intended to serve up a localized version of the index.??.html file.

But, if the site's toplevel index page has a discussion page, that is "/index/discussion/index.html". Or, if the img plugin is used to scale an image on the index page, that will be "/index/foo.jpg". In either case, the "index" directory exists, and so apache happily displays that directory, rather than the site's index page!

--Joey

Ack, we do have a problem. Seems like ikiwiki's use of index/ as the directory for homepage's sub-pages and attachments makes it conflict deeply with Apache's MultiViews: as the MultiViews documentation says, index.* are considered as possible matches only if the index/ directory does not exist. Neither type maps nor mod_mime config parameters seem to allow overriding this behavior. Worse even, I guess any page called index would have the same issues, not only the wiki homepage.

I can think of two workarounds, both kinda stink:

  1. Have the homepage's targetpage be something else than index.html.
  2. Have the directory for the homepage's sub-pages and attachments be something else than index.

I doubt either of those can be implemented without ugly special casing. Any other idea? --intrigeri

As I understand it, this is how you'd do it with type maps:

  • turn off MultiViews
  • AddHandler type-map .var
  • DirectoryIndex index.var
  • make index.var a typemap (text file) pointing to index.en.html, index.fr.html, etc.

I'm not sure how well that fits into IkiWiki's structure, though; perhaps the master language could be responsible for generating the type-map on behalf of all slave languages, or something?

Another possibility would be to use filenames like index.html.en and index.html.fr, and set DirectoryIndex index.html? This could get problematic for languages whose ISO codes conventionally mean something else as extensions (Polish, .pl, is the usual example, since many sites interpret .pl as "this is a (Perl) CGI"). --smcv

There is something to be said about "index/foo" being really ugly and perhaps it would be nice to use something else. There does not appear to even be one function that could be changed; "$page/foo" is hardwired into ikiwiki in many places as a place to dump subsidiary content -- and it's not even consistent, since there is also eg, "$page.rss". I agree, approaching it from this direction would be a mess or a lot of work.

Type maps seem like a valid option, but also a lot of clutter.

index.html.pl does seem to be asking for trouble, even if apache can be configured to DTRT. It would make serving actual up perl scripts hard, at least. But that is some good out of the box thinking.. perhaps "index.foo.pl.html"?

However, that would mean that web servers need to be configured differently to serve translated and non-translated sites. The current apache configuration for po can be used with non-po sites and they still work. --Joey

I am vulnerable to the same problem because I use MultiViews, though I don't use the po module; I have to serve both Australian English and American English for my company's website (for SEO purposes; certain words that relate to our products are spelt differently in US and Australian English, and we need to be able to be googled with both spellings). I'm just fortunate that nobody has thought to add attachments to the front page yet. I raise this to point out that this is going to be a recurring problem that won't necessarily be fixed by changing the po module in isolation.

One could argue that "index" is already a special case, since it is the top page of the site. Things like parentlinks already use a special case for the top page (checking the variable HAS_PARENTLINKS). Likewise, when --usedirs is true, index is treated as a special case, since it generates "index.html" and not "index/index.html".

Unfortunately, I'm not sure what the best approach to solving this would be. --KathrynAndersen

Posted Fri Apr 1 11:49:48 2011

For security reasons, one of the sites I'm in charge of uses a Reverse Proxy to grab the content from another machine behind our firewall. Let's call the out-facing machine Alfred and the one behind the firewall Betty.

For the static pages, everything is fine. However, when trying to use the search, all the links break. This is because, when Alfred passes the search query on to Betty, the search result has a "base" tag which points to Betty, and all the links to the "found" pages are relative. So we have

<base href="Betty.example.com"/>
...
<a href="./path/to/found/page/">path/to/found/page</a>

This breaks things for anyone on Alfred, because Betty is behind a firewall and they can't get there.

What would be better is if it were possible to have a "base" which didn't reference the hostname, and for the "found" links not to be relative. Something like this:

<base href="/"/>
...
<a href="/path/to/found/page/">path/to/found/page</a>

The workaround I've come up with is this.

  1. Set the "url" in the config to ' ' (a single space). It can't be empty because too many things complain if it is.
  2. Patch the search plugin so that it saves an absolute URL rather than a relative one.

Here's a patch:

diff --git a/IkiWiki/Plugin/search.pm b/IkiWiki/Plugin/search.pm
index 3f0b7c9..26c4d46 100644
--- a/IkiWiki/Plugin/search.pm
+++ b/IkiWiki/Plugin/search.pm
@@ -113,7 +113,7 @@ sub indexhtml (@) {
        }
        $sample=~s/\n/ /g;

-       my $url=urlto($params{destpage}, "");
+       my $url=urlto($params{destpage}, undef);
        if (defined $pagestate{$params{page}}{meta}{permalink}) {
                $url=$pagestate{$params{page}}{meta}{permalink}
        }

It works for me, but it has the odd side-effect of prefixing links with a space. Fortunately that doesn't seem to break browsers. And I'm sure someone else could come up with something better and more general.

--KathrynAndersen

The <base href> is required to be genuinely absolute (HTML 4.01 §12.4). Have you tried setting url to the public-facing URL, i.e. with alfred as the hostname? That seems like the cleanest solution to me; if you're one of the few behind the firewall and you access the site via betty directly, my HTTP vs. HTTPS cleanup in recent versions should mean that you rarely get redirected to alfred, because most URLs are either relative or "local" (start with '/'). --smcv

I did try setting url to the "Alfred" machine, but that doesn't seem clean to me at all, since it forces someone to go to Alfred when they started off on Betty. Even worse, it prevents me from setting up a test environment on, say, Cassandra, because as soon as one tries to search, one goes to Alfred, then Betty, and not back to Cassandra at all. Hardcoded solutions make me nervous.

I suppose what I would like would be to not need to use a <base href> in searching at all. --KathrynAndersen

<base href> is not required to be absolute in HTML5, so when html5: 1 is used, I've changed it to be host-relative in most cases. I think that at least partially addresses this bug report, particularly if we generate HTML5 by default like I've suggested.

The <base> is there so we can avoid having to compute how to get to (the virtual directory containing) the root of the wiki from ikiwiki.cgi, which might well be somewhere odd like /cgi-bin/. I think there are probably other things that it fixes or simplifies. --smcv

Posted Sun Mar 20 23:08:39 2011

On FreeBSD, perl defaults to installation in /usr/local/bin/perl since it is not a part of the base system. If the option to create symlinks in /usr/bin is not selected, > building and running ikiwiki will fail because the shebang lines use #!/usr/bin/perl [args]. Changing this to #!/usr/bin/env -S perl [args] fixes the issue.

I think this should be a concern of ikiwiki's official FreeBSD port.

At any rate, even if it is decided that ikiwiki should be fixed, then it is probably better to use $installbin/perl from -MConfig and not the env hack.

Posted Sun Feb 27 17:38:52 2011

The inline and comments plugins both generate feed links.

In both cases, the generated markup include an element with id="feedlink".

XHTML 1.0 Strict (Ikiwiki's default output type) forbids multiple elements with the same ID:

In XML, fragment identifiers are of type ID, and there can only be a single attribute of type ID per element. Therefore, in XHTML 1.0 the id attribute is defined to be of type ID. In order to ensure that XHTML 1.0 documents are well-structured XML documents, XHTML 1.0 documents MUST use the id attribute when defining fragment identifiers on the elements listed above. See the HTML Compatibility Guidelines for information on ensuring such anchors are backward compatible when serving XHTML documents as media type text/html.

As does W3C's HTML5.

Any page with both a comments feed and an inline feed will be invalid XHTML 1.0 Strict or HTML 5.

-- Jon

?version 3.2011012 suggests this is fixed for inline, at least, I will test to see if it is cleared up for comments too. -- Jon

Posted Tue Jan 25 16:53:34 2011

At least my setup on kapsi.fi always prints 404 Not Found after adding a page with non-ascii characters in name. But the page exists and is visible after the 404 with url encoding and the blog page is inlined correctly on the feed page.

Apparently ikiwiki.info does not complain with 404. Should the character encoding be set in wiki config?

Happens also after editing the page. Here's an example:

  • page name displayed in 404: http://mcfrisk.kapsi.fi/skiing/posts/Iso-Sy%F6te%20Freeride%202011%20Teaser.html?updated
  • page name in the blog feed: http://mcfrisk.kapsi.fi/skiing/posts/Iso-Sy%C3%B6te%20Freeride%202011%20Teaser.html

Difference is in the word Iso-Syöte. Pehaps also the browsers is part of the game, I use Iceweasel from Debian unstable with default settings.

I remember seeing this problem twice before, and both times it was caused by a bug in the web server configuration. I think at least one case it was due to an apache rewrite rule that did a redirect and mangled the correct encoding.

I recommend you check there. If you cannot find the problem with your web server, I recommend you get a http protocol dump while saving the page, and post it here for analysis. You could use tcpdump, or one of the browser plugins that allows examining the http protocol. --Joey

Server runs Debian 5.0.8 but I don't have access to the Apache configs. Here's the tcp stream from wireshark without cookie data, page name is testiä.html. I guess page name is in utf-8 but in redirect after post it is given to browser with 8859-1.

POST /ikiwiki.cgi HTTP/1.1
Host: mcfrisk.kapsi.fi
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20110107 Iceweasel/3.5.16 (like Firefox/3.5.16)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://mcfrisk.kapsi.fi/ikiwiki.cgi
Cookie: XXXX
Content-Type: multipart/form-data; boundary=---------------------------138059850619952014921977844406
Content-Length: 1456

-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="_submitted"

2
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="do"

edit
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="sid"

93c956725705aa0bbdff98e57efb28f4
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="from"


-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="rcsinfo"

5419fbf402e685643ca965d577dff3dafdd0fde9
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="page"

testi..
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="type"

mdwn
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="editcontent"

test
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="editmessage"


-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="_submit"

Save Page
-----------------------------138059850619952014921977844406
Content-Disposition: form-data; name="attachment"; filename=""
Content-Type: application/octet-stream


-----------------------------138059850619952014921977844406--
HTTP/1.1 302 Found
Date: Wed, 02 Feb 2011 19:45:49 GMT
Server: Apache/2.2
Location: /testi%E4.html?updated
Content-Length: 0
Keep-Alive: timeout=5, max=500
Connection: Keep-Alive
Content-Type: text/plain

GET /testi%E4.html?updated HTTP/1.1
Host: mcfrisk.kapsi.fi
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20110107 Iceweasel/3.5.16 (like Firefox/3.5.16)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://mcfrisk.kapsi.fi/ikiwiki.cgi
Cookie: XXXX

HTTP/1.1 404 Not Found
Date: Wed, 02 Feb 2011 19:45:55 GMT
Server: Apache/2.2
Content-Length: 279
Keep-Alive: timeout=5, max=499
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /testi..html was not found on this server.</p>
<hr>
<address>Apache/2.2 Server at mcfrisk.kapsi.fi Port 80</address>
</body></html>

Getting the pages has worked every time:

GET /testi%C3%A4.html HTTP/1.1
Host: mcfrisk.kapsi.fi
User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20110107 Iceweasel/3.5.16 (like Firefox/3.5.16)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: XXXX
If-Modified-Since: Wed, 02 Feb 2011 19:45:54 GMT
If-None-Match: "1b518d-7c0-49b51e5a55c5f"
Cache-Control: max-age=0

HTTP/1.1 304 Not Modified
Date: Wed, 02 Feb 2011 20:01:43 GMT
Server: Apache/2.2
Connection: Keep-Alive
Keep-Alive: timeout=5, max=500
ETag: "1b518d-7c0-49b51e5a55c5f"
Posted Thu Jan 20 15:39:24 2011

It looks like there is no way to logout of ikiwiki at present, meaning that if you edit the ikiwiki in, say, a cybercafe, the cookie remains... is there some other security mechanism in place that can check for authorization, or should I hack in a logout routine into ikiwiki.cgi?

Click on "Preferences". There is a logout button there. --liw

It would be nice if it were not buried there, but putting it on the action bar statically would be confusing. The best approach might be to use javascript. --Joey

I agree that javascript seems to be a solution, but my brain falls off the end of the world while looking at ways to manipulate the DOM. (I'd argue also in favor of the openid_provider cookie expiring in less time than it does now, and being session based)

(The openid_provider cookie is purely a convenience cookie to auto-select the user's openid provider the next time they log in. As such, it cannot be a session cookie. It does not provide any personally-identifying information so it should not really matter when it expires.) --Joey

It would be nice to move navigational elements to the upper right corner of the page...

I have two kinds of pages (wiki and blog), and three classes of users

anonymous users - display things like login, help, and recentchanges,

non-admin users - on a per subdir basis (blog and !blog) display logout, help, recentchanges, edit, comment

admin users - logout, help, recentchanges, edit, comment, etc

I was referred to this page from posting to the forum. I am also interested in being able to use user class and status to modify the page. I will try to put together a plugin. From what I can see there needs to be a few items in it.

  • It should expose a link to a dedicated login page that, once logged in, returns the user to the calling page, or at least the home page. I have started a plugin to do this: justlogin

  • it needs to expose a link to a little json explaining the type of user and login status.

  • it should expose a link that logs the person out and returns to the calling page, or at least the home page.

Then there would need to be a little javascript to use these links appropriately. I have little javascript experience but I know that can be done. I am less sure if it is possible to add this functionality to a plugin so I'll start with that. If no one objects I will continue to post here if I make progress. If anyone has any suggestions on how to modify my approach to code it in an easier way I'd appreciate the input. justint

Posted Thu Oct 28 10:38:32 2010

I'd like the more plugin and RSS to play better together. In the case of the html generation of the main page of a blog, I'd like to get the first paragraph out, but keep RSS as a full feed.

Maybe there is a different plugin (I also tried toggle)?

I am not a fan of the more directive (thus the rant about it sucking embedded in its example). But I don't think that weakening it to not work in rss feeds is a good idea, if someone wants to force users to go somewhere to view their full content, they should be able to do it, even though it does suck.

The toggle directive will degrade fairly well in an rss feed to display the full text. (There is an annoying toggle link that does nothing when embedded in an rss feed). --Joey

I also note, that at least currently, more seems to break on a few pages, not being parsed at all when aggregated into the front page.

It's just a simple directive, it should work anywhere any directive will, and does as far as I can see. Details? --Joey

see also: rss feeds do not use recommended encoding of entities for some fields

Posted Thu Oct 28 10:28:04 2010

Wide characters should probably be supported, or, at the very least, warned about.

Test case:

mkdir -p ikiwiki-utf-test/raw ikiwiki-utf-test/rendered
for page in txt mdwn; do
  echo hello > ikiwiki-utf-test/raw/$page.$page
  for text in 8 16 16BE 16LE 32 32BE 32LE; do
    iconv -t UTF$text ikiwiki-utf-test/raw/$page.$page > ikiwiki-utf-test/raw/$page-utf$text.$page;
  done
done
ikiwiki --verbose --plugin txt --plugin mdwn ikiwiki-utf-test/raw/ ikiwiki-utf-test/rendered/
www-browser ikiwiki-utf-test/rendered/ || x-www-browser ikiwiki-utf-test/rendered/
# rm -r ikiwiki-utf-test/ # some browsers rather stupidly daemonize themselves, so this operation can't easily be safely automated

BOMless LE and BE input is probably a lost cause.

Optimally, UTF-16 (which is ubiquitous in the Windows world) and UTF-32 should be fully supported, probably by converting to mostly-UTF-8 and using &#xXXXX; or &#DDDDD; XML escapes where necessary.

Suboptimally, UTF-16 and UTF-32 should be converted to UTF-8 where cleanly possible and a warning printed where impossible.


Reading the wikipedia pages about UTF-8 and UTF-16, all valid Unicode characters are representable in UTF-8, UTF-16 and UTF-32, and the only errors possible with UTF-16/32 -> UTF-8 translation are when there are encoding errors in the original document.

Of course, it's entirely possible that not all browsers support utf-8 correctly, and we might need to support the option of encoding into CESU-8 instead, which has the side-effect of allowing the transcription of UTF-16 or UTF-32 encoding errors into the output byte-stream, rather than pedantically removing those bytes.

An interesting question would be how to determine the character set of an arbitrary new file added to the repository, unless the repository itself handles character-encoding, in which case, we can just ask the repository to hand us a UTF-8 encoded version of the file.

-- Martin Rudat

Posted Mon Oct 4 21:09:42 2010

Consider this:

$ wget http://nic-nac-project.de/~schwinge/ikiwiki/cutpaste_filter.tar.bz2
$ wget http://nic-nac-project.de/~schwinge/ikiwiki/0001-cutpaste.pm-missing-filter-call.patch

$ tar -xj < cutpaste_filter.tar.bz2
$ cd cutpaste_filter/
$ ./render_locally
$ find "$PWD".rendered/ -type f -print0 | xargs -0 grep -H -E 'FOO|BAR'
[notice one FOO in there]
$ rm -rf .ikiwiki "$PWD".rendered

$ cp /usr/share/perl5/IkiWiki/Plugin/cutpaste.pm .library/IkiWiki/Plugin/
$ patch -p0 < ../cutpaste_filter.patch
$ ./render_locally
$ find "$PWD".rendered/ -type f -print0 | xargs -0 grep -H -E 'FOO|BAR'
[correct; notice no more FOO]

I guess this needs a general audit -- there are other places where preprocess is being doing without filtering first, for example in the same file, copy function.

--tschwinge

So, in English, page text inside a cut directive will not be filtered. Because the cut directive takes the text during the scan pass, before filtering happens.

Commit 192ce7a238af9021b0fd6dd571f22409af81ebaf and po vs templates has to do with this. There I decided that filter hooks should only act on the complete text of a page.

I also suggested that anything that wants to reliably s/FOO/BAR/ should probably use a sanitize hook, not a filter hook. I think that would make sense in this example.

I don't see any way to make cut text be filtered while satisfying these constraints, without removing cutpaste's ability to have forward pastes of text cut laster in the page. (That does seems like an increasingly bad idea..) --Joey

OK -- so the FOO/BAR thing was only a very stripped-down example, of course, and the real thing is being observed with the getfield plugin. This one needs to run before preprocessing, for its {{$page#field}} syntax is (a) meant to be usable inside ikiwiki directives, and (b) the field values are meant to still be preprocessed before being embedded. That's why it's using the filter hook instead of sanitize.

Would adding another kind of hook be a way to fix this? My idea is that cut (and others) would then take their data not during scanning, but after filtering.

--tschwinge

Posted Fri Sep 17 07:48:29 2010

If comments_allowdirectives is set, previewing a comment can run directives that create files. (Eg, img.) Unlike editpage, it does not keep track of those files and expire them. So the files will linger in destdir forever.

Probably when the user then tries to save the comment, ikiwiki will refuse to overwrite the unknown file, and will crash. --Joey

Posted Wed Jun 9 18:40:18 2010

I'd like a way to always ask the RCS (Git) to update a file's mtime in refresh mode. This is currently only done on the first build, and later for --gettime --rebuild. But always rebuilding is too heavy-weight for this use-case. My options are to either manually set the mtime before refreshing, or to have ikiwiki do it at command. I used to do the former, but would now like the latter, as ikiwiki now generally does this timestamp handling.

From a quick look, the code in IkiWiki/Render.pm:find_new_files is relevant: if (! $pagemtime{$page}) { [...].

How would you like to tackle this?

--tschwinge

This could be done via a needsbuild hook. The hook is passed the list of changed files, and it should be safe to call rcs_getmtime and update the pagemtime for each.

That lets the feature be done by a plugin, which seems good, since rcs_getmtime varies between very slow and not very fast, depending on VCS.

AFAICS, the only use case for doing this is if you commit changes and then delay pushing them to a DVCS repo. Since then the file mtime will be when the change was pushed, not when it was committed. But I've generally felt that recording when a change was published to the repo of a wiki as its mtime is good enough. --Joey

Posted Tue Jun 8 05:23:05 2010

I'm attempting a merge with the SVN plugin via the web interface with ikiwiki-3.20100403 and subversion 1.6.11.

The web interface says

Your changes conflict with other changes made to the page.

Conflict markers have been inserted into the page content. Reconcile the conflict and commit again to save your changes.

However there are no merge conflict markers in the page. My apache error log says:

[Fri Apr 30 16:43:57 2010] [error] [client 10.64.64.42] svn: Commit failed (details follow):, referer: https://unixwiki.ncl.ac.uk/ikiwiki.cgi
[Fri Apr 30 16:43:57 2010] [error] [client 10.64.64.42] svn: Authorization failed, referer: https://unixwiki.ncl.ac.uk/ikiwiki.cgi

-- Jon

Only way for this to be improved would be for the svn plugin to explicitly check the file for conflict markers. I guess it could change the error message then, but the actual behavior of putting the changed file back in the editor so the user can recommit is about right as far as error recovery goes. --Joey

Posted Fri Apr 30 14:10:17 2010

Lighttpd apparently sets REDIRECT_STATUS=200 for the server.error-handler-404 page. This breaks the 404 plugin which checks this variable for 404 before processing the URI. It also doesn't seem to set REDIRECT_URL.

For what it's worth, the first half is http://redmine.lighttpd.net/issues/1828. One workaround would be to make this script your 404 handler:

#!/bin/sh
REDIRECT_STATUS=404; export REDIRECT_STATUS
REDIRECT_URL="$SERVER_NAME$REQUEST_URI"; export REDIRECT_URL
exec /path/to/your/ikiwiki.cgi "$@"

--smcv

I was able to fix my server to check the REQUEST_URI for ikiwiki.cgi and to continue processing if it was not found, passing $ENV{SEVER_NAME} . $ENV{REQUEST_URI} as the first parameter to cgi_page_from_404. However, my perl is terrible and I just made it work rather than figuring out exactly what to do to get it to work on both lighttpd and apache.

This is with lighttpd 1.4.19 on Debian.

/cgi-bin/ikiwiki.cgi?do=goto also provides redirection in the same way, if that's any help? You might need to set the lighttpd 404 handler to that, then compose REDIRECT_URL from other variables if necessary.

I originally wrote the plugin for Apache; weakish contributed the lighttpd docs and might know more about how to make it work there. --smcv

As I said, I got it working for me, but somebody who knows perl should probably look at it with the aim of making it work for everyone. I considered having lighttpd construct a proper url for the 404 redirect itself, but I don't know if it can do something like that or not. For what it's worth, here's the change I made to the module:

sub cgi ($) {
        my $cgi=shift;
        if ($ENV{REQUEST_URI} !~ /ikiwiki\.cgi/) {
                my $page = cgi_page_from_404(
                        Encode::decode_utf8($ENV{SERVER_NAME} . $ENV{REQUEST_URI}),
                        $config{url}, $config{usedirs});
                IkiWiki::Plugin::goto::cgi_goto($cgi, $page);
        }

#       if (exists $ENV{REDIRECT_STATUS} &&
#           $ENV{REDIRECT_STATUS} eq '404') {
#               my $page = cgi_page_from_404(
#                       Encode::decode_utf8($ENV{REDIRECT_URL}),
#                       $config{url}, $config{usedirs});
#               IkiWiki::Plugin::goto::cgi_goto($cgi, $page);
#       }
}
Posted Tue Mar 2 11:47:23 2010

It seems that rebuild a wiki (ikiwiki --rebuild) after changing the underlaydir config option doesn't remove the pages coming from the previous underlaydir.

I've noticed this with the debian package version 3.20100102.3~bpo50+1.

Perhaps it is possible to improve this or mention it in the manual page?

--prosper

--rebuild causes ikiwiki to throw away all its info about what it built before, so it will never clean up pages that have been removed, by any means. Suggest you do a --refresh, possibly followed by a --rebuild if that is really necessary. --Joey

Posted Fri Feb 19 06:23:26 2010

I have a page with the name "umläute". When I try to remove it, ikiwiki says:

Error: ?umläute does not exist

I'm curious about the '?' in the "?umläute" message. Suggests that the filename starts with another strange character. Can I get a copy of a git repository or tarball containing this file? --Joey

I wrote the following patch, which seems to work on my machine. I'm running on FreeBSD 6.3-RELEASE with ikiwiki-3.20100102.3 and perl-5.8.9_3.

--- remove.pm.orig      2009-12-14 23:26:20.000000000 +0100
+++ remove.pm   2010-01-18 17:49:39.000000000 +0100
@@ -193,6 +193,7 @@
                        # and that the user is allowed to edit(/remove) it.
                        my @files;
                        foreach my $page (@pages) {
+                               $page = Encode::decode_utf8($page);
                                check_canremove($page, $q, $session);

                                # This untaint is safe because of the

The problem with this patch is that, in a recent fix to the same plugin, I made @pages come from $form->field("page"), and that, in turn is already run through decode_form_utf8 just above the code you patched. So I need to understand why that is apparently not working for you. (It works fine for me, even when deleting a file named "umläute" --Joey


Update, having looked at the file in the src of the wiki that is causing trouble for remove, it is: uml\303\203\302\244ute.mdwn
And that is not utf-8 encoded, which, represented the same would be: uml\303\244ute.mdwn

I think it's doubly-utf-8 encoded, which perhaps explains why the above patch works around the problem (since the page name gets doubly-decoded with it). The patch doesn't fix related problems when using remove, etc.

Apparently, on apoca's system, perl encodes filenames differently depending on locale settings. On mine, it does not. Ie, this perl program always creates a file named uml\303\244ute, no matter whether I run it with LANG="" or LANG="en_US.UTF-8":

perl -e 'use IkiWiki; writefile("umläute", "./", "baz")'

Remains to be seen if this is due to the older version of perl used there, or perhaps FreeBSD itself. --Joey

Update: Perl 5.10 fixed the problem. --Joey

Posted Mon Jan 18 13:34:19 2010

To reproduce:

  1. Add the backlinkbug plugin below to ikiwiki.
  2. Create a page named test.mdwn somewhere in the wiki.
  3. Refresh ikiwiki in verbose mode. Pages whose bestlink is the test.mwdn page will be printed to the terminal.
  4. Delete test.mdwn.
  5. Refresh ikiwiki in verbose mode again. The same pages will be printed to the terminal again.
  6. Refresh ikiwiki in verbose mode another time. Now no pages will be printed.

bestlink() checks %links (and %pagecase) to confirm the existance of the page. However, find_del_files() does not remove the deleted page from %links (and %pagecase).

Since find_del_files removes the deleted page from %pagesources and %destsources, won't it make sense for bestlink() to check %pagesources first? --harishcm

This same problem turned out to also be the root of half of ikiwiki's second-oldest bug, bestlink change update issue.

Fixing it is really a bit involved, see commit f1ddf4bd98821a597d8fa1532092f09d3d9b5483. The fix I committed fixes bestlink to not return deleted pages, but only after the needsbuild and scan hooks are called. So I was able to fix it for every case except the one you gave! Sorry for that. To fix it during beedsbuild and scan, a much more involved approach would be needed. AFAICS, no existing plugin in ikiwiki uses bestlink in needsbuild or scan though.

If the other half of bestlink change update issue is fixed, maybe by keeping a copy of the old backlinks info, then that fix could be applied here too. --Joey

Cool that was fast! Well at least half the bug is solved :) For now I'll probably try using a workaround if using bestlink within the needsbuild or scan hooks. Maybe by testing if pagemtime equals zero. --harishcm

Yeah, and bestlink could also do that. However, it feels nasty to have it need to look at pagemtime. --Joey


#!/usr/bin/perl
# Plugin to reproduce bestlink returning deleted pages. 
# Run with ikiwiki in verbose mode.

package IkiWiki::Plugin::bestlinkbug;

use warnings;
use strict;
use IkiWiki 3.00;

sub import {
    hook(type => "getsetup", id => "bestlinkbug", call => \&getsetup);
    hook(type => "needsbuild", id => "bestlinkbug", call => \&needsbuild);
}

sub getsetup () {
    return
        plugin => {
            safe => 1,
            rebuild => 0,
        },
}

sub needsbuild (@) {
    my $needsbuild=shift;

    foreach my $page (keys %pagestate) {
        my $testpage=bestlink($page, "test") || next;

        debug("$page");
    }
}   

1
Posted Wed Nov 25 10:15:40 2009

The map directive sort by pagename. That looks kind of odd, when used together with show=title. I would expect it to sort by title then.

This would be quite hard to fix. Map sorts the pages it displays by page name, which has the happy effect of making "foo/bar" come after "foo"; which it has to do, so that it can be displayed as a child of the page it's located in. If sorting by title, that wouldn't hold. So, map would have to be effectively totally rewritten, to build up each group of child pages, and then re-sort those. --Joey

Ok, you are right, that does would break the tree. This made me think that I do not need to generate a tree for my particular use case just a list, so i thought i could use inline instead. This created two new issues:

  1. inline also does sort by pagename even when explicitly told to sort by title.

  2. I cannot get inline to create a list when the htmltidy plugin is switched on. I have a template which is enclosed in an li tag, and i put the ul tag around the inline manually, but htmltidy breaks this. --martin

You might want to check if the report plugin solves your problem. It can sort by title, among other things. --KathrynAndersen

See also: sort parameter for map plugin and directive --smcv

Posted Sat Nov 14 22:02:34 2009

[[!inline ]] is rendered with a space in front of the first closing bracket. --tschwinge

I don't think that complicating the directive parser is warrented by the minorness of this bug. The result that it outputs is still valid. --Joey

Posted Fri Oct 16 03:14:03 2009

I'm trying to make a pretty theme for ikiwiki and I'm making progress (or at least I think I am :-). However I've noticed an issue when it comes to theming. On the front page the wiki name is put inside the "title" span and on all the other pages, it's put in the "parentlinks" span. See here:

From my dev home page:

<div class="header">
<span>
<span class="parentlinks">

</span>
<span class="title">
adam.shand.net/iki-dev
</span>
</span><!--.header-->

</div>

From a sub-page of my dev home page:

<div class="header">
<span>
<span class="parentlinks">

<a href="../">adam.shand.net/iki-dev/

</span>
<span class="title">
recipes </span>
</span><!--.header-->

</div>

I understand the logic behind doing this (on the front page it is the title as well as the name of the wiki) however if you want to do something different with the title of a page vs. the name of the wiki it makes things pretty tricky.

I'll just modify the templates for my own site but I thought I'd report it as a bug in the hopes that it will be useful to others.

Cheers,
AdamShand.


I just noticed that it's also different on the comments, preferences and edit pages. I'll come up with a diff and see what you guys think. -- Adam.

Posted Wed Aug 5 06:02:00 2009

Example:

[[`\[[!taglink TAG\]\]`|plugins/tag]]

gives:

[[\[[!taglink TAG\]\]|plugins/tag]]

Expected: there is a wikilink with the complex text as the displayed text. --Ivan Z.

Posted Tue May 26 11:57:12 2009

aggregate takes a name parameter that specifies a global name for a feed. This causes some problems:

  • If a site has multiple pages that aggregate, and they use the same name, one will win and get the global name, the other will claim it's working, but it's really showing what the other aggregated.
  • If an aggregate directive is moved from page A to page B, and the wiki refreshed, aggregate does not realize the feed moved, and so it will keep aggregated pages under A/feed_name/*. To work around this bug, you have to delete A, refresh (maybe with --aggregate?), and then add B.

Need to find a way to not make the name be global. Perhaps it needs to include the name of the page that contains the directive?

Posted Fri May 22 15:24:42 2009

Escaping pipe-symbol in ?taglink targets doesn't work as I wanted:

?smth with a pipe|about the </span> ?smth with a pipe|about the

as opposed to simple wikilinks:

?a link to smth with a pipe ?a link to smth with a pipe

And it seems to work in pagespecs:

tagged:

link:

Posted Sun May 17 13:27:05 2009 Tags: ?-symbol bugs

The remove plugin does not report an error if git rm fails. (It probably doesn't if other VCS backends fail too). This can happen for example if a page in your source directory is not a tracked file for whatever reason (in my case, due to renaming the files and forgetting to commit that change).

-- Jon

Posted Thu May 14 09:56:27 2009

I was just hovering over the '...' next to the backlinks on a page on http://ikiwiki.info/. In terms of the size of my browser window, this was towards the bottom-right of the screen.

When I hovered over the '...', the additional backlinks float appeared. This caused the page length to grow down, meaning a horizontal scrollbar was added to the page. This meant the text reflowed, and the '...' moved outside of my mouse pointer region.

This caused an infinite loop of box appears... text moves, box disappears... box re-appears.. which was not very visually pleasant.

In general I think that the onhover float is a bit of bad UI. Even a truncated list of backlinks looks cluttered due to there being no delimiters. I moved to having an always-complete list of backlinks and having them as LI elements inside a UL to make it look neater, although I appreciate that would make some pages very long indeed.

How about doing something a little like toggle for the excess items instead?

-- Jon


An additional, related issue: if the box expands beyond the bottom of the page, you might move your mouse pointer to the scrollbar in order to move further down the list, but of course then you are outside the hover region.

-- Jon

I agree, browser handling of this CSS is often not good.

A toggle would be the perfect UI, but the heaviness of needing to include 30 lines of javascript to do it, plus then it only working with javascript enabled, is also not optimal.

Another idea would be to make the "..." a link to the ikiwiki cgi. The cgi could then have a mode that displays all the backlinks of a page in a list.

Yet another idea: Find some more refined CSS for handling a variable size popup.. --Joey

Posted Fri Apr 24 09:40:07 2009

Error received when clicking on the "edit" link:

Error: [CGI::FormBuilder::AUTOLOAD] Fatal: Attempt to address non-existent field 'text' by name at /home/tealart/bin/share/perl/5.8.4/IkiWiki/CGI.pm line 112

Error received when following a "Create New Page" (eg. ?) link:

Error: [CGI::FormBuilder::AUTOLOAD] Fatal: Attempt to address non-existent field 'param' by name at /home/tealart/bin/share/perl/5.8.4/IkiWiki/Plugin/editpage.pm line 122

I could probably find several other flavors of this error if I went looking, but I trust you get the idea.

The CGI starts to render (this isn't the "you forgot to set the permissions/turn on the CGI" error) and then fails.

Further details:

  • Running on shared hosting (dreamhost; but everything compiles, dependencies installed, the site generates perfectly, other CGIs work, the file permissions work).

  • It's running perl 5.8.4, but I did upgrade gettext to 0.17

  • the server is running gcc v3.3.5 (at this point, this is the main difference between the working system and my box.)

  • I've removed the locale declarations from both the config file and the environment variable.

  • I've also modified the page template and have my templates in a non standard location. The wiki compiles fine, with the template, but might this be an issue? The CGI script doesn't (seem) to load under the new template, but I'm not sure how to address this issue.

  • All of the required/suggested module dependencies are installed (finally) to the latest version including (relevantly) CGI::FormBuilder 3.0501.

  • I'm running ikiwiki v3.08. Did I mention that it works perfectly in nearly every other way that I've managed to test thusfar?


I suspect that your perl is too old and is incompatible with the version of CGI::FormBuilder you have installed.

Is so, it seems likely that the same error message can be reproduced by running a simple command like this at the command line:

perl -e 'use warnings; use strict; use CGI::FormBuilder; my $form=CGI::FormBuilder->new; $form->text("boo")'

--Joey

nope, that command produces no output. :/

I considered downgrading CGI::FormBuilder but I saw evidence of previous versions being incompatible with ikiwiki so I decided against that.

-- tychoish

Posted Mon Mar 30 17:25:43 2009

In ikiwiki 2.66, SVG images are not recognized as images. In ikiwiki.pm, the hardcoded list of image file extensions does not include ".svg", which it probably should unless there's some other issue about rendering SVGs?

The 'img' plugin also seems to not support SVGs.

SVG images can only be included via an <object>, <embed>, or <iframe> tag. Or, perhaps as inline SVG. The htmlscrubber strips all three tags since they can easily be used maliciously. If doing inline SVG, I'd worry that the svg file could be malformed and mess up the html, or even inject javascript. So, the only options seem to be only supporting svgs on wikis that do not sanitize their html, or assuming that svgs are trusted content and embedding them inline. None of which seem particularly palatable.

I suppose the other option would be converting the svg file to a static image (png). The img plugin could probably do that fairly simply. --Joey

This seems to have improved since; at least chromium can display svg images from <img> tags. Firefox 3.5.19 did not in my testing.

So, svgs can now be included on pages by linking to them, or by using the img directive. The most portable thing is to use the img directive plus some size, which forces them to be resized and a png to actually be displayed.

I have not yet tried to do anything with sanitizing them. --Joey

I'm working on inline SVG and MathML support in ikiwiki and I've modified my htmlscrubber to sanitize SVG and MathML using the whitelists from html5lib. Here's a patch. I've also made some notes about this here: svg.

I suspect that this bug may have caught the eye of anyone interested in this sort of thing. I'll elaborate a bit on my user page to avoid getting off-topic here. --JasonBlevins, October 21, 2008

Posted Mon Oct 20 21:34:02 2008

A lot of strings in ikiwiki are hardcoded and not taken for locales resources through gettext. This is bad because ikiwiki is thus difficult to spread for non-english users.

I mean that, for instance in CGI.pm, line like:

my @buttons=("Save Page", "Preview", "Cancel");

should be written as

my @buttons=(gettext("Save Page"), gettext("Preview"), gettext("Cancel"));

Yes, these need to be fixed. But note that the localised texts come back into ikiwiki and are used in various places, including plugins. Including, possibly, third-party plugins. So localising the buttons would seem to require converting from the translations back into the C locale when the form is posted. --Joey

Wouldn't it be more easy to change all calls to the corrects ones (including in plugins) ? For instance in the same file (CGI.pm): elsif ($form->submitted eq gettext("Save Page")) {. That way no conversion to the C locale is needed. gettext use should just be publicized in documentation (at least in write). --bbb

It would be easy, but it could break third-party plugins that hardcode the english strings. It's also probably less efficient to run gettext over and over. --Joey

In standards templates things seems wrongly written too. For instance in page.tmpl line like:

<li><a href="<TMPL_VAR EDITURL>" rel="nofollow">Edit</a></li>

should be written as

<li><a href="<TMPL_VAR EDITURL>" rel="nofollow"><TMPL_VAR EDITURL_TEXT</a></li>

with EDITURL_TEXT variable initialized in Render.pm through a gettext call.

Am I wrong ?

No, that's not a sane way to localise the templates. The templates can be translated by making a copy and modifying it, or by using a tool to generate .mo files from the templates, and generate translated templates from .po files. (See l10n for one attempt.) But pushing the localisation of random strings in the templates through the ikiwiki program defeats the purpose of having templates at all. --Joey

If not I can spend some time preparing patches for such corrections if it can help.

-- bbb

Posted Thu Oct 2 17:42:33 2008

A ?PageSpec that is entirely negated terminals, such as "!foo and !bar" matches all other pages, including all internal pages. This can lead to unexpected results, since it will match a bunch of recentchanges pages, etc.

Recall that internal-use pages are not matched by a glob. So "*" doesn't match them. So if the pagespec is "* and !foo and !bar", it won't match them. This is the much more common style.

There's an odd inconsistency with entirely negated pagespecs. If "!foo" matches page bar, shouldn't "" also match bar? But, the empty pagespec is actually special-cased to not match anything.

Indeed, it seems what would be best would be for "!foo" to not match any pages, unless it's combined with a terminal that positively matches pages ("* and !foo"). Although this would be a behavior change, with transition issues.

Another approach would be to try to detect the case of an entirely negated pagespec, and implicitly add "and !internal()" to it.

Either approach would require fully parsing the pagespec. And consider cases like "!(foo and !bar)". Doesn't seem at all easy to solve. --Joey

It occurs to me that at least one place in ikiwiki optimizes by assuming that pagespecs not mentioning the word "internal" never match internal pages. I wonder whether this bug could be solved by making that part of the definition of a pagespec, rather than a risky optimization like it is now? That seems strange, though - having this special case would make pagespecs significantly harder to understand. --smcv

Posted Tue Sep 30 16:35:44 2008

The Atom and RSS templates use ESCAPE=HTML in the title elements. However, HTML-escaped characters aren't valid according to http://feedvalidator.org/.

Removing ESCAPE=HTML works fine, but I haven't checked to see if there are any characters it won't work for.

For Atom, at least, I believe adding type="xhtml" to the title element will work. I don't think there's an equivalent for RSS.

Removing the ESCAPE=HTML will not work, feed validator hates that just as much. It wants rss feeds to use a specific style of escaping that happens to work in some large percentage of all rss consumers. (Most of which are broken). http://www.rssboard.org/rss-profile#data-types-characterdata There's also no actual spec about how this should work.

This will be a total beast to fix. The current design is very clean in that all (well, nearly all) xml/html escaping is pushed back to the templates. This allows plugins to substitute fields in the templates without worrying about getting escaping right in the plugins -- and a plugin doesn't even know what kind of template is being filled out when it changes a field's value, so it can't do different types of escaping for different templates.

The only reasonable approach seems to be extending HTML::Template with an ESCAPE=RSS and using that. Unfortunately its design does not allow doing so without hacking its code in several places. I've contacted its author to see if he'd accept such a patch.

(A secondary bug is that using meta title currently results in unnecessry escaping of the title value before it reaches the template. This makes the escaping issues show up much more than they need to, since lots more characters are currently being double-escaped in the rss.)

--Joey

Update: Ok, I've fixed this for titles, as a special case, but the underlying problem remains for other fields in rss feeds (such as author), so I'm leaving this bug report open. --Joey

I'm curious if there has been any progress on better RSS output? I've been prototyping a new blog and getting good RSS out of it seems important as the bulk of my current readers use RSS. I note, in passing that the "more" plugin doesn't quite do what I want either - I'd like to pass a full RSS feed of a post and only have "more" apply to the front page of the blog. Is there a way to do that? -- ?dtaht

To be clear, the RSS spec sucks to such an extent that, as far as I know, there is no sort of title escaping that will work in all RSS consumers. Titles are currently escaped in the way that tends to break the fewest according to what I've read. If you're unlucky enough to have a "&" or "<" in your name, then you may still run into problems with how that is escaped in rss feeds. --Joey

Posted Thu Jul 31 15:52:20 2008

When committing a page like this one, with an escaped toc directive in it:

[[!toc  ]]

The recentchangesdiff comes back with it unescaped. Which can be confusing.

Posted Fri Jul 11 09:52:17 2008

It would be nice if the aggregate plugin would try to extract the m/ctime out of each post and touch the files on the filesystem appropriately, so that ikiwiki reflects the actual time of the post via the inline plugin, rather than the time when the aggregation ran to pull the post in. --madduck

Like this? (Existing code in aggregate.pm...) --Joey

# Set the mtime, this lets the build process get the right creation
# time on record for the new page.
utime $mtime, $mtime, pagefile($guid->{page})
    if defined $mtime && $mtime <= time;

I'll have to debug this, it's not working here... and this is an ikiwiki aggregator scraping another ikiwiki site.

Any news about this? --Joey

That would be useful to avoid "flooding" with old content when something new is added with aggregate and then listed with the inline directive. -- hugo

Posted Wed Mar 12 08:30:06 2008

I am using mercurial as RCS backend and ikiwiki 2.40.

It seems that, when adding a blog post, it is not immediately commited to the mercurial repo. I have a page with this directive:

[[!inline  pages="journal/blog2008/* and !*/Discussion" show="0" feeds="no" actions="yes" rootpage="journal/blog2008"]]

When I add a blog post, I see it on the wiki but it doesn't appear on History or RecentChanges. If I run hg status on the wiki source dir, I see the new file has been marked as A (ie, a new file that has not been commited).

If I then edit the blog post, then the file gets commited and I can see the edit on History and RecentChanges. The creation of the file remains unrecorded. --?buo

Ikiwiki calls rcs_add() if the page is new, followed by rcs_commit(). For mercurial, these run respectively hg add and hg commit. If the add or commit fails, it will print a warning to stderr, you might check apache's error.log to see if there's anything there. --Joey

The problem was using accented characters (é, í) on the change comments. I didn't have an UTF-8 locale enabled in my setup file. By coincidence this happened for the first time in a couple of consecutive blog posts, so I was mistaken about the root of the problem. I don't know if you will consider this behavior a bug, since it's strictly speaking a misconfiguration but it still causes ikiwiki's mercurial backend to fail. A quick note in the docs might be a good idea. For my part, please close this bug, and thanks for the help. --?buo

So, in a non-utf8 locale, mercurial fails to commit if the commit message contains utf8? --Joey

(Sorry for the delay, I was AFK for a while.) What I am seeing is this: in a non-utf8 locale, using mercurial "stand-alone" (no ikiwiki involved), mercurial fails to commit if the commit message has characters such as á. If the locale is utf8, mercurial works fine (this is with mercurial 1.0).

However, the part that seems a bit wrong to me, is this: even if my locale is utf8, I have to explicitly set a utf8 locale in the wiki's setup file, or the commit fails. It looks like ikiwiki is not using this machine's default locale, which is utf8. Also, I'm not getting any errors on apache's error log.

Wouldn't it make sense to use the machine's default locale if 'locale' is commented out in the setup file?

Ikiwiki wrappers only allow whitelisted environment variables through, and the locale environment variables are not included currently.

But that's not the whole story, because "machine's default locale" is not very well defined. For example, my laptop is a Debian system. It has a locale setting in /etc/environment (LANG="en_US.UTF-8"). But even if I start apache, making sure that LANG is set and exported in the environment, CGI scripts apache runs do not see LANG in their environment. (I notice that /etc/init.d/apache explocitly forces LANG=C. But CGI scripts don't see the C value either.) Apache simply does not propigate its runtime environment to CGI scripts, and this is probably to comply with the CGI specification (although it doesn't seem to completly rule out CGI's being passed other variables).

If mercurial needs a utf-8 locale, I guess the mercurial plugin needs to check if it's not in one, and do something sane (either fail earlier, or complain, or strip utf-8 out of comments). --Joey

Posted Wed Mar 5 12:34:49 2008

As far as I can tell, ikiwiki is not checking the SSL certificate of the remote host when using openid authentication. If so, this would allow for man-in-the-middle type attacks. Alternatively, maybe I am getting myself confused.

Test #1: Enter URL as openid server that cannot be verified (either because the certificate is self signed or signed by an unknown CA). I get no SSL errors.

Test #2: Download net_ssl_test from dodgy source (it uses the same SSL perl library, and test again. It seems to complain (on same site ikiwiki worked with) when it can't verify the signature. Although there is other breakage with the version I managed to download (eg. argument parsing is broken; also if I try to connect to a proxy server, it instructs the proxy server to connect to itself for some weird reason).

For now, I want to try and resolve the issues with net_ssl_test, and run more tests. However, in the meantime, I thought I would document the issue here.

-- Brian May

Openid's security model does not rely on the openid consumer (ie, ikiwiki) performing any sanity checking of the openid server. All the security authentication goes on between your web browser and the openid server. This may involve ssl, or not.

Note that I'm not an openid expert, and the above may need to be taken with a grain of salt. I also can make no general statements about openid being secure. ;-) --Joey

For example, my openid is "http://joey.kitenet.net/". If I log in with this openid, ikiwiki connects to that http url to determine what openid server it uses, and then redirects my browser to the server (https://www.myopenid.com/server), which validates the user and redirects the browser back to ikiwiki with a flag set indicating that the openid was validated. At no point does ikiwiki need to verify that the https url is good. --Joey

Ok, so I guess the worst that could happen when ikiwiki talks to the http address is that it gets intercepted, and ikiwiki gets the wrong address. ikiwiki will then redirect the browser to the wrong address. An attacker could trick ikiwiki to redirect to their site which always validates the user and then redirects back to ikiwiki. The legitimate user may not even notice. That doesn't so seem secure to me...

All the attacker needs is access to the network somewhere between ikiwiki and http://joey.kitenet.net/ or the ability to inject false DNS host names for use by ikiwiki and the rest is simple.

-- Brian May

I guess that the place to add SSL cert checking would be in either LWPx::ParanoidAgent or Net::OpenID::Consumer. Adding it to ikiwiki itself, which is just a user of those libraries, doesn't seem right.

It's not particularly clear to me how a SSL cert can usefully be checked at this level, where there is no way to do anything but succeed, or fail; and where the extent of the check that can be done is that the SSL cert is issued by a trusted party and matches the domain name of the site being connected to. I also don't personally think that SSL certs are the right fix for DNS poisoning issues. --Joey

I was a bit vague myself on the details on openid. So I looked up the standard. I was surprised to note that they have already considered these issues, in section 15.1.2, http://openid.net/specs/openid-authentication-2_0.html#anchor41.

It says:

"Using SSL with certificates signed by a trusted authority prevents these kinds of attacks by verifying the results of the DNS look-up against the certificate. Once the validity of the certificate has been established, tampering is not possible. Impersonating an SSL server requires forging or stealing a certificate, which is significantly harder than the network based attacks."

With regards to implementation, I am surprised that the libraries don't seem to do this checking, already, and by default. Unfortunately, I am not sure how to test this adequately, see Debian bug #466055. -- Brian May


I think Crypt::SSLeay already supports checking the certificate. The trick is to get LWP::UserAgent, which is used by LWPx::ParanoidAgent to enable this checking.

I think the trick is to set one of the the following environment variables before retrieving the data:

$ENV{HTTPS_CA_DIR} = "/etc/ssl/certs/";
$ENV{HTTPS_CA_FILE} = "/etc/ssl/certs/file.pem";

Unfortunately I get weird results if the certificate verification fails, see Debian bug #503440. It still seems to work though, regardless.

-- Brian May

Posted Sat Feb 16 05:19:30 2008

If I try to authenticate using openid to my site, it tries to create a http or https connection to the openid server. This doesn't work, because the direct connection is blocked by the firewall.

It would be good if ikiwiki supported setting up a proxy server to solve this.

I have found if I add:

newenviron[i++]="HTTPS_PROXY=http://host.domain.com:3128";

to IkiWiki/Wrapper.pm it solves the problem for https requests, however it obviously would be preferred if the proxy name is not hard coded.

Also, the ability to set HTTPS_CA_FILE and HTTPS_CA_DIR might benefit some people. Then again, it I can't see any evidence that the SSL certificate of the server is being checked. See the bug report I filed on this separate issue.

Unfortunately, HTTP_PROXY doesn't work for http:// requests, it looks like that library is different.


Update 2008-10-26:

Better solution, one that works for both http and https, and uses config options. It appears to work...

Note that using $ua->proxy(['https'], ...); won't work, you get a "Not Implemented" error, see http://community.activestate.com/forum-topic/lwp-https-requests-proxy. Also see Debian bug #129528.

Also note that the proxy won't work with liblwpx-paranoidagent-perl, I had to remove liblwpx-paranoidagent-perl first.

louie:/usr/share/perl5/IkiWiki/Plugin# diff -u openid.pm.old openid.pm
--- openid.pm.old   2008-10-26 12:18:58.094489360 +1100
+++ openid.pm   2008-10-26 12:40:05.763429880 +1100
@@ -165,6 +165,14 @@
            $ua=LWP::UserAgent->new;
    }
 
+   if (defined($config{"http_proxy"})) {
+       $ua->proxy(['http'], $config{"http_proxy"});
+   }
+
+   if (defined($config{"https_proxy"})) {
+       $ENV{HTTPS_PROXY} = $config{"https_proxy"};
+   }
+
    # Store the secret in the session.
    my $secret=$session->param("openid_secret");
    if (! defined $secret) {

Brian May

Rather than adding config file settings for every useful environment variable, there is a ENV config file setting that can be used to set any environment variables you like. So, no changed needed. --Joey

One thing I don't like about using ikiwiki for tracking bugs is I don't get notified when changes are made :-(.

Anyway, if you look at the code I pasted above, the environment variables do not work for http:// - you have to use $ua->proxy(...) for them. This is significant, because all openid servers in my version appear to have been defined with http:// not https:// in /usr/share/ikiwiki/openid-selector/ikiwiki/openid/openid-jquery.js

Use $ua->env_proxy() to get it to read the environment variables. Then http:// does work.

Unfortunately this breaks https:// even more - but nothing I do seems to make https:// work anymore.

LWP::UserAgent defaults to not caring about proxy settings in the environment. (To give control over the result, I guess?) To get it to care, pass env_proxy => 1 to the constructor. Affected plugins: aggregate, openid, pinger. This probably wants to be on by default, and might not need to be configurable. --schmonz

Okay, in a real-world scenario it does need to be configurable. A working implementation (tested with aggregate, not tested with the other two plugins) is in my git, commit 91c46819dee237a281909b0c7e65718eb43f4119. --schmonz

Oh, and according to the LWPx::ParanoidAgent docs, "proxy support is explicitly removed", so if ikiwiki can preferentially find that installed, even with the above commit, openid won't be able to traverse a proxy. --schmonz

Available in a git repository branch.
Branch: schmonz/proxy
Author: schmonz

I've redone this from scratch, much more simply, on a new branch. --schmonz.

Posted Thu Feb 14 05:45:28 2008 Tags:

After installing IkiWiki 2.16 on Mac OS X 10.4 server I attempted to use "/Library/Application\ Support/IkiWiki/Working\ Copies" as the parent of my $SRCPATH and get "skipping bad filename" errors for any .mdwn file in that directory:

skipping bad filename /Library/Application Support/IkiWiki/Working Copies/ikiwikinewt/index.mdwn

Tthe .ikiwiki directory is correctly created in that directory. I switched to using a path with no spaces and it works correctly.

Posted Sat Dec 22 11:25:15 2007

The brokenlinks plugin falsely complains that formatting has a broken link to smileys, if the smiley plgin is disabled. While the page links to it inside a conditional, and so doesn't show the link in this case, ikiwiki scans for links w/o looking at conditionals and so still thinks the page contains the link.

Posted Mon Aug 27 21:59:01 2007

If sandbox/page.mdwn has been generated and sandbox/sidebar.mdwn is created, the sidebar is only added to sandbox and none of the subpages. --TaylorKillian

Yes, a known bug. As noted in the code: --Joey

# FIXME: This isn't quite right; it won't take into account
# adding a new sidebar page. So adding such a page
# currently requires a wiki rebuild.
add_depends($page, $sidebar_page);
Posted Fri Jun 1 16:46:06 2007

In markdown syntax, none of the other special characters get processed inside a code block. However, in ikiwiki, wiki links and preprocessor directives still get processed inside a code block, requiring additional escaping. For example, [links don't work](#here), but a <a href="../ikiwiki/wikilink/">wikilink</a> becomes HTML. --JoshTriplett

Indented lines provide a good way to escape a block of text containing markdown syntax, but ikiwiki links like [[this]] are still interpreted within such a block. I think that intepretation should not be happening. That is I should be able to write:

<span class="createlink"><a href="/ikiwiki.cgi?page=this&amp;from=bugs%2Fwiki_links_still_processed_inside_code_blocks&amp;do=create" rel="nofollow">?</a>this</span>

and have it render like:

[[this]]

--?cworth


Has there been any progress or ideas on this bug recently? I use an expanded CamelCase regexp, and without much escaping in freelink text, or url links, or in codeblocks I get IkiWiki's attempt at creating a "link within a link".

I have no ideas other than perhaps once IkiWiki encounters [[ or the position is reset with a backreference from a CamelCased word, further processing of wikilinks is disabled until the position is reset and a "no not makelinks" flag or variable is cleared.

I've come up with some really ugly workarounds to handle case specific stuff like codeblocks but the problem creeps up again and again in unexpected places. I'd be happy to come up with a patch if anyone has a bright idea on a nice clean way (in theroy) to fix this. I'm out of ideas.

--CharlesMauch

I've moved the above comment here because it seems to be talking about this bug, not the similar Smileys bug.

In the case of either bug, no, I don't have an idea of a solution yet. --Joey

I've now solved a similar bug involving the smiley plugin. The code used there should give some strong hints how to fix this bug, though I haven't tried to apply the method yet. --Joey

As far, as I can see, smileys bug is solved by checking for code/pre. In this case, however, this is not applicable. WikiLinks/directives should be expanded before passing text to formatter, as their expansion may contain markup. Directives should be processed before, as they may provide partial markup (eg template ones), that have no sense except when in the page cotext. Links should be processed before, because, at least multimarkdown may try to expand them as anchor-links.

For now, my partial solution is to restrict links to not have space at the start, this way in many cases escaping in code may be done in natural way and not break copypastability. For example, shell 'if [[ condition ]];' will work fine with this.

Maybe directives can also be restricted to only be allowed on the line by themselves (not separated by blank lines, however) or something similar.

--?isbear

Debian bug #487397

Posted Fri Dec 8 16:17:38 2006

The header of subpages always links to its "superpage", even if it doesn't exist. I'm not sure if this is a feature or a bug, but I would certainly prefer that superpages weren't mandatory.

For example, if you are in 'example/page.html', the header will be something like 'wiki / example / page'. Now, if 'example.html' doesn't exist, you'll have a dead link for every subpage.


This is a bug, but fixing it is very tricky. Consider what would happen if example.mdwn were created: example/page.html and the rest of example/ would need to be updated to change the parentlink from a bare word to a link to the new page. Now if example.mdwn were removed again, they'd need to be updated again. So example/ depends on example. But it's even more tricky, because if example.mdwn is modified, we don't want to rebuild example/*!

ikiwiki doesn't have a way to represent this dependency and can't get one without a lot of new complex code being added.

Note that this code has now been added. In new terms, example/* has a presence dependency on example. So this bug is theoretically fixable now. --Joey

For now the best thing to do is to make sure that you always create example if you create example/foo. Which is probably a good idea anyway..


Note that this bug does not exist if the wiki is built with the "usedirs" option, since in that case, the parent link will link to a subdirectory, that will just be missing the index.html file, but still nicely usable. --Joey


http://www.gnu.org/software/hurd/hurd/translator/writing.html does not exist. Then, on http://www.gnu.org/software/hurd/hurd/translator/writing/example.html, in the parentlinks line, writing links to the top-level index file. It should rather not link anywhere at all. --tschwinge

So, the bug has changed behavior a bit. Rather than a broken link, we get a link to the toplevel page. This, FWIW, is because the template now uses this for each parentlink:

<a href="<TMPL_VAR URL>"><TMPL_VAR PAGE></a>/

Best workaround is still to enable usedirs. --Joey

Posted Mon Aug 21 17:48:26 2006
  • Has bugs updating things if the bestlink of a page changes due to adding/removing a page. For example, if Foo/Bar links to "Baz", which is Foo/Baz, and Foo/Bar/Baz gets added, it will update the links in Foo/Bar to point to it, but will forget to update the backlinks in Foo/Baz.

    The buggy code is in refresh(), when it determines what links, on what pages, have changed. It only looks at changed/added/deleted pages when doing this. But when Foo/Bar/Baz is added, Foo/Bar is not changed -- so the change it its backlinks is not noticed.

    To fix this, it needs to consider, when rebuilding Foo/Bar for the changed links, what oldlinks Foo/Bar had. If one of the oldlinks linked to Foo/Baz, and not links to Foo/Bar/Baz, it could then rebuild Foo/Baz.

    Problem is that in order to do that, it needs to be able to tell that the oldlinks linked to Foo/Baz. Which would mean either calculating all links before the scan phase, or keeping a copy of the backlinks from the last build, and using that. The first option would be a lot of work for this minor issue.. it might be less expensive to just rebuild all pages that Foo/Bar links to.

    Keeping a copy of the backlinks has some merit. It could also be incrementally updated.

    This old bug still exists as of 031d1bf5046ab77c796477a19967e7c0c512c417.

  • And if Foo/Bar/Baz is then removed, Foo/Bar gets a broken link, instead of changing back to linking to Foo/Baz.

    This part was finally fixed by commit f1ddf4bd98821a597d8fa1532092f09d3d9b5483.

Posted Thu Jun 22 20:51:19 2006

RSS output contains relative links. Ie. http://kitenet.net/~joey/blog/index.rss contains a link to http://kitenet.net/~joey/blog/../blog.html

Posted Thu Jun 22 20:51:19 2006

If a file in the srcdir is removed, exposing a file in the underlaydir, ikiwiki will not notice the removal, and the page from the underlay will not be built. (However, it will be if the wiki gets rebuilt.)

This problem is caused by ikiwiki storing only filenames relative to the srcdir or underlay, and mtime comparison not handling this case.

A related problem occurs if changing a site's theme with the theme plugin. The style.css of the old and new theme often has the same mtime, so ikiwiki does not update it w/o a rebuild. This is worked around in theme.pm with a special-purpose needsbuild hook. --Joey

Posted Thu Jun 22 20:51:19 2006

Web browsers don't word-wrap lines in submitted text, which makes editing a page that someone wrote in a web browser annoying (gqip is vim user's friend here). Is there any way to improve this?

See "using the web interface with a real text editor" on the tips page. --JoshTriplett

Would it be useful to allow a "max width" plugin, which would force on commit the split of long lines ?

Please, no. That would wreak havoc on code blocks and arguments to preprocessor directives, and it would make bulleted lists and quoted blocks look bogus (because the subsequent lines would not match), among other problems. On the other hand, if you want to propose a piece of client-side JavaScript that looks at the active selection in a text area and word-wraps it, and have a plugin that adds a "Word-Wrap Selection" button to the editor, that seems fine. --JoshTriplett

Posted Thu Jun 22 20:51:19 2006