Howto avoid heavy files in ikiwiki git repo

Continuation of discussion at git-annex forum turns out the git-annex tricks could be avoided.

Setup on remote server

On the server activate album and underlay plugins in $wiki.setup file

add_plugins:
- album
- underlay

Configure underlay plugin

add_underlays:
- /home/$user/$wiki.underlay

Create underlay directory and init git annex in direct mode

mkdir ~/$wiki.underlay
cd ~/$wiki.underlay;git init;git annex init $srcunderlay; git annex direct

Build ikiwiki for good measure ikiwiki --setup $wiki.setup --rebuild

Setup on local laptop

Clone to laptop and initialise annex repo

git clone ssh://$server/$wiki.git ~/$wiki
git clone ssh://$server/$wiki.underlay ~/$wiki.underlay
cd $wiki.underday;git-annex init $wrkunderlay
git remote add $srcunderlay ssh://$server/$wiki.underlay

You now have an annex repo in the local $wiki.underlay called $wrkunderlay and one in the $wiki.underlay directory on the remote server called $srcunderlay.

Add content locally

Add content to local $wiki directory in this case create $album.mdwn files for every album you have. Then git add;git commit files containing at the minimum the following

[[!album  ]]

Create directories in the local $wiki.underlay corresponding to the album files in the local $wiki dir. Ie. create a directory named $album for every $album.mdwn file. Copy hi-res jpg files to each directory in the local $wiki.underlay and add + commit.

git annex add .
git commit -m 'jpgs added'

Push to remote

cd $wrkunderlay; git-annex copy --to $srcunderlay .; git-annex sync
cd $wrkdir;git push

That's it! Ikiwiki should update the website and treat the jpg's as if they were part of the standard file structure.

How to accomplish this using the web interface is another questions. I guess the plugins have to setup and upload to underlaydir somehow.

My guess is that you have to git-annex copy the $wiki.underlay files to $srcunderlay before running git push from the local $wiki directory. Haven't tested this yet though.

"I guess the plugins have to setup and upload to underlaydir somehow" - yes, the hypothetical specialized CGI interface mentioned at the end of album would ideally be able to do that.

I'd also like to be able to keep full-resolution photos on my laptop but mangle them down to a more web-compatible resolution in a separate underlay that is what actually gets uploaded, also as described on that page - but that doesn't make a great deal of sense for a non-CGI workflow, since if you're uploading full-resolution photos to the CGI, you've already done the big data transfer whether you intended to or not :-)

Comment by smcv Thu Sep 26 09:11:55 2013

I recreated this post in the tips section Ikiwiki with git-annex, the album and the underlay plugins as per anarcats suggestion

@smcv

> I'd also like to be able to keep full-resolution photos on my laptop 
> but mangle them down to a more web-compatible resolution in a separate 
> underlay that is what actually gets uploaded, also as described on that page

Yes I can see that some have use for that function. I try to provide hi-res version of all my images as they are more useful for people. For the stockholm site though I've halved the size already as there are to many photos and of to dubious quality... and it's easy enough to do that locally before uploading.

As you can see on the about page I use a shell script to do the metadata stuff even adding the actual download link for the hi-res version. Couldn't figure out how to do in in album.pm before I opted for a simpler solution ;) So for me local work is necessary anyway atm.

Another potential problem with the underlay is how changes to underlay files would be detected?

Feel free to delete this whole forum post in favor of the version in tips.

Comment by Kalle Thu Sep 26 09:38:49 2013

"how changes to underlay files would be detected?"

Changes to files in underlays are picked up automatically, as long as their mtime changed.

Couldn't figure out how to do [metadata] in in album.pm

Yeah, I need to implement a hook mechanism or something (and work out where I put the exif plugin mentioned in the page).

Comment by smcv Thu Sep 26 10:38:45 2013