pastebin - collaborative debugging

TECHNICAL README FOR IKIWIKI ATTACHMENT PLUGIN
==============================================

DISCLAIMER: This is a draft copy. Clumsy grammar, omitted detail, typos, and non-working examples are likely.

INTRODUCTION
============
This plugin is a work in progress attempt to provide file upload functionality to an Ikiwiki installation.

OVERVIEW
========
* An attachment is a file which has been associated with a specific page in the wiki.
* One page can have multiple attachments.
* The attachment is stored alongside the page in the ikiwiki source directory. For example, if _img.gif_ were attached to _bugs/done_ it would be stored in $config{srcdir}_/bugs/done/img.gif_.
* A file can be attached via the web interface or by simply moving it to the appropriate place in the $config{srcdir}_source directory.
* Ikiwiki administrators can configure their site to allow attachments on every page (by setting $config{attach}{every_page} to a true value), or only on pages containing the '_attach_' preprocessor directive (this approach requires $config{attach}{every_page} to be false, if set at all). Note: If pursuing the latter route, administrators should remember to lock the pages on which he wishes to prohibit attachments.
* Before files are attached they are temporarily stored in a subdirectory of the $config{srcdir} directory, as specified by the $config{attach}{dir} setting. (This directory is automatically created as hidden, so it does not appear in $config{destdir}) This allows post-upload checks to be made without disturbing the main source directory / RCS repository. It also enables the administrator to recover files which were attached but failed validation.

THE UPLOAD PROCESS
==================
1. The user locates the attachment form on the page he wishes to attach a file to. (If the form isn't present, attachments aren't allowed on that page).
2. He uses the Browse button to locate the desired file on his local machine. He use the Attach button to submit the form.
3. The form is POSTed to the Ikiwiki CGI as a "multipart/form-data" with the following fields:
  a. do=attach. This ensures that the CGI invokes the attachments plugin to handle this request.
  b. pagename=$pagename. This is the name of the page to which the file is being attached. The URL http://localhost/ikiwiki/roadmap/index.html would result in a _pagename_ of _roadmap_, for example.
  c. datafile=$filename. The local filename of the attachment.
4. The $config{attach}{enabled} flag is checked for truth. If false, uploads are disabled for the entire wiki, so the CGI aborts.
5. The $pagename is checked for validity by determining whether it is a member of the set of existing pages. If not, the CGI aborts.
6. If attachments are enabled, but 'every_page' is false, attachments are only allowed on pages containing an '[[attach ]]' directive. To confirm that this condition is met, we manually scrape the page in question and check for the presence of said directive; if it isn't found, the CGI aborts.
7. The reported MIME type of the file is examined to determine whether the administrator has prohibited attachments of this type. If he has, the CGI aborts. (The MIME checking process is explained in detail later.)
8. The reported size of the file is examined to determine whether the administrator has prohibited attachments of this type. If he has, the CGI aborts. (The size checking process is explained in detail later.)
9. The user's IP address is compared to an administrator-defined blacklist; if it matches the CGI aborts. The IP-address checking process is explained in detail later.)
10. The user-supplied filename is stripped of dangerous / odd characters.
11. Having passed all pre-upload checks, the attached file is spooled from the temporary location where it was buffered to $config{attach}{dir}.
12. Post-upload checks occur next.
13. As we use the filesystem to unite attachments with pages, it is essential that an attachment resides in a subdirectory of $config{srcdir}. If the page in question doesn't have subpages, then it may not live in it's own directory. In this case, the directory is created.
14. The attachment is moved from $config{attach}{dir} to the appropriate subdirectory of $config{srcdir}.
15. IkiWiki::render is called to copy the attachment to the output directory, and alert other interested plugins of its existence.
16. Ikiwiki::add_depends() is called to associate the attached file with its page. This allows the wiki to update the page if the attachment is deleted (the link from the page to the attachment is removed) or modified (metadata plugins may like to update the page if the file has changed substantially).
17. Ikiwiki::refresh() is called so the page to which the file has been attached is updated with a link to said file.

MIME FILTERING
=============
* Attachments added via the web can be filtered on their MIME type.
* Administrators must choose one of two strategies for dealing with MIME filtering by setting $config{attach}{mime_strategy} to either 'allow,deny' o 'deny,allow'.
* With an 'allow,deny' strategy: "First, all Allow directives are evaluated; at least one must match, or the [attachment] is rejected. Next, all Deny directives are evaluated. If any matches, the [attachment] is rejected. Last, any requests which do not match an Allow or a Deny directive are denied by default." (Excerpt from http://httpd.apache.org/docs/2.0/mod/mod_access.html on which this feature is based).
* With a 'deny,allow' strategy: "First, all Deny  directives are evaluated; if any match, the request is denied unless it also matches an Allow directive. Any requests which do not match any Allow or Deny directives are permitted." (Excerpt from http://httpd.apache.org/docs/2.0/mod/mod_access.html on which this feature is based).
* The 'Allow' and 'Deny' directives are specified as space-separated lists of MIME types, using $config{attach}{mime_allow} and $config{attach}{mime_deny}, respectively.

SIZE FILTERING
==============
* Attachments added via the web can be denied on the basis of their file size.
* $config{attach}{max_kbs} expects a numeric value reflecting the maximum size in kilobytes of any attachment. For example, a value of 1024 would ensure that attachments larger than a megabyte were rejected.
* Setting $config{attach}{max_kbs} to 0 disables size filtering: attachments of any size can be added.

IP FILTERING
===========
* Attachments added via the web can be denied on the basis of their uploader's IP address matching a blacklist.
* The blacklist is defined as a space-separated list of regular expressions as a value of $config{attach}{ban_ips}. For example, a value of '3 127\. ^2[45]' blocks all addresses containing the number 3, containing the octet 127, and starting with a 2 followed by a 4 or 5.

TODO
====
* Sensible way of limiting total size of attachments.
* Optionally, delete files from $config{attach}{dir} if they fail validation.
* Implement per-user quotas in terms of total attachments size.
* Factor out the IP checking functionality into its own plugin, so IPs can be blocked wiki wide.
* Look again at metadata handling via separate plugin.
* Decide how to deal with re-attaching an existing file.
* Extend preprocessor directive to allow more granularity in configuring attachment preferences on a page-by-page basis.
* Enable allowable attachment targets to be specified by means of a pagespec via the Preferences screen.
* Filter on username.
* Allow anonymous users to be prevented from attaching files.
* Implement, probably in separate plugin, the ability to manage (edit, move, delete) attached files.
* Improve appearance of "attached successfully" screen.
* Make statable metadata about attachments available to page.tmpl, so, for example, each attachment can be listed along with its size.
* Enable administrators to define arbitrary external pipes for post-upload validation. Allowing virus checking, for example.
* Offer optional AJAX version of attachment form.
* Provide control over which attachments are listed on a page.
etc.