# Web Document Editor Spec (WIP)

## Context and Overview

Across a number of our projects in recent years, we have discerned a growing need for an online editor that can span the gulf between less structured or narrative **documents** and structured or templated **forms**.  Note that these opposed terms are meant to be intelligible to the non-programmer, and interpreted in that light; "document" here does not refer to document-based non-relational storage or any other technical implementation.

### Documents, in user perspective

A **document** refers to a linear body of formatted text, ie. the object one would author in any word-processing application (Word, Google Docs,...). Its contents are at least implicitly and historically organized around the metaphor of a paper document, with the structure of information built around formatting requirements of sections, paragaphs, headers, and so forth. A document is a singular and complete entity, in "one file" so to speak.

**Advantages of the document paradigm**:
 - intelligible to the typical business or academic (in our case) end-user
 - compositional in nature; encourages writing rather than filling in isolated blanks under prompts
 - more easily translated between web and paper (or PDF); the document already *is* its own report, unlike a form which produces a separate report

**Disadvantages**:
 - while a starting template or outline can be produced for the user, it then becomes a black box once it enters a word-processing application, such that the additions of the user cannot be retrieved in a structured manner
 - formatting tied to the particular application used, difficult to represent faithfully elsewhere
 - difficult to systematically customize template or starting document (see data discussion below)
 - impossible to integrate other inputs smoothly and react to them (checkboxes which open / close additional writing prompts)
 - no ability to require completion or set rules of validation

### Forms, in user perspective

A **form** is broadly a set of structured questions, inputs, or open blanks that must be completed and submitted. Forms involve familiar controls (checkboxes, selections, add-another, short input) and spaces for writing (text box, rich text box). A form must be completed in a certain fashion, and may respond to the user's interaction by adding or removing steps.

**Advantages of the form paradigm**:
 - clear steps to completion
 - validation of entries
 - ability to change in response to input
 - built for producing usable and structured data

**Disadvantages**:
 - in the singular context of writing extended or richly formatted text, we hit a wall of usability
 - traditional WYSIWYG solutions loading in the middle of a form are heavy and awkward, rightly met with opposition from users
 - plain text is often insufficient; markdown and analogous options are a no-go for many of our users
 - when presented with the need to enter lengthy formatted text into a web form, users often compose in their favorite word-processing suite and then paste into the editor, producing even worse formatting

### The Problem

As indicated above, the crux of the matter is formatted text. When the need emerges for a kind of filled-out report that involves lengthy written responses, one is left either (A) sending a document template off into the black box of a word processor and having the user upload the result, or (B) asking a user to enter rich text into a series of inputs within a traditional form. The latter solution does not usually "feel" like a continuous document to the user, and requires too many steps for setting up (editing a document template is quick, restructuring a form is slow).

#### Scenarios

**(A. Program Review)** Periodic academic program reviews require a department to complete a series of documents, each of which must contain pre-loaded excerpts of data (tables, graphs) and a set of prompts for writing narrative or extended text responses. The appearance of these documents should be print-ready and clean, and much of the interaction is via lengthy prose replies to the prompts. Therefore, a traditional document approach is more appropriate at face value, yet (1) each document needs data to be interpolated into it at various points, and (2) certain text replies to prompts will need to be extracted later for further response (a later follow-up report might quote the user's response to a specific item and prompt for further comment on progress). Therefore, a form would be required for the structured data, whereas a traditional document is more amenable to the writing and throught process of those completing it.

We have attempted compromises with little success:
 - One strategy is to separate all the data (charts, tables) into a seperate report, so that there is no need to inject data into the document. Then, a traditional document approach can be used, with question prompts referencing eg. "data report item 3A." This, however, does not help with extracting responses to particular questions for later follow-up.
 - A second strategy is to facilitate working with documents by using Google Docs. There, we are able to (1) quickly distribute a template to a set of documents, (2) inject data at various points, (3) share these with a list of users, (4) obtain filled out responses rapidly. A Google Document is an advance over a traditional word-processing document for its deployment and more consistent formatting (which still leaves much to be desired, but feels sufficiently like a document for most end users). However, (1) the template that we deploy cannot contain "locked" elements and may therefore be accidentally overwritten by the user, and (2) there is still very limited ability to systematically fetch responses to particular prompts.