BRUTE

/*

# brute

BRUTE stands for: Bibliotik Retail Uploader Torrent Epubs.
It does not mean anything, but BRUTEs are not known to be particularly
articulate.

In short, BRUTE allows uploading a retail epub to Bib from the comfort of the
command line.

It's not the prettiest thing, being hacked together and mashed up in a single
file for ease of sharing, but BRUTE will try its best to help make quality
uploads.
This means it will automate the boring stuff, and let the user focus on making
sure the torrent metadata is accurate and complete.

BRUTE retrieves metadata from the epub and Goodreads automatically, and forces
the user to merge and edit the results as needed.
Everything is saved to a yaml file, for (optional) manual editing before the
actual upload.

## Building

You need a working Go installation: see https://golang.org/doc/install.
Put this file in a 'brute' directory in your $GO_PATH, cd to it and just run:

    $ go get .
    $ go install ...brute

or if you don't want to install it:

    $ go get .
    $ go run brute.go

To update its dependancies, rebuild from time to time:

    $ go get -u .
    $ go install ...brute

BRUTE was tested on Linux, but it should probably work wherever you can install
Go.

## Requirements

- a WhatIMG account
- a GoodReads account for which you've requested an API Key: see
  https://www.goodreads.com/api/keys

## Usage

BRUTE only requires one existing epub file as argument.

    $ brute /path/to/book.epub [/path/to/cover.jpg]

It will:

 - read epub metadata
 - retrieve information from Goodreads if ISBN is found
 - allow (alright, force) the user to edit the merged information to prepare
   the bibliotik upload form
 - save that information to /path/to/book.yaml for manual editing if deemed
   necessary
 - upload the cover to whatimg and retrieve its url

At this point BRUTE will ask if you want to upload directly or not.
If you say no, you will be able to manually edit the yaml file as you wish.
Run BRUTE again with the same argument, and it will load the yaml file with your
changes, and proceed to:

 - generate the .torrent from the epub with the user's passkey
 - upload the torrent with the cover and information collected in previous steps
   or loaded from a previously saved yaml file
 - copy the .torrent and .epub where you want to begin seeding
 - archive everything for later reference: epub, cover, torrent, and yaml in a
   tarball.
 - (optionally) remove the original files

Conventions:
- if the cover jpg is not given as argument, the script assumes
/path/to/book.jpg exists and is the cover for this epub.
- it will create /path/to/book.torrent and /path/to/book.yaml

## Configuration

BRUTE needs a configuration file ("config.yaml") with the following information:

    whatimg_login: xx
    whatimg_password: xx
    imgur_client_id: xx
    ptpimg_email: xx
    ptpimg_password: xx
    bibliotik_login: xx
    bibliotik_password: xx
    bibliotik_passkey: https://bibliotik.me/announce.php?passkey=xx
    torrent_watch_directory: /home/user/torrents
    torrent_seed_directory: /home/user/downloads
    archive_directory: /home/user/uploads
    goodreads_api_key: xx
    tag_aliases:
        science-fiction:
        - sf
        - sci-fi

You must configure at least whatimg, imgur or ptpimg to upload covers.
If more than one are defined, BRUTE will ask you which to use.

You can request an anonymous imgur client_id at: http://imgur.com/register/api_anon
You need to head over PTP and look in the forums if you want to use PTPIMG.
WhatIMG is gone.

tag_aliases allows automatic removal of duplicate tags (from duplicate shelves
on Goodreads).

This configuration file must be in the working directory.
Lots of information in a clear text file, I know.
Keep it close to your heart.

## Things BRUTE isn't good at

BRUTE is not good at:
- retrieving information when it cannot find an ISBN number or if Goodreads
does not know about an ISBN
- retrieving editors, contributors, translators, languages (defaults to english)
- it will try to extract tags from Goodreads shelves, and clean out the most
blatantly awful ones. ALWAYS EDIT TAGS MANUALLY.

Fortunately, you can overcome these shortcomings by manually editing
/path/to/book.yaml!
No excuses!

## Things BRUTE is useless at

BRUTE can't help you if you're dealing with anything other than epubs.

You could use it to upload non-retail epubs, but then you MUST manually set the
RetailField to "0" in the yaml file.

## Changelog

v8: Updated to work with latest version of third party packages.
v7: Added support for PTPIMG. Irrationally reluctant to remove WhatIMG support for now.
v6: Added support for imgur (anonymous uploads) now that WhatIMG is down :(
v5: Quick search for duplicates before uploading; code cleanups.
v4: Menus have changed a little; updated to work with latest version of third
party packages.
v3: ask for ISBN if none found; better info if upload form submission fails and
cleanup of generated torrent file; option to upload anonymously; a jpg file can
be given as second parameter, to be used as a cover.
v2: Updated to work with latest version of third party packages, added fetching
cover from GR and asking the user which version to upload.
v1.0.6: Updated to work with latest version of third party packages, and fixed
bug with publishers containing ",". Added possibility to directly edit the yaml
file, using $EDITOR (falling back to nano). When merging epub and GR metadata,
it is safe to ignore the 'type' field.
v1.0.5: Updated to work with latest version of third party packages.
v1.0.4: Updated to work with latest version of third party packages.
v1.0.3: Updated to work with latest version of third party packages.
v1.0.2: In description, title is in italics and author name in bold.
v1.0.1: BRUTE should now correctly retrieve edition year instead of original
publication year from GR.
v1.0: original release.

*/

package main

import (
    "bytes"
    "errors"
    "fmt"
    "io"
    "io/ioutil"
    "log"
    "mime/multipart"
    "net/http"
    "net/http/cookiejar"
    "net/url"
    "os"
    "path/filepath"
    "regexp"
    "strings"
    "time"

    "github.com/PuerkitoBio/goquery"
    "github.com/anacrolix/torrent/bencode"
    "github.com/anacrolix/torrent/metainfo"
    b "github.com/barsanuphe/endive/book"
    e "github.com/barsanuphe/endive/endive"
    u "github.com/barsanuphe/helpers/ui"
    h "github.com/barsanuphe/helpers"
    "github.com/jhoonb/archivex"
    "github.com/skratchdot/open-golang/open"
    "github.com/spf13/viper"
    "golang.org/x/net/publicsuffix"
    "gopkg.in/yaml.v2"
    "github.com/mattn/go-scan"
    "encoding/json"
)

const (
    BRUTE   = "BRUTE"
    BibRoot = "https://bibliotik.me"
    ptpimgURL = "https://ptpimg.me/"

    EPUBFormat      = "15"
    EnglishLanguage = "1"

    EPUBExtension = ".epub"
    JPGExtension  = ".jpg"
    Usage         = "Usage: brute book.epub [cover.jpg]"

    UploadImageAgain                 = "Image URL found, upload again"
    UploadEPUBCover                  = "Upload epub cover"
    UploadAnonymously                = "Do you want the upload to be anonymous"
    UseExistingYAML                  = "Info file already exists! Load"
    AllInfoRetrievedLastChanceToEdit = "\nAll relevant information has been retrieved.\n" +
        "If you are satisfied everything is perfect, you can upload to bibliotik directly.\n" +
        "If unsure, it is not too late to fix things manually.\n"
    FullManualEdit    = "Do you want to manually edit the YAML file before uploading?"
    SearchingForDupes = "\nNow checking if the book you want to upload already exists on bibliotik.\n" +
        "If something is found, check the links to make sure you are not uploading a duplicate" +
        "(check the official rules about what is and what is not considered a dupe).\n" +
        "Search uses exact author and title only, it may not find relevant books if your information " +
        "is different from existing books on bibliotik, or show different editions.\n"
    ConfirmNotADupe   = "After checking, are you certain you can upload this book"
    ConfirmUpload     = "Upload to bibliotik"
    GRCoverDownloaded = "\nThe Goodreads cover was downloaded alongside the epub cover.\n" +
        "Please take a moment to choose the best version.\n"

    ErrorDirectoryDoesNotExist = "%s directory does not exist"
    ErrorCouldNotLogIn         = "Could not login"
    ErrorUploadFailed          = "Upload failed. Check the yaml file for incorrect or forbidden information and try again."
    ErrorAddingToTar           = "Error adding %s"
)

func main() {
    var ui u.UserInterface
    var epubFilename, coverFilename string
    ui = &u.UI{}

    // 1. load configuration
    cfg := Config{}
    if err := cfg.load("config.yaml"); err != nil {
        fmt.Println("Error loading config.")
        return
    }
    if err := cfg.check(); err != nil {
        fmt.Println("Error checking config: " + err.Error())
        return
    }
    // 2. check arguments
    switch len(os.Args) {
    case 2:
        epubFilename = os.Args[1]
        coverFilename = strings.TrimSuffix(os.Args[1], EPUBExtension) + JPGExtension
    case 3:
        epubFilename = os.Args[1]
        coverFilename = os.Args[2]
    default:
        ui.Error("Incorrect input!")
        ui.Info(Usage)
        return
    }
    c := UploadCandidate{
        epubFile:    epubFilename,
        torrentFile: strings.TrimSuffix(epubFilename, EPUBExtension) + ".torrent",
        imageFile:   coverFilename,
        imageFileGR: strings.TrimSuffix(coverFilename, JPGExtension) + "_gr" + JPGExtension,
        infoFile:    strings.TrimSuffix(epubFilename, EPUBExtension) + ".yaml",
    }
    if err := c.check(); err != nil {
        fmt.Println(err.Error())
        return
    }
    // 3. get info from file + GR
    if err := c.getInfo(cfg, ui); err != nil {
        ui.Error(err.Error())
        return
    }
    // 4. upload cover if necessary
    if c.info.ImageField == "" || ui.Accept(UploadImageAgain) {
        // retrieve GR cover
        coverToUpload := c.imageFile
        if err := c.downloadCover(); err != nil {
            ui.Error(err.Error())
            ui.Info("Could not download cover from GR, uploading epub version.")
        } else {
            ui.Title(GRCoverDownloaded)
            if ui.Accept(UploadEPUBCover) {
                ui.Info("Uploading epub cover.")
            } else {
                ui.Info("Uploading GR cover.")
                coverToUpload = c.imageFileGR
            }

        }
        whatimg, imgur, ptpimg := cfg.knownCoverHosts()
        var uploadErr error
        var uploaded bool
        if !uploaded && whatimg && ui.Accept("Upload cover with WhatIMG?") {
            uploadErr = c.uploadCoverToWhatIMG(coverToUpload, cfg)
            uploaded = (uploadErr == nil)
        }
        if !uploaded && ptpimg && ui.Accept("Upload cover with PTPIMG?") {
            uploadErr = c.uploadCoverToPTPIMG(coverToUpload, cfg)
            uploaded = (uploadErr == nil)
        }
        if !uploaded && imgur && ui.Accept("Upload cover with Imgur?") {
            uploadErr = c.uploadCoverToImgur(coverToUpload, cfg)
            uploaded = (uploadErr == nil)
        }
        if uploadErr != nil {
            fmt.Println(uploadErr.Error())
            return
        }
    }
    // 5. save info
    if ui.Accept(UploadAnonymously) {
        c.makeAnonymous()
    }
    err := c.saveInfo()
    if err != nil {
        fmt.Println(err.Error())
    }
    // 6. manual edit
    ui.Title(AllInfoRetrievedLastChanceToEdit)
    if ui.Accept(FullManualEdit) {
        accepted := false
        for !accepted {
            if err := c.manualEdit(ui); err == nil {
                if ui.Accept("Confirm all fields are awesome") {
                    accepted = true
                }
            }
        }
    }
    // 7. check bib for existing uploads
    client, err := c.loginBib(cfg)
    if err != nil {
        ui.Error(err.Error())
        return
    }
    ui.Title(SearchingForDupes)
    foundExisting, err := c.checkBib(client)
    if err != nil {
        ui.Error(err.Error())
        return
    }
    if foundExisting {
        if !ui.Accept(ConfirmNotADupe) {
            ui.Info("Stopping everything.")
            return
        }
    } else {
        ui.Info("Nothing found.")
    }

    if ui.Accept(ConfirmUpload) {
        // 8. generate torrent
        if err := c.generateTorrent(cfg); err != nil {
            ui.Error(err.Error())
            return
        }
        // 9. upload to bibliotik
        if err := c.uploadTorrent(cfg, client); err != nil {
            ui.Error(err.Error())
            if err := os.Remove(c.torrentFile); err != nil {
                ui.Error("Error cleaning torrent file")
            }
            return
        }
        // 10. copy torrent to watch dir && epub to incoming
        if err := c.seed(cfg); err != nil {
            ui.Error(err.Error())
            return
        }
        // 11. backup all relevant files to zip with date
        if err := c.archive(cfg); err != nil {
            ui.Error(err.Error())
            return
        }
        // 12. clean up
        if ui.Accept("Remove original files") {
            if err := c.cleanUp(); err != nil {
                ui.Error(err.Error())
                return
            }
        }
    } else if ui.Accept("Launch this book's Goodreads page in your browser for reference?") {
        if err := open.Start("https://www.goodreads.com/search?q=" + c.info.IsbnField); err != nil {
            ui.Error("Could not open goodreads page!")
        }
    }
}

//------------------------------------------------------------------------------

type Config struct {
    bibUser         string
    bibPassword     string
    bibPasskey      string
    whatimgUser     string
    whatimgPassword string
    ptpimgEmail    string
    ptpimgPassword string
    imgurClientID   string
    torrentWatchDir string
    torrentSeedDir  string
    archiveDir      string
    grApiKey        string
    TagAliases      map[string][]string
}

func (c *Config) load(path string) (err error) {
    conf := viper.New()
    conf.SetConfigType("yaml")
    conf.SetConfigFile(path)

    err = conf.ReadInConfig()
    if err != nil {
        return
    }
    c.bibUser = conf.GetString("bibliotik_login")
    c.bibPassword = conf.GetString("bibliotik_password")
    c.bibPasskey = conf.GetString("bibliotik_passkey")
    c.whatimgUser = conf.GetString("whatimg_login")
    c.whatimgPassword = conf.GetString("whatimg_password")
    c.ptpimgEmail = conf.GetString("ptpimg_email")
    c.ptpimgPassword = conf.GetString("ptpimg_password")
    c.imgurClientID = conf.GetString("imgur_client_id")
    c.torrentSeedDir = conf.GetString("torrent_seed_directory")
    c.torrentWatchDir = conf.GetString("torrent_watch_directory")
    c.archiveDir = conf.GetString("archive_directory")
    c.grApiKey = conf.GetString("goodreads_api_key")
    c.TagAliases = conf.GetStringMapStringSlice("tag_aliases")
    return
}

func (c *Config) check() error {
    if !h.DirectoryExists(c.torrentWatchDir) {
        return fmt.Errorf(ErrorDirectoryDoesNotExist, "Torrent watch")
    }
    if !h.DirectoryExists(c.torrentSeedDir) {
        return fmt.Errorf(ErrorDirectoryDoesNotExist, "Torrent download")
    }
    if !h.DirectoryExists(c.archiveDir) {
        return fmt.Errorf(ErrorDirectoryDoesNotExist, "Archive")
    }
    if whatimg, imgur, ptpimg := c.knownCoverHosts(); !whatimg && !imgur && !ptpimg {
        return errors.New("Image hosting service must be configured to upload covers")
    }
    return nil
}

func (c *Config) knownCoverHosts() (whatimg, imgur, ptpimg bool) {
    if c.whatimgPassword != "" && c.whatimgUser != "" {
        whatimg = true
    }
    if c.ptpimgPassword != "" && c.ptpimgEmail != "" {
        ptpimg = true
    }
    if c.imgurClientID != "" {
        imgur = true
    }
    return
}

//------------------------------------------------------------------------------

// newTrue allows setting the value of a *bool in a struct.
// it is arguably awful.
func newTrue() *bool {
    b := true
    return &b
}

func checkErrors(errs ...error) error {
    for _, err := range errs {
        if err != nil {
            return err
        }
    }
    return nil
}

func login(siteUrl, username, password string) (hc *http.Client, returnData string, err error) {
    form := url.Values{}
    form.Add("username", username)
    form.Add("password", password)
    req, err := http.NewRequest("POST", siteUrl, strings.NewReader(form.Encode()))
    if err != nil {
        fmt.Println(err.Error())
    }
    req.Header.Add("Content-Type", "application/x-www-form-urlencoded")

    options := cookiejar.Options{
        PublicSuffixList: publicsuffix.List,
    }
    jar, err := cookiejar.New(&options)
    if err != nil {
        log.Fatal(err)
    }
    hc = &http.Client{Jar: jar}

    resp, err := hc.Do(req)
    if err != nil {
        fmt.Println(err.Error())
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        err = errors.New("Returned status: " + resp.Status)
    }

    data, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }
    returnData = string(data)
    return
}

func uploadImage(client *http.Client, uploadUrl, image string) (responseData string, err error) {
    // preparing a form
    b := new(bytes.Buffer)
    w := multipart.NewWriter(b)
    // adding image to form
    f, err := os.Open(image)
    if err != nil {
        return "", err
    }
    defer f.Close()
    fw, err := w.CreateFormFile("userfile[]", filepath.Base(image))
    if err != nil {
        return "", err
    }
    if _, err = io.Copy(fw, f); err != nil {
        return
    }
    if err = w.WriteField("upload_to", "0"); err != nil {
        return
    }
    if err = w.WriteField("private_upload", "1"); err != nil {
        return
    }
    if err = w.WriteField("upload_type", "standard"); err != nil {
        return
    }
    w.Close()

    req, err := http.NewRequest("POST", uploadUrl, b)
    if err != nil {
        return "", err
    }
    req.Header.Set("Content-Type", w.FormDataContentType())

    resp, err := client.Do(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        err = errors.New("Returned status: " + resp.Status)
    }

    data, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return "", err
    }
    responseData = string(data)
    return
}

func loginPTPImg(siteUrl, username, password string) (hc *http.Client, returnData string, err error) {
    form := url.Values{}
    form.Add("email", username)
    form.Add("pass", password)
    req, err := http.NewRequest("POST", siteUrl, strings.NewReader(form.Encode()))
    if err != nil {
        fmt.Println(err.Error())
    }
    req.Header.Add("Content-Type", "application/x-www-form-urlencoded")

    options := cookiejar.Options{
        PublicSuffixList: publicsuffix.List,
    }
    jar, err := cookiejar.New(&options)
    if err != nil {
        log.Fatal(err)
    }
    hc = &http.Client{Jar: jar}

    resp, err := hc.Do(req)
    if err != nil {
        fmt.Println(err.Error())
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        err = errors.New("Returned status: " + resp.Status)
    }

    data, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }
    returnData = string(data)
    return
}

func uploadImageToPTPIMG(client *http.Client, uploadUrl, image, apiKey string) (link string, err error) {
    // preparing a form
    b := new(bytes.Buffer)
    w := multipart.NewWriter(b)
    // adding image to form
    f, err := os.Open(image)
    if err != nil {
        return "", err
    }
    defer f.Close()
    fw, err := w.CreateFormFile("file-upload[]", filepath.Base(image))
    if err != nil {
        return "", err
    }
    if _, err = io.Copy(fw, f); err != nil {
        return
    }
    if err = w.WriteField("api_key", apiKey); err != nil {
        return
    }
    w.Close()

    req, err := http.NewRequest("POST", uploadUrl, b)
    if err != nil {
        return "", err
    }
    req.Header.Set("Content-Type", w.FormDataContentType())

    resp, err := client.Do(req)
    if err != nil {
        return "", err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        err = errors.New("Returned status: " + resp.Status)
    }

    data, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }
    ptpimgJSON := []map[string]string{}
    if err := json.Unmarshal(data, &ptpimgJSON); err != nil {
        return "", err
    }
    link = fmt.Sprintf("%s%s.%s", ptpimgURL, ptpimgJSON[0]["code"], ptpimgJSON[0]["ext"])
    return
}


//------------------------------------------------------------------------------

// BUInfo tracks all of the upload form fields.
type BUInfo struct {
    TorrentFileField  string
    authkey           string
    upload            string // default: empty
    TitleField        string
    EditorsField      string
    ContributorsField string
    TranslatorsField  string
    PublishersField   string
    PagesField        string
    AuthorsField      string
    FormatField       string // EPUB == "15"
    IsbnField         string
    TagsField         string
    DescriptionField  string
    RetailField       string // retail == "1"
    NotifyField       string // default "1"
    LanguageField     string // english == "1"
    YearField         string
    ImageField        string
    AnonymousField    string // default "0"
}

// ShowInfo returns a table with relevant information about a book.
func (i *BUInfo) ShowInfo() string {
    var rows [][]string
    rows = append(rows, []string{"TorrentFile", i.TorrentFileField})
    rows = append(rows, []string{"Title", i.TitleField})
    rows = append(rows, []string{"Authors", i.AuthorsField})
    rows = append(rows, []string{"Editors", i.EditorsField})
    rows = append(rows, []string{"Contributors", i.ContributorsField})
    rows = append(rows, []string{"Translators", i.TranslatorsField})
    rows = append(rows, []string{"Publishers", i.PublishersField})
    rows = append(rows, []string{"Isbn", i.IsbnField})
    rows = append(rows, []string{"Pages", i.PagesField})
    rows = append(rows, []string{"Year", i.YearField})
    rows = append(rows, []string{"Format", i.FormatField})
    rows = append(rows, []string{"Language", i.LanguageField})
    rows = append(rows, []string{"Retail", i.RetailField})
    rows = append(rows, []string{"Tags", i.TagsField})
    rows = append(rows, []string{"Image", i.ImageField})
    rows = append(rows, []string{"Description", i.DescriptionField})
    rows = append(rows, []string{"Notify", i.NotifyField})
    rows = append(rows, []string{"Anonymous", i.AnonymousField})
    return e.TabulateRows(rows, "Info", "Book")
}

func (i *BUInfo) load(path string) (err error) {
    data, err := ioutil.ReadFile(path)
    if err != nil {
        return
    }
    return yaml.Unmarshal(data, i)
}

func (i *BUInfo) save(path string) (err error) {
    d, err := yaml.Marshal(i)
    if err != nil {
        return err
    }
    return ioutil.WriteFile(path, d, 0777)
}

func (i *BUInfo) fill(m b.Metadata, torrentPath string) {
    i.TorrentFileField = torrentPath
    i.authkey = ""
    i.upload = ""
    if m.HasAny() {
        seriesInfo := ""
        if m.Series.HasAny() {
            for i := range m.Series {
                seriesInfo += fmt.Sprintf(" (%s, Book %s)", m.Series[i].Name, m.Series[i].Position)
            }
        }
        i.TitleField = m.Title() + seriesInfo
        i.EditorsField = ""      // TODO
        i.ContributorsField = "" // TODO
        i.TranslatorsField = ""  // TODO
        // some publishers in the form 'X, Inc' make the upload form angry
        i.PublishersField = strings.Replace(m.Publisher, ",", " ", -1)
        i.PagesField = m.NumPages
        i.AuthorsField = strings.Join(m.Authors, ", ")
        i.FormatField = EPUBFormat
        i.IsbnField = m.ISBN
        i.TagsField = m.Category + ", " + m.Tags.String()

        r := strings.NewReplacer(
            i.AuthorsField, "[b]"+i.AuthorsField+"[/b]",
            m.Title(), "[i]"+m.Title()+"[/i]",
        )
        i.DescriptionField = r.Replace(m.Description)
        i.RetailField = "1"
        i.NotifyField = "1"
        i.AnonymousField = "0"
        i.LanguageField = EnglishLanguage
        i.YearField = m.EditionYear
    }
}

func (i *BUInfo) generateUploadRequest(uploadURL string) (req *http.Request, err error) {
    // setting up the form
    b := new(bytes.Buffer)
    w := multipart.NewWriter(b)
    // adding the torrent file
    f, err := os.Open(i.TorrentFileField)
    if err != nil {
        return nil, err
    }
    defer f.Close()
    fw, err := w.CreateFormFile("TorrentFileField", filepath.Base(i.TorrentFileField))
    if err != nil {
        return nil, err
    }
    if _, err = io.Copy(fw, f); err != nil {
        return
    }
    errors := []error{}
    errors = append(errors, w.WriteField("authkey", i.authkey))
    errors = append(errors, w.WriteField("upload", i.upload))
    errors = append(errors, w.WriteField("TitleField", i.TitleField))
    errors = append(errors, w.WriteField("EditorsField", i.EditorsField))
    errors = append(errors, w.WriteField("ContributorsField", i.ContributorsField))
    errors = append(errors, w.WriteField("TranslatorsField", i.TranslatorsField))
    errors = append(errors, w.WriteField("PublishersField", i.PublishersField))
    errors = append(errors, w.WriteField("PagesField", i.PagesField))
    errors = append(errors, w.WriteField("AuthorsField", i.AuthorsField))
    errors = append(errors, w.WriteField("FormatField", i.FormatField))
    errors = append(errors, w.WriteField("IsbnField", i.IsbnField))
    errors = append(errors, w.WriteField("TagsField", i.TagsField))
    errors = append(errors, w.WriteField("DescriptionField", i.DescriptionField))
    errors = append(errors, w.WriteField("RetailField", i.RetailField))
    errors = append(errors, w.WriteField("NotifyField", i.NotifyField))
    errors = append(errors, w.WriteField("LanguageField", i.LanguageField))
    errors = append(errors, w.WriteField("YearField", i.YearField))
    errors = append(errors, w.WriteField("ImageField", i.ImageField))
    errors = append(errors, w.WriteField("AnonymousField", i.AnonymousField))
    if err := checkErrors(errors...); err != nil {
        return nil, err
    }
    w.Close()

    req, err = http.NewRequest("POST", uploadURL, b)
    if err != nil {
        return nil, err
    }
    req.Header.Set("Content-Type", w.FormDataContentType())
    return
}

//------------------------------------------------------------------------------

type UploadCandidate struct {
    epubFile    string
    torrentFile string
    imageFile   string
    imageFileGR string
    imageURLGR  string
    infoFile    string
    info        BUInfo
}

func (t *UploadCandidate) loadInfo() error {
    return t.info.load(t.infoFile)
}

func (t *UploadCandidate) saveInfo() error {
    fmt.Println("Saving all gathered information to " + t.infoFile)
    return t.info.save(t.infoFile)
}

func (t *UploadCandidate) makeAnonymous() {
    t.info.AnonymousField = "1"
}

func (t *UploadCandidate) manualEdit(ui u.UserInterface) error {
    data, err := ioutil.ReadFile(t.infoFile)
    if err != nil {
        return err
    }
    if output, err := ui.Edit(string(data)); err == nil {
        if err := ioutil.WriteFile(t.infoFile, []byte(output), 0777); err != nil {
            ui.Error("error writing file")
            return err
        }
    }
    if err := t.loadInfo(); err != nil {
        return err
    }
    fmt.Println(t.info.ShowInfo())
    return nil
}

func (t *UploadCandidate) check() error {
    //  assert epub exists
    if _, err := h.FileExists(t.epubFile); err != nil {
        return errors.New("Epub does not exist!")
    }
    // assert it's an epub
    if !strings.HasSuffix(strings.ToLower(t.epubFile), EPUBExtension) {
        return errors.New(t.epubFile + " is not an epub")
    }
    // assert jpg exists
    if _, err := h.FileExists(t.imageFile); err != nil {
        return errors.New("Cover " + t.imageFile + " does not exist!")
    }
    // assert it's a jpg
    if !strings.HasSuffix(strings.ToLower(t.imageFile), JPGExtension) {
        return errors.New(t.imageFile + " is not an jpg")
    }
    //  assert t does not exist yet
    if _, err := h.FileExists(t.torrentFile); err == nil {
        return errors.New("Torrent " + t.torrentFile + " should not exist!")
    }
    return nil
}

func (t *UploadCandidate) getInfo(cfg Config, ui u.UserInterface) (err error) {
    loadedFromFile := false
    if _, err := h.FileExists(t.infoFile); err == nil {
        if ui.Accept(UseExistingYAML) {
            if err := t.loadInfo(); err != nil {
                return err
            }
            fmt.Println(t.info.ShowInfo())
            if !ui.Accept("Confirm") {
                return errors.New("Incorrect book information rejected.")
            } else {
                loadedFromFile = true
            }
        }
    }
    if !loadedFromFile {
        bk := b.NewBook(ui, 0, t.epubFile, e.Config{GoodReadsAPIKey: cfg.grApiKey, TagAliases: cfg.TagAliases}, true)
        // read epub metadata
        bk.Metadata, err = bk.RetailEpub.ReadMetadata()
        if err != nil {
            if err.Error() == "ISBN not found in epub" {
                isbn, isbnInputErr := e.AskForISBN(ui)
                if isbnInputErr != nil {
                    return err
                }
                bk.Metadata.ISBN = isbn
            } else {
                return err
            }
        }
        // find GoodReads metadata and merge
        fields := []string{"author", "title", "edition_year", "publisher", "description", "category", "tags", "series", "isbn"}
        err = bk.Metadata.SearchOnline(ui, bk.Config, fields...)
        if err != nil {
            return err
        }
        t.imageURLGR = bk.Metadata.ImageURL
        t.info.fill(bk.Metadata, t.torrentFile)
        fmt.Println(t.info.ShowInfo())
        // review/edit field by field, confirm
        for !ui.Accept("Confirm") {
            for _, f := range fields {
                if err := bk.EditField(f); err != nil {
                    fmt.Println("Could not assign new value to field " + f + ", continuing.")
                }
            }
            t.info.fill(bk.Metadata, t.torrentFile)
            fmt.Println(t.info.ShowInfo())
        }
    }
    return
}

func (t *UploadCandidate) downloadCover() error {
    if t.imageURLGR == "" {
        return errors.New("unknown image url")
    }
    if _, err := h.FileExists(t.imageFileGR); err == nil {
        // already downloaded
        return nil
    }
    response, err := http.Get(t.imageURLGR)
    if err != nil {
        return err
    }
    defer response.Body.Close()
    file, err := os.Create(t.imageFileGR)
    if err != nil {
        return err
    }
    defer file.Close()
    _, err = io.Copy(file, response.Body)
    return err
}


func (t *UploadCandidate) uploadCoverToPTPIMG(cover string, cfg Config) error {
    // login
    hc, data, err := loginPTPImg(ptpimgURL + "login.php?1", cfg.ptpimgEmail, cfg.ptpimgPassword)
    if err != nil {
        return err
    }
    // extract api_key
    var apiKey string
    r := regexp.MustCompile(`name='api_key' value='(.*)'`)
    if r.MatchString(data) {
        apiKey = r.FindStringSubmatch(data)[1]
    } else {
        return errors.New("Could not find apî_key: " + ErrorCouldNotLogIn)
    }

    // upload image and get link
    fmt.Printf("Uploading " + filepath.Base(cover) + " to PTPIMG... ")
    link, err := uploadImageToPTPIMG(hc, ptpimgURL + "upload.php", cover, apiKey)
    if err != nil {
        fmt.Println("KO")
        fmt.Println(err.Error())
        return err
    }
    t.info.ImageField = link
    fmt.Println("OK: " + t.info.ImageField)
    return nil
}

func (t *UploadCandidate) uploadCoverToImgur(cover string, cfg Config) error {
    // preparing a form
    b := new(bytes.Buffer)
    w := multipart.NewWriter(b)
    // adding image to form
    f, err := os.Open(cover)
    if err != nil {
        return err
    }
    defer f.Close()
    fw, err := w.CreateFormFile("image", filepath.Base(cover))
    if err != nil {
        return err
    }
    if _, err = io.Copy(fw, f); err != nil {
        return err
    }
    w.Close()
    // post image
    fmt.Printf("Uploading " + filepath.Base(cover) + " to Imgur... ")
    req, err := http.NewRequest("POST", "https://api.imgur.com/3/image", b)
    if err != nil {
        return err
    }
    req.Header.Set("Content-Type", w.FormDataContentType())
    req.Header.Add("Authorization", "Client-ID " + cfg.imgurClientID)
    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    // parse response
    if resp.StatusCode != http.StatusOK {
        return errors.New("Returned status: " + resp.Status)
    }
    // retrieve link from response
    var link string
    if err := scan.ScanJSON(resp.Body, "data/link", &link); err != nil {
        fmt.Println("KO")
        err = errors.New("Could not find image URL.")
    }
    t.info.ImageField = link
    fmt.Println("OK: " + t.info.ImageField)
    return nil
}

func (t *UploadCandidate) uploadCoverToWhatIMG(cover string, cfg Config) error {
    // login
    hc, data, err := login("https://whatimg.com/users.php?act=login-d", cfg.whatimgUser, cfg.whatimgPassword)
    if err != nil {
        return err
    }
    if !strings.Contains(data, "You have been successfully logged in.") {
        return errors.New(ErrorCouldNotLogIn)
    }
    // upload image and get link
    fmt.Printf("Uploading " + filepath.Base(cover) + " to WhatIMG... ")
    data, err = uploadImage(hc, "https://whatimg.com/upload.php", cover)
    r := regexp.MustCompile(".*value=\"(https://whatimg.com/i/[[:alnum:]]+?.jpg)\".*")
    if r.MatchString(data) {
        t.info.ImageField = r.FindStringSubmatch(data)[1]
        fmt.Println("OK: " + t.info.ImageField)
    } else {
        fmt.Println("KO")
        err = errors.New("Could not find image URL.")
    }
    return nil
}

func (t *UploadCandidate) generateTorrent(cfg Config) error {
    fmt.Println("Building torrent file " + filepath.Base(t.torrentFile))
    mi := metainfo.MetaInfo{Announce: cfg.bibPasskey}
    mi.Comment = "for bibliotik"
    mi.CreatedBy = BRUTE
    mi.CreationDate = time.Now().Unix()
    i := metainfo.Info{PieceLength: 256 * 1024, Private: newTrue()}
    if err := i.BuildFromFilePath(t.epubFile); err != nil {
        return err
    }
    bytes, err := bencode.Marshal(i)
    if err != nil {
        return err
    }
    mi.InfoBytes = bytes
    f, err := os.Create(t.torrentFile)
    if err != nil {
        return err
    }
    defer f.Close()
    return mi.Write(f)
}

func (t *UploadCandidate) loginBib(cfg Config) (*http.Client, error) {
    // login
    client, data, err := login(BibRoot, cfg.bibUser, cfg.bibPassword)
    if err != nil {
        return nil, errors.New(ErrorCouldNotLogIn)
    }
    // getting authkey
    r := regexp.MustCompile(".*authkey=([[:alnum:]]{40}).*")
    if r.MatchString(data) {
        t.info.authkey = r.FindStringSubmatch(data)[1]
    } else {
        return nil, errors.New("Could not find authkey: " + ErrorCouldNotLogIn)
    }
    return client, err
}

func (t *UploadCandidate) generateSearchRequest(searchURL string) (string, error) {
    u, err := url.Parse(searchURL)
    if err != nil {
        return "", err
    }
    q := u.Query()
    // split authors if there are more than one
    // otherwise bib will only return hits if the authors are in the same order in its database.
    authors := []string{}
    for _, a := range strings.Split(t.info.AuthorsField, ",") {
        authors = append(authors, `@authors "`+a+`"`)
    }
    q.Set("search", fmt.Sprintf(`%s @title "%s"`, strings.Join(authors, " "), t.info.TitleField))
    q.Set("for[]", EPUBFormat)
    u.RawQuery = q.Encode()
    return u.String(), nil
}

func (t *UploadCandidate) checkBib(client *http.Client) (bool, error) {
    foundExisting := false
    searchURL, err := t.generateSearchRequest(BibRoot + "/torrents/")
    if err != nil {
        return foundExisting, err
    }

    resp, err := client.Get(searchURL)
    if err != nil {
        return foundExisting, errors.New("Could not search for book")
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusOK {
        return foundExisting, errors.New("Returned status: " + resp.Status)
    }
    doc, err := goquery.NewDocumentFromResponse(resp)
    if err != nil {
        return foundExisting, err
    }
    // find torrents
    doc.Find(".torrent").Each(func(i int, s *goquery.Selection) {
        title := s.Find(".title a").Text()
        link, _ := s.Find(".title a").Attr("href")
        authors := []string{}
        s.Find(".authorLink").Each(func(i int, s *goquery.Selection) {
            authors = append(authors, s.Text())
        })
        publisher := s.Find(".publisherLink").Text()
        year := s.Find(".torYear").Text()
        retail := s.Find(".torRetail").Text()
        if retail == "" {
            retail = "[NON RETAIL]"
        }
        fmt.Printf("%d. %s %s %s [%s] %s\n\t%s\n", i+1, strings.Join(authors, ", "), year, title, publisher, retail, BibRoot+link)
        foundExisting = true
    })
    if foundExisting {
        fmt.Println()
    }
    return foundExisting, err
}

func (t *UploadCandidate) uploadTorrent(cfg Config, client *http.Client) error {
    // prepare request
    req, err := t.info.generateUploadRequest(BibRoot + "/upload/ebooks")
    if err != nil {
        return errors.New("Could not prepare upload form")
    }
    // submit the request
    resp, err := client.Do(req)
    if err != nil {
        return errors.New("Could not upload torrent")
    }
    defer resp.Body.Close()

    // check what URL the response came from
    finalURL := resp.Request.URL.String()
    if finalURL == BibRoot+"/upload/ebooks" {
        // response was from the upload form, meaning the fields contained errors and were rejected.
        return errors.New(ErrorUploadFailed)
    }
    return err
}

func (t *UploadCandidate) seed(cfg Config) (err error) {
    fmt.Println("Copying files to begin seeding.")
    err = h.CopyFile(t.epubFile, filepath.Join(cfg.torrentSeedDir, filepath.Base(t.epubFile)))
    if err != nil {
        return
    }
    return h.CopyFile(t.torrentFile, filepath.Join(cfg.torrentWatchDir, filepath.Base(t.torrentFile)))
}

func (t *UploadCandidate) archive(cfg Config) error {
    // generate archive name
    currentTime := time.Now().Local()
    currentDay := currentTime.Format("2006-01-02")
    archiveName := filepath.Join(cfg.archiveDir, fmt.Sprintf("%s - %s - %s (%s) %s.tar.gz", currentDay, t.info.IsbnField, t.info.AuthorsField, t.info.YearField, t.info.TitleField))
    if _, err := h.FileExists(archiveName); err == nil {
        return errors.New("Archive " + archiveName + " already exists")
    }
    fmt.Println("Backing up files to " + filepath.Base(archiveName))

    // generate archive
    tar := new(archivex.TarFile)
    if err := tar.Create(archiveName); err != nil {
        return errors.New("Error creating tar.")
    }
    if err := tar.AddFile(t.epubFile); err != nil {
        return fmt.Errorf(ErrorAddingToTar, t.epubFile)
    }
    if err := tar.AddFile(t.imageFile); err != nil {
        return fmt.Errorf(ErrorAddingToTar, t.imageFile)
    }
    if err := tar.AddFile(t.imageFileGR); err != nil {
        fmt.Println("Error adding GR cover, probably it could not be downloaded.")
    }
    if err := tar.AddFile(t.torrentFile); err != nil {
        return fmt.Errorf(ErrorAddingToTar, t.torrentFile)
    }
    if err := tar.AddFile(t.infoFile); err != nil {
        return fmt.Errorf(ErrorAddingToTar, t.infoFile)
    }
    if err := tar.Close(); err != nil {
        return errors.New("Error writing tar.")
    }
    return nil
}

func (t *UploadCandidate) cleanUp() error {
    if err := os.Remove(t.imageFile); err != nil {
        return errors.New("Could not remove " + t.imageFile)
    }
    if err := os.Remove(t.imageFileGR); err != nil {
        fmt.Println("Error removing GR cover, probably it could not be downloaded.")
    }
    if err := os.Remove(t.torrentFile); err != nil {
        return errors.New("Could not remove " + t.torrentFile)
    }
    if err := os.Remove(t.epubFile); err != nil {
        return errors.New("Could not remove " + t.epubFile)
    }
    if err := os.Remove(t.infoFile); err != nil {
        return errors.New("Could not remove " + t.infoFile)
    }
    return nil
}