SHOW:
|
|
- or go back to the newest paste.
1 | /* | |
2 | ||
3 | # brute | |
4 | ||
5 | BRUTE stands for: Bibliotik Retail Uploader Torrent Epubs. | |
6 | It does not mean anything, but BRUTEs are not known to be particularly | |
7 | articulate. | |
8 | ||
9 | In short, BRUTE allows uploading a retail epub to Bib from the comfort of the | |
10 | command line. | |
11 | ||
12 | It's not the prettiest thing, being hacked together and mashed up in a single | |
13 | file for ease of sharing, but BRUTE will try its best to help make quality | |
14 | uploads. | |
15 | This means it will automate the boring stuff, and let the user focus on making | |
16 | sure the torrent metadata is accurate and complete. | |
17 | ||
18 | BRUTE retrieves metadata from the epub and Goodreads automatically, and forces | |
19 | the user to merge and edit the results as needed. | |
20 | Everything is saved to a yaml file, for (optional) manual editing before the | |
21 | actual upload. | |
22 | ||
23 | ## Building | |
24 | ||
25 | You need a working Go installation: see https://golang.org/doc/install. | |
26 | Put this file in a 'brute' directory in your $GO_PATH, cd to it and just run: | |
27 | ||
28 | $ go get . | |
29 | $ go install ...brute | |
30 | ||
31 | or if you don't want to install it: | |
32 | ||
33 | $ go get . | |
34 | $ go run brute.go | |
35 | ||
36 | To update its dependancies, rebuild from time to time: | |
37 | ||
38 | $ go get -u . | |
39 | $ go install ...brute | |
40 | ||
41 | BRUTE was tested on Linux, but it should probably work wherever you can install | |
42 | Go. | |
43 | ||
44 | ## Requirements | |
45 | ||
46 | - a WhatIMG account | |
47 | - a GoodReads account for which you've requested an API Key: see | |
48 | https://www.goodreads.com/api/keys | |
49 | ||
50 | ## Usage | |
51 | ||
52 | BRUTE only requires one existing epub file as argument. | |
53 | ||
54 | - | $ brute /path/to/book.epub [/path/to/cover.jpg] |
54 | + | $ brute /path/to/book.epub |
55 | ||
56 | It will: | |
57 | ||
58 | - read epub metadata | |
59 | - retrieve information from Goodreads if ISBN is found | |
60 | - allow (alright, force) the user to edit the merged information to prepare | |
61 | the bibliotik upload form | |
62 | - save that information to /path/to/book.yaml for manual editing if deemed | |
63 | necessary | |
64 | - upload the cover to whatimg and retrieve its url | |
65 | ||
66 | At this point BRUTE will ask if you want to upload directly or not. | |
67 | If you say no, you will be able to manually edit the yaml file as you wish. | |
68 | Run BRUTE again with the same argument, and it will load the yaml file with your | |
69 | changes, and proceed to: | |
70 | ||
71 | - generate the .torrent from the epub with the user's passkey | |
72 | - upload the torrent with the cover and information collected in previous steps | |
73 | or loaded from a previously saved yaml file | |
74 | - copy the .torrent and .epub where you want to begin seeding | |
75 | - archive everything for later reference: epub, cover, torrent, and yaml in a | |
76 | tarball. | |
77 | - (optionally) remove the original files | |
78 | ||
79 | Conventions: | |
80 | - | - if the cover jpg is not given as argument, the script assumes |
80 | + | - the script assumes /path/to/book.jpg exists and is the cover for this epub. |
81 | - | /path/to/book.jpg exists and is the cover for this epub. |
81 | + | |
82 | ||
83 | ## Configuration | |
84 | ||
85 | BRUTE needs a configuration file ("config.yaml") with the following information: | |
86 | ||
87 | whatimg_login: xx | |
88 | whatimg_password: xx | |
89 | bibliotik_login: xx | |
90 | - | imgur_client_id: xx |
90 | + | |
91 | - | ptpimg_email: xx |
91 | + | |
92 | - | ptpimg_password: xx |
92 | + | |
93 | torrent_seed_directory: /home/user/downloads | |
94 | archive_directory: /home/user/uploads | |
95 | goodreads_api_key: xx | |
96 | tag_aliases: | |
97 | science-fiction: | |
98 | - sf | |
99 | - sci-fi | |
100 | ||
101 | tag_aliases allows automatic removal of duplicate tags (from duplicate shelves | |
102 | on Goodreads). | |
103 | This configuration file must be in the working directory. | |
104 | Lots of information in a clear text file, I know. | |
105 | - | You must configure at least whatimg, imgur or ptpimg to upload covers. |
105 | + | |
106 | - | If more than one are defined, BRUTE will ask you which to use. |
106 | + | |
107 | ## Things BRUTE isn't good at | |
108 | - | You can request an anonymous imgur client_id at: http://imgur.com/register/api_anon |
108 | + | |
109 | - | You need to head over PTP and look in the forums if you want to use PTPIMG. |
109 | + | |
110 | - | WhatIMG is gone. |
110 | + | |
111 | does not know about an ISBN | |
112 | - retrieving editors, contributors, translators, languages (defaults to english) | |
113 | - it will try to extract tags from Goodreads shelves, and clean out the most | |
114 | blatantly awful ones. ALWAYS EDIT TAGS MANUALLY. | |
115 | ||
116 | Fortunately, you can overcome these shortcomings by manually editing | |
117 | /path/to/book.yaml! | |
118 | No excuses! | |
119 | ||
120 | ## Things BRUTE is useless at | |
121 | ||
122 | BRUTE can't help you if you're dealing with anything other than epubs. | |
123 | ||
124 | You could use it to upload non-retail epubs, but then you MUST manually set the | |
125 | RetailField to "0" in the yaml file. | |
126 | ||
127 | ## Changelog | |
128 | ||
129 | v1.0.4: Updated to work with latest version of third party packages. | |
130 | v1.0.3: Updated to work with latest version of third party packages. | |
131 | v1.0.2: In description, title is in italics and author name in bold. | |
132 | v1.0.1: BRUTE should now correctly retrieve edition year instead of original | |
133 | publication year from GR. | |
134 | v1.0: original release. | |
135 | ||
136 | */ | |
137 | ||
138 | package main | |
139 | ||
140 | import ( | |
141 | - | v8: Updated to work with latest version of third party packages. |
141 | + | |
142 | - | v7: Added support for PTPIMG. Irrationally reluctant to remove WhatIMG support for now. |
142 | + | |
143 | - | v6: Added support for imgur (anonymous uploads) now that WhatIMG is down :( |
143 | + | |
144 | - | v5: Quick search for duplicates before uploading; code cleanups. |
144 | + | |
145 | - | v4: Menus have changed a little; updated to work with latest version of third |
145 | + | |
146 | - | party packages. |
146 | + | |
147 | - | v3: ask for ISBN if none found; better info if upload form submission fails and |
147 | + | |
148 | - | cleanup of generated torrent file; option to upload anonymously; a jpg file can |
148 | + | |
149 | - | be given as second parameter, to be used as a cover. |
149 | + | |
150 | - | v2: Updated to work with latest version of third party packages, added fetching |
150 | + | |
151 | - | cover from GR and asking the user which version to upload. |
151 | + | |
152 | - | v1.0.6: Updated to work with latest version of third party packages, and fixed |
152 | + | |
153 | - | bug with publishers containing ",". Added possibility to directly edit the yaml |
153 | + | |
154 | - | file, using $EDITOR (falling back to nano). When merging epub and GR metadata, |
154 | + | |
155 | - | it is safe to ignore the 'type' field. |
155 | + | |
156 | - | v1.0.5: Updated to work with latest version of third party packages. |
156 | + | |
157 | "github.com/anacrolix/torrent/metainfo" | |
158 | b "github.com/barsanuphe/endive/book" | |
159 | e "github.com/barsanuphe/endive/endive" | |
160 | u "github.com/barsanuphe/endive/ui" | |
161 | "github.com/jhoonb/archivex" | |
162 | "github.com/skratchdot/open-golang/open" | |
163 | "github.com/spf13/viper" | |
164 | "golang.org/x/net/publicsuffix" | |
165 | "gopkg.in/yaml.v2" | |
166 | ) | |
167 | ||
168 | func main() { | |
169 | var ui e.UserInterface | |
170 | ui = u.UI{} | |
171 | ||
172 | // 1. load configuration | |
173 | cfg := Config{} | |
174 | if err := cfg.load("config.yaml"); err != nil { | |
175 | fmt.Println("Error loading config.") | |
176 | return | |
177 | } | |
178 | if err := cfg.check(); err != nil { | |
179 | fmt.Println("Error checking config: " + err.Error()) | |
180 | return | |
181 | } | |
182 | // 2. check argument | |
183 | if len(os.Args) != 2 { | |
184 | fmt.Println("Please provide an epub filename.") | |
185 | - | "github.com/PuerkitoBio/goquery" |
185 | + | |
186 | - | "github.com/anacrolix/torrent/bencode" |
186 | + | |
187 | c := UploadCandidate{ | |
188 | epubFile: os.Args[1], | |
189 | torrentFile: strings.TrimSuffix(os.Args[1], ".epub") + ".torrent", | |
190 | - | u "github.com/barsanuphe/helpers/ui" |
190 | + | imageFile: strings.TrimSuffix(os.Args[1], ".epub") + ".jpg", |
191 | - | h "github.com/barsanuphe/helpers" |
191 | + | infoFile: strings.TrimSuffix(os.Args[1], ".epub") + ".yaml", |
192 | } | |
193 | if err := c.check(); err != nil { | |
194 | fmt.Println(err.Error()) | |
195 | return | |
196 | } | |
197 | - | "github.com/mattn/go-scan" |
197 | + | |
198 | - | "encoding/json" |
198 | + | |
199 | fmt.Println(err.Error()) | |
200 | return | |
201 | - | const ( |
201 | + | |
202 | - | BRUTE = "BRUTE" |
202 | + | |
203 | - | BibRoot = "https://bibliotik.me" |
203 | + | if c.info.ImageField == "" || ui.YesOrNo("Image URL found, upload again") { |
204 | - | ptpimgURL = "https://ptpimg.me/" |
204 | + | if err := c.uploadImage(cfg); err != nil { |
205 | fmt.Println(err.Error()) | |
206 | - | EPUBFormat = "15" |
206 | + | |
207 | - | EnglishLanguage = "1" |
207 | + | |
208 | } | |
209 | - | EPUBExtension = ".epub" |
209 | + | |
210 | - | JPGExtension = ".jpg" |
210 | + | |
211 | - | Usage = "Usage: brute book.epub [cover.jpg]" |
211 | + | |
212 | fmt.Println(err.Error()) | |
213 | - | UploadImageAgain = "Image URL found, upload again" |
213 | + | |
214 | - | UploadEPUBCover = "Upload epub cover" |
214 | + | ui.Title("You can now either manually edit " + filepath.Base(c.infoFile) + " and run this script again, or directly upload to bibliotik if you are satisfied the quality of the metadata is excellent.") |
215 | - | UploadAnonymously = "Do you want the upload to be anonymous" |
215 | + | if ui.YesOrNo("Upload to bibliotik") { |
216 | - | UseExistingYAML = "Info file already exists! Load" |
216 | + | // 6. generate torrent |
217 | - | AllInfoRetrievedLastChanceToEdit = "\nAll relevant information has been retrieved.\n" + |
217 | + | |
218 | - | "If you are satisfied everything is perfect, you can upload to bibliotik directly.\n" + |
218 | + | fmt.Println(err.Error()) |
219 | - | "If unsure, it is not too late to fix things manually.\n" |
219 | + | |
220 | - | FullManualEdit = "Do you want to manually edit the YAML file before uploading?" |
220 | + | |
221 | - | SearchingForDupes = "\nNow checking if the book you want to upload already exists on bibliotik.\n" + |
221 | + | // 7. upload to bibliotik |
222 | - | "If something is found, check the links to make sure you are not uploading a duplicate" + |
222 | + | if err := c.uploadTorrent(cfg); err != nil { |
223 | - | "(check the official rules about what is and what is not considered a dupe).\n" + |
223 | + | fmt.Println(err.Error()) |
224 | - | "Search uses exact author and title only, it may not find relevant books if your information " + |
224 | + | |
225 | - | "is different from existing books on bibliotik, or show different editions.\n" |
225 | + | |
226 | - | ConfirmNotADupe = "After checking, are you certain you can upload this book" |
226 | + | // 8. copy torrent to watch dir && epub to incoming |
227 | - | ConfirmUpload = "Upload to bibliotik" |
227 | + | |
228 | - | GRCoverDownloaded = "\nThe Goodreads cover was downloaded alongside the epub cover.\n" + |
228 | + | fmt.Println(err.Error()) |
229 | - | "Please take a moment to choose the best version.\n" |
229 | + | |
230 | } | |
231 | - | ErrorDirectoryDoesNotExist = "%s directory does not exist" |
231 | + | // 9. backup all relevant files to zip with date |
232 | - | ErrorCouldNotLogIn = "Could not login" |
232 | + | |
233 | - | ErrorUploadFailed = "Upload failed. Check the yaml file for incorrect or forbidden information and try again." |
233 | + | fmt.Println(err.Error()) |
234 | - | ErrorAddingToTar = "Error adding %s" |
234 | + | |
235 | } | |
236 | // 10. clean up | |
237 | if ui.YesOrNo("Remove original files") { | |
238 | - | var ui u.UserInterface |
238 | + | |
239 | - | var epubFilename, coverFilename string |
239 | + | fmt.Println(err.Error()) |
240 | - | ui = &u.UI{} |
240 | + | |
241 | } | |
242 | } | |
243 | } else if ui.YesOrNo("Launch this book's Goodreads page in your browser for reference?") { | |
244 | err = open.Start("https://www.goodreads.com/search?q=" + c.info.IsbnField) | |
245 | if err != nil { | |
246 | fmt.Println("Error: could not open goodreads page!") | |
247 | } | |
248 | } | |
249 | } | |
250 | ||
251 | //------------------------------------------------------------------------------ | |
252 | - | // 2. check arguments |
252 | + | |
253 | - | switch len(os.Args) { |
253 | + | |
254 | - | case 2: |
254 | + | |
255 | - | epubFilename = os.Args[1] |
255 | + | |
256 | - | coverFilename = strings.TrimSuffix(os.Args[1], EPUBExtension) + JPGExtension |
256 | + | |
257 | - | case 3: |
257 | + | |
258 | - | epubFilename = os.Args[1] |
258 | + | |
259 | - | coverFilename = os.Args[2] |
259 | + | |
260 | - | default: |
260 | + | |
261 | - | ui.Error("Incorrect input!") |
261 | + | |
262 | - | ui.Info(Usage) |
262 | + | |
263 | TagAliases map[string][]string | |
264 | } | |
265 | ||
266 | - | epubFile: epubFilename, |
266 | + | |
267 | - | torrentFile: strings.TrimSuffix(epubFilename, EPUBExtension) + ".torrent", |
267 | + | |
268 | - | imageFile: coverFilename, |
268 | + | |
269 | - | imageFileGR: strings.TrimSuffix(coverFilename, JPGExtension) + "_gr" + JPGExtension, |
269 | + | |
270 | - | infoFile: strings.TrimSuffix(epubFilename, EPUBExtension) + ".yaml", |
270 | + | |
271 | err = conf.ReadInConfig() | |
272 | if err != nil { | |
273 | return | |
274 | } | |
275 | c.bibUser = conf.GetString("bibliotik_login") | |
276 | c.bibPassword = conf.GetString("bibliotik_password") | |
277 | c.bibPasskey = conf.GetString("bibliotik_passkey") | |
278 | - | ui.Error(err.Error()) |
278 | + | |
279 | c.whatimgPassword = conf.GetString("whatimg_password") | |
280 | c.torrentSeedDir = conf.GetString("torrent_seed_directory") | |
281 | c.torrentWatchDir = conf.GetString("torrent_watch_directory") | |
282 | - | if c.info.ImageField == "" || ui.Accept(UploadImageAgain) { |
282 | + | |
283 | - | // retrieve GR cover |
283 | + | |
284 | - | coverToUpload := c.imageFile |
284 | + | |
285 | - | if err := c.downloadCover(); err != nil { |
285 | + | |
286 | - | ui.Error(err.Error()) |
286 | + | |
287 | - | ui.Info("Could not download cover from GR, uploading epub version.") |
287 | + | |
288 | - | } else { |
288 | + | func (c *Config) check() (err error) { |
289 | - | ui.Title(GRCoverDownloaded) |
289 | + | if !e.DirectoryExists(c.torrentWatchDir) { |
290 | - | if ui.Accept(UploadEPUBCover) { |
290 | + | return errors.New("Torrent Watch directory does not exist") |
291 | - | ui.Info("Uploading epub cover.") |
291 | + | |
292 | if !e.DirectoryExists(c.torrentSeedDir) { | |
293 | - | ui.Info("Uploading GR cover.") |
293 | + | return errors.New("Torrent download directory does not exist") |
294 | - | coverToUpload = c.imageFileGR |
294 | + | |
295 | if !e.DirectoryExists(c.archiveDir) { | |
296 | return errors.New("Archive directory does not exist") | |
297 | } | |
298 | - | whatimg, imgur, ptpimg := cfg.knownCoverHosts() |
298 | + | |
299 | - | var uploadErr error |
299 | + | |
300 | - | var uploaded bool |
300 | + | |
301 | - | if !uploaded && whatimg && ui.Accept("Upload cover with WhatIMG?") { |
301 | + | |
302 | - | uploadErr = c.uploadCoverToWhatIMG(coverToUpload, cfg) |
302 | + | |
303 | - | uploaded = (uploadErr == nil) |
303 | + | |
304 | // it is arguably awful. | |
305 | - | if !uploaded && ptpimg && ui.Accept("Upload cover with PTPIMG?") { |
305 | + | |
306 | - | uploadErr = c.uploadCoverToPTPIMG(coverToUpload, cfg) |
306 | + | |
307 | - | uploaded = (uploadErr == nil) |
307 | + | |
308 | } | |
309 | - | if !uploaded && imgur && ui.Accept("Upload cover with Imgur?") { |
309 | + | |
310 | - | uploadErr = c.uploadCoverToImgur(coverToUpload, cfg) |
310 | + | |
311 | - | uploaded = (uploadErr == nil) |
311 | + | |
312 | if err != nil { | |
313 | - | if uploadErr != nil { |
313 | + | |
314 | - | fmt.Println(uploadErr.Error()) |
314 | + | |
315 | } | |
316 | return nil | |
317 | } | |
318 | ||
319 | - | if ui.Accept(UploadAnonymously) { |
319 | + | |
320 | - | c.makeAnonymous() |
320 | + | |
321 | form.Add("username", username) | |
322 | form.Add("password", password) | |
323 | req, err := http.NewRequest("POST", siteUrl, strings.NewReader(form.Encode())) | |
324 | if err != nil { | |
325 | fmt.Println(err.Error()) | |
326 | - | // 6. manual edit |
326 | + | |
327 | - | ui.Title(AllInfoRetrievedLastChanceToEdit) |
327 | + | |
328 | - | if ui.Accept(FullManualEdit) { |
328 | + | |
329 | - | accepted := false |
329 | + | |
330 | - | for !accepted { |
330 | + | |
331 | - | if err := c.manualEdit(ui); err == nil { |
331 | + | |
332 | - | if ui.Accept("Confirm all fields are awesome") { |
332 | + | |
333 | - | accepted = true |
333 | + | |
334 | log.Fatal(err) | |
335 | } | |
336 | hc = &http.Client{Jar: jar} | |
337 | ||
338 | - | // 7. check bib for existing uploads |
338 | + | |
339 | - | client, err := c.loginBib(cfg) |
339 | + | |
340 | fmt.Println(err.Error()) | |
341 | - | ui.Error(err.Error()) |
341 | + | |
342 | defer resp.Body.Close() | |
343 | ||
344 | - | ui.Title(SearchingForDupes) |
344 | + | |
345 | - | foundExisting, err := c.checkBib(client) |
345 | + | |
346 | } | |
347 | - | ui.Error(err.Error()) |
347 | + | |
348 | data, err := ioutil.ReadAll(resp.Body) | |
349 | if err != nil { | |
350 | - | if foundExisting { |
350 | + | |
351 | - | if !ui.Accept(ConfirmNotADupe) { |
351 | + | |
352 | - | ui.Info("Stopping everything.") |
352 | + | |
353 | return | |
354 | } | |
355 | ||
356 | - | ui.Info("Nothing found.") |
356 | + | |
357 | // preparing a form | |
358 | b := new(bytes.Buffer) | |
359 | - | if ui.Accept(ConfirmUpload) { |
359 | + | |
360 | - | // 8. generate torrent |
360 | + | |
361 | f, err := os.Open(image) | |
362 | - | ui.Error(err.Error()) |
362 | + | |
363 | return "", err | |
364 | } | |
365 | - | // 9. upload to bibliotik |
365 | + | |
366 | - | if err := c.uploadTorrent(cfg, client); err != nil { |
366 | + | |
367 | - | ui.Error(err.Error()) |
367 | + | |
368 | - | if err := os.Remove(c.torrentFile); err != nil { |
368 | + | |
369 | - | ui.Error("Error cleaning torrent file") |
369 | + | |
370 | if _, err = io.Copy(fw, f); err != nil { | |
371 | return | |
372 | } | |
373 | - | // 10. copy torrent to watch dir && epub to incoming |
373 | + | |
374 | return | |
375 | - | ui.Error(err.Error()) |
375 | + | |
376 | if err = w.WriteField("private_upload", "1"); err != nil { | |
377 | return | |
378 | - | // 11. backup all relevant files to zip with date |
378 | + | |
379 | if err = w.WriteField("upload_type", "standard"); err != nil { | |
380 | - | ui.Error(err.Error()) |
380 | + | |
381 | } | |
382 | w.Close() | |
383 | - | // 12. clean up |
383 | + | |
384 | - | if ui.Accept("Remove original files") { |
384 | + | |
385 | if err != nil { | |
386 | - | ui.Error(err.Error()) |
386 | + | |
387 | } | |
388 | req.Header.Set("Content-Type", w.FormDataContentType()) | |
389 | ||
390 | - | } else if ui.Accept("Launch this book's Goodreads page in your browser for reference?") { |
390 | + | |
391 | - | if err := open.Start("https://www.goodreads.com/search?q=" + c.info.IsbnField); err != nil { |
391 | + | |
392 | - | ui.Error("Could not open goodreads page!") |
392 | + | |
393 | } | |
394 | defer resp.Body.Close() | |
395 | ||
396 | if resp.StatusCode != http.StatusOK { | |
397 | err = errors.New("Returned status: " + resp.Status) | |
398 | } | |
399 | ||
400 | data, err := ioutil.ReadAll(resp.Body) | |
401 | if err != nil { | |
402 | return "", err | |
403 | } | |
404 | responseData = string(data) | |
405 | - | ptpimgEmail string |
405 | + | |
406 | - | ptpimgPassword string |
406 | + | |
407 | - | imgurClientID string |
407 | + | |
408 | //------------------------------------------------------------------------------ | |
409 | ||
410 | // BUInfo tracks all of the upload form fields. | |
411 | type BUInfo struct { | |
412 | TorrentFileField string | |
413 | authkey string | |
414 | upload string // default: empty | |
415 | TitleField string | |
416 | EditorsField string | |
417 | ContributorsField string | |
418 | TranslatorsField string | |
419 | PublishersField string | |
420 | PagesField string | |
421 | AuthorsField string | |
422 | FormatField string // EPUB == "15" | |
423 | IsbnField string | |
424 | TagsField string | |
425 | DescriptionField string | |
426 | RetailField string // retail == "1" | |
427 | NotifyField string // default "1" | |
428 | LanguageField string // english == "1" | |
429 | - | c.ptpimgEmail = conf.GetString("ptpimg_email") |
429 | + | |
430 | - | c.ptpimgPassword = conf.GetString("ptpimg_password") |
430 | + | |
431 | - | c.imgurClientID = conf.GetString("imgur_client_id") |
431 | + | |
432 | ||
433 | // ShowInfo returns a table with relevant information about a book. | |
434 | func (i *BUInfo) ShowInfo() (desc string) { | |
435 | var rows [][]string | |
436 | rows = append(rows, []string{"TorrentFile", i.TorrentFileField}) | |
437 | rows = append(rows, []string{"Title", i.TitleField}) | |
438 | rows = append(rows, []string{"Authors", i.AuthorsField}) | |
439 | rows = append(rows, []string{"Editors", i.EditorsField}) | |
440 | - | func (c *Config) check() error { |
440 | + | |
441 | - | if !h.DirectoryExists(c.torrentWatchDir) { |
441 | + | |
442 | - | return fmt.Errorf(ErrorDirectoryDoesNotExist, "Torrent watch") |
442 | + | |
443 | rows = append(rows, []string{"Isbn", i.IsbnField}) | |
444 | - | if !h.DirectoryExists(c.torrentSeedDir) { |
444 | + | |
445 | - | return fmt.Errorf(ErrorDirectoryDoesNotExist, "Torrent download") |
445 | + | |
446 | rows = append(rows, []string{"Format", i.FormatField}) | |
447 | - | if !h.DirectoryExists(c.archiveDir) { |
447 | + | |
448 | - | return fmt.Errorf(ErrorDirectoryDoesNotExist, "Archive") |
448 | + | |
449 | rows = append(rows, []string{"Tags", i.TagsField}) | |
450 | - | if whatimg, imgur, ptpimg := c.knownCoverHosts(); !whatimg && !imgur && !ptpimg { |
450 | + | |
451 | - | return errors.New("Image hosting service must be configured to upload covers") |
451 | + | |
452 | rows = append(rows, []string{"Notify", i.NotifyField}) | |
453 | return e.TabulateRows(rows, "Info", "Book") | |
454 | } | |
455 | ||
456 | - | func (c *Config) knownCoverHosts() (whatimg, imgur, ptpimg bool) { |
456 | + | |
457 | - | if c.whatimgPassword != "" && c.whatimgUser != "" { |
457 | + | |
458 | - | whatimg = true |
458 | + | |
459 | return | |
460 | - | if c.ptpimgPassword != "" && c.ptpimgEmail != "" { |
460 | + | |
461 | - | ptpimg = true |
461 | + | |
462 | } | |
463 | - | if c.imgurClientID != "" { |
463 | + | |
464 | - | imgur = true |
464 | + | |
465 | d, err := yaml.Marshal(i) | |
466 | if err != nil { | |
467 | return err | |
468 | } | |
469 | return ioutil.WriteFile(path, d, 0777) | |
470 | } | |
471 | ||
472 | func (i *BUInfo) fill(m b.Metadata, torrentPath string) { | |
473 | i.TorrentFileField = torrentPath | |
474 | i.authkey = "" | |
475 | i.upload = "" | |
476 | if m.HasAny() { | |
477 | seriesInfo := "" | |
478 | if m.Series.HasAny() { | |
479 | for i := range m.Series { | |
480 | seriesInfo += fmt.Sprintf(" (%s, Book %s)", m.Series[i].Name, m.Series[i].Position) | |
481 | } | |
482 | } | |
483 | i.TitleField = m.Title() + seriesInfo | |
484 | i.EditorsField = "" // TODO | |
485 | i.ContributorsField = "" // TODO | |
486 | i.TranslatorsField = "" // TODO | |
487 | i.PublishersField = m.Publisher | |
488 | i.PagesField = m.NumPages | |
489 | i.AuthorsField = strings.Join(m.Authors, ", ") | |
490 | i.FormatField = "15" // EPUB | |
491 | i.IsbnField = m.ISBN | |
492 | i.TagsField = m.Category + ", " + m.MainGenre + ", " + m.Tags.String() | |
493 | ||
494 | r := strings.NewReplacer( | |
495 | i.AuthorsField, "[b]"+i.AuthorsField+"[/b]", | |
496 | m.Title(), "[i]"+m.Title()+"[/i]", | |
497 | ) | |
498 | i.DescriptionField = r.Replace(m.Description) | |
499 | i.RetailField = "1" | |
500 | i.NotifyField = "1" | |
501 | i.LanguageField = "1" // English | |
502 | i.YearField = m.EditionYear | |
503 | } | |
504 | } | |
505 | ||
506 | func (i *BUInfo) generateUploadRequest(uploadURL string) (req *http.Request, err error) { | |
507 | // setting up the form | |
508 | b := new(bytes.Buffer) | |
509 | w := multipart.NewWriter(b) | |
510 | // adding the torrent file | |
511 | f, err := os.Open(i.TorrentFileField) | |
512 | if err != nil { | |
513 | return nil, err | |
514 | } | |
515 | defer f.Close() | |
516 | fw, err := w.CreateFormFile("TorrentFileField", filepath.Base(i.TorrentFileField)) | |
517 | if err != nil { | |
518 | return nil, err | |
519 | } | |
520 | if _, err = io.Copy(fw, f); err != nil { | |
521 | return | |
522 | } | |
523 | errors := []error{} | |
524 | errors = append(errors, w.WriteField("authkey", i.authkey)) | |
525 | errors = append(errors, w.WriteField("upload", i.upload)) | |
526 | errors = append(errors, w.WriteField("TitleField", i.TitleField)) | |
527 | errors = append(errors, w.WriteField("EditorsField", i.EditorsField)) | |
528 | errors = append(errors, w.WriteField("ContributorsField", i.ContributorsField)) | |
529 | errors = append(errors, w.WriteField("TranslatorsField", i.TranslatorsField)) | |
530 | errors = append(errors, w.WriteField("PublishersField", i.PublishersField)) | |
531 | errors = append(errors, w.WriteField("PagesField", i.PagesField)) | |
532 | errors = append(errors, w.WriteField("AuthorsField", i.AuthorsField)) | |
533 | errors = append(errors, w.WriteField("FormatField", i.FormatField)) | |
534 | errors = append(errors, w.WriteField("IsbnField", i.IsbnField)) | |
535 | errors = append(errors, w.WriteField("TagsField", i.TagsField)) | |
536 | errors = append(errors, w.WriteField("DescriptionField", i.DescriptionField)) | |
537 | errors = append(errors, w.WriteField("RetailField", i.RetailField)) | |
538 | errors = append(errors, w.WriteField("NotifyField", i.NotifyField)) | |
539 | errors = append(errors, w.WriteField("LanguageField", i.LanguageField)) | |
540 | errors = append(errors, w.WriteField("YearField", i.YearField)) | |
541 | errors = append(errors, w.WriteField("ImageField", i.ImageField)) | |
542 | if err := checkErrors(errors...); err != nil { | |
543 | return nil, err | |
544 | } | |
545 | w.Close() | |
546 | ||
547 | req, err = http.NewRequest("POST", uploadURL, b) | |
548 | if err != nil { | |
549 | return nil, err | |
550 | } | |
551 | req.Header.Set("Content-Type", w.FormDataContentType()) | |
552 | return | |
553 | } | |
554 | ||
555 | //------------------------------------------------------------------------------ | |
556 | ||
557 | type UploadCandidate struct { | |
558 | epubFile string | |
559 | torrentFile string | |
560 | imageFile string | |
561 | infoFile string | |
562 | info BUInfo | |
563 | } | |
564 | ||
565 | func (t *UploadCandidate) loadInfo() (err error) { | |
566 | return t.info.load(t.infoFile) | |
567 | } | |
568 | ||
569 | func (t *UploadCandidate) saveInfo() (err error) { | |
570 | fmt.Println("Saving all gathered information to " + t.infoFile) | |
571 | return t.info.save(t.infoFile) | |
572 | } | |
573 | ||
574 | func (t *UploadCandidate) check() (err error) { | |
575 | // assert epub exists | |
576 | - | func loginPTPImg(siteUrl, username, password string) (hc *http.Client, returnData string, err error) { |
576 | + | if _, err := e.FileExists(t.epubFile); err != nil { |
577 | return errors.New("Epub does not exist!") | |
578 | - | form.Add("email", username) |
578 | + | |
579 | - | form.Add("pass", password) |
579 | + | |
580 | if !strings.HasSuffix(strings.ToLower(t.epubFile), ".epub") { | |
581 | return errors.New(t.epubFile + " is not an epub") | |
582 | } | |
583 | // assert jpg exists | |
584 | if _, err := e.FileExists(t.imageFile); err != nil { | |
585 | return errors.New("Cover " + t.imageFile + " does not exist!") | |
586 | } | |
587 | // assert t does not exist yet | |
588 | if _, err := e.FileExists(t.torrentFile); err == nil { | |
589 | return errors.New("Torrent " + t.torrentFile + " should not exist!") | |
590 | } | |
591 | return | |
592 | } | |
593 | ||
594 | func (t *UploadCandidate) getInfo(cfg Config, ui e.UserInterface) (err error) { | |
595 | loadedFromFile := false | |
596 | if _, err := e.FileExists(t.infoFile); err == nil { | |
597 | if ui.YesOrNo("Info file already exists! Load") { | |
598 | if err := t.loadInfo(); err != nil { | |
599 | return err | |
600 | } | |
601 | fmt.Println(t.info.ShowInfo()) | |
602 | if !ui.YesOrNo("Confirm") { | |
603 | return errors.New("Incorrect book information rejected.") | |
604 | } else { | |
605 | loadedFromFile = true | |
606 | } | |
607 | } | |
608 | } | |
609 | if !loadedFromFile { | |
610 | bk := b.NewBook(ui, 0, t.epubFile, e.Config{GoodReadsAPIKey: cfg.grApiKey, TagAliases: cfg.TagAliases}, true) | |
611 | // read epub metadata | |
612 | bk.Metadata, err = bk.RetailEpub.ReadMetadata() | |
613 | - | func uploadImageToPTPIMG(client *http.Client, uploadUrl, image, apiKey string) (link string, err error) { |
613 | + | |
614 | return err | |
615 | } | |
616 | // find GoodReads metadata and merge | |
617 | err = bk.SearchOnline() | |
618 | if err != nil { | |
619 | return err | |
620 | } | |
621 | t.info.fill(bk.Metadata, t.torrentFile) | |
622 | fmt.Println(t.info.ShowInfo()) | |
623 | - | fw, err := w.CreateFormFile("file-upload[]", filepath.Base(image)) |
623 | + | |
624 | for !ui.YesOrNo("Confirm") { | |
625 | relevantFields := []string{"series", "title", "author", "year", "publisher", "description", "isbn", "category", "maingenre", "tags"} | |
626 | for _, f := range relevantFields { | |
627 | if err := bk.EditField(f); err != nil { | |
628 | fmt.Println("Could not assign new value to field " + f + ", continuing.") | |
629 | } | |
630 | - | if err = w.WriteField("api_key", apiKey); err != nil { |
630 | + | |
631 | t.info.fill(bk.Metadata, t.torrentFile) | |
632 | fmt.Println(t.info.ShowInfo()) | |
633 | } | |
634 | } | |
635 | return | |
636 | } | |
637 | ||
638 | func (t *UploadCandidate) uploadImage(cfg Config) error { | |
639 | // login | |
640 | hc, data, err := login("https://whatimg.com/users.php?act=login-d", cfg.whatimgUser, cfg.whatimgPassword) | |
641 | if err != nil { | |
642 | return err | |
643 | } | |
644 | if !strings.Contains(data, "You have been successfully logged in.") { | |
645 | return errors.New("Failed to log in") | |
646 | } | |
647 | // upload image and get link | |
648 | fmt.Printf("Uploading " + filepath.Base(t.imageFile) + " to WhatIMG... ") | |
649 | data, err = uploadImage(hc, "https://whatimg.com/upload.php", t.imageFile) | |
650 | r := regexp.MustCompile(".*value=\"(https://whatimg.com/i/[[:alnum:]]+?.jpg)\".*") | |
651 | if r.MatchString(data) { | |
652 | t.info.ImageField = r.FindStringSubmatch(data)[1] | |
653 | fmt.Println("OK: " + t.info.ImageField) | |
654 | } else { | |
655 | - | ptpimgJSON := []map[string]string{} |
655 | + | |
656 | - | if err := json.Unmarshal(data, &ptpimgJSON); err != nil { |
656 | + | |
657 | } | |
658 | return nil | |
659 | - | link = fmt.Sprintf("%s%s.%s", ptpimgURL, ptpimgJSON[0]["code"], ptpimgJSON[0]["ext"]) |
659 | + | |
660 | ||
661 | func (t *UploadCandidate) generateTorrent(cfg Config) error { | |
662 | fmt.Println("Building torrent file " + filepath.Base(t.torrentFile)) | |
663 | mi := &metainfo.MetaInfo{Announce: cfg.bibPasskey} | |
664 | mi.Comment = "for bibliotik" | |
665 | mi.CreatedBy = "BRUTE" | |
666 | mi.CreationDate = time.Now().Unix() | |
667 | mi.Info.PieceLength = 256 * 1024 | |
668 | mi.Info.BuildFromFilePath(t.epubFile) | |
669 | mi.Info.Private = newTrue() | |
670 | ||
671 | f, err := os.Create(t.torrentFile) | |
672 | if err != nil { | |
673 | return err | |
674 | } | |
675 | defer f.Close() | |
676 | return mi.Write(f) | |
677 | } | |
678 | ||
679 | func (t *UploadCandidate) uploadTorrent(cfg Config) (err error) { | |
680 | // login | |
681 | client, data, err := login("https://bibliotik.me", cfg.bibUser, cfg.bibPassword) | |
682 | if err != nil { | |
683 | fmt.Println(err.Error()) | |
684 | return err | |
685 | } | |
686 | // getting authkey | |
687 | - | AnonymousField string // default "0" |
687 | + | |
688 | if r.MatchString(data) { | |
689 | t.info.authkey = r.FindStringSubmatch(data)[1] | |
690 | } else { | |
691 | - | func (i *BUInfo) ShowInfo() string { |
691 | + | return errors.New("Could not log in.") |
692 | } | |
693 | // prepare request | |
694 | req, err := t.info.generateUploadRequest("https://bibliotik.me/upload/ebooks") | |
695 | if err != nil { | |
696 | return err | |
697 | } | |
698 | // submit the request | |
699 | resp, err := client.Do(req) | |
700 | if err != nil { | |
701 | return err | |
702 | } | |
703 | defer resp.Body.Close() | |
704 | ||
705 | // check what URL the response came from | |
706 | finalURL := resp.Request.URL.String() | |
707 | if finalURL == "https://bibliotik.me/upload/ebooks" { | |
708 | return errors.New("Upload probably failed, response was from upload form.") | |
709 | } | |
710 | - | rows = append(rows, []string{"Anonymous", i.AnonymousField}) |
710 | + | |
711 | } | |
712 | ||
713 | func (t *UploadCandidate) seed(cfg Config) (err error) { | |
714 | fmt.Println("Copying files to begin seeding.") | |
715 | err = e.CopyFile(t.epubFile, filepath.Join(cfg.torrentSeedDir, filepath.Base(t.epubFile))) | |
716 | if err != nil { | |
717 | return | |
718 | } | |
719 | err = e.CopyFile(t.torrentFile, filepath.Join(cfg.torrentWatchDir, filepath.Base(t.torrentFile))) | |
720 | if err != nil { | |
721 | return | |
722 | } | |
723 | return | |
724 | } | |
725 | ||
726 | func (t *UploadCandidate) archive(cfg Config) (err error) { | |
727 | // generate archive name | |
728 | currentTime := time.Now().Local() | |
729 | currentDay := currentTime.Format("2006-01-02") | |
730 | archiveName := filepath.Join(cfg.archiveDir, fmt.Sprintf("%s - %s - %s (%s) %s.tar.gz", currentDay, t.info.IsbnField, t.info.AuthorsField, t.info.YearField, t.info.TitleField)) | |
731 | if _, err := e.FileExists(archiveName); err == nil { | |
732 | return errors.New("Archive " + archiveName + " already exists") | |
733 | } | |
734 | fmt.Println("Backing up files to " + filepath.Base(archiveName)) | |
735 | ||
736 | // generate archive | |
737 | tar := new(archivex.TarFile) | |
738 | if err := tar.Create(archiveName); err != nil { | |
739 | return errors.New("Error creating tar.") | |
740 | } | |
741 | if err := tar.AddFile(t.epubFile); err != nil { | |
742 | return errors.New("Error adding epub " + t.epubFile) | |
743 | } | |
744 | if err := tar.AddFile(t.imageFile); err != nil { | |
745 | - | // some publishers in the form 'X, Inc' make the upload form angry |
745 | + | return errors.New("Error adding image " + t.imageFile) |
746 | - | i.PublishersField = strings.Replace(m.Publisher, ",", " ", -1) |
746 | + | |
747 | if err := tar.AddFile(t.torrentFile); err != nil { | |
748 | return errors.New("Error adding torrent " + t.torrentFile) | |
749 | - | i.FormatField = EPUBFormat |
749 | + | |
750 | if err := tar.AddFile(t.infoFile); err != nil { | |
751 | - | i.TagsField = m.Category + ", " + m.Tags.String() |
751 | + | return errors.New("Error adding yaml " + t.infoFile) |
752 | } | |
753 | if err := tar.Close(); err != nil { | |
754 | return errors.New("Error writing tar.") | |
755 | } | |
756 | return | |
757 | } | |
758 | ||
759 | func (t *UploadCandidate) cleanUp() (err error) { | |
760 | - | i.AnonymousField = "0" |
760 | + | |
761 | - | i.LanguageField = EnglishLanguage |
761 | + | |
762 | } | |
763 | if err := os.Remove(t.torrentFile); err != nil { | |
764 | return errors.New("Could not remove " + t.torrentFile) | |
765 | } | |
766 | if err := os.Remove(t.epubFile); err != nil { | |
767 | return errors.New("Could not remove " + t.epubFile) | |
768 | } | |
769 | if err := os.Remove(t.infoFile); err != nil { | |
770 | return errors.New("Could not remove " + t.infoFile) | |
771 | } | |
772 | return | |
773 | } |