API tools faq
paste
Advertisement
Guest User

get_iplayer Download Olympics In Chunks Guide

a guest
Aug 7th, 2012
726
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 11.57 KB | None | 0 0
raw download clone embed print report
  1.  
  2. get_iplayer Download Olympics In Chunks Guide
  3. ---------------------------------------------
  4.  
  5. This is for Linux users, other OSes may vary.
  6.  
  7. Linux software used: get_iplayer, VLC, ffmpeg, and avidemux.
  8.  
  9.  
  10. This tutorial is how to download Olympic event sessions in chunks using get_iplayer, and then to do a rough and ready edit (not slick with nice transitions or anything), and to end up with a well-formed .mp4 file. It is not easy and it is time consuming, but worked well for me.
  11.  
  12.  
  13. Step 1:
  14.  
  15. Download the get_iplayer Olympics fix version from this page:
  16.  
  17. http://git.infradead.org/get_iplayer.git
  18.  
  19. "Allow Olympics live streams to work using new mediaselector/5 URLs"
  20.  
  21. Click on 'snapshot', save to disk, then extract the archive.
  22.  
  23. Either replace your existing get_iplayer with this version, or always run this version by using the full path to the new get_iplayer script, or from within its own directory using ./get_iplayer notation.
  24.  
  25. e.g.
  26.  
  27. Using './' notation:
  28.  
  29. cd /home/user/Scripts/Get-IPlayer/get_iplayer-c3aafce/
  30. ./get_iplayer "http://www.bbc.co.uk/sport/olympics/2012/live-video/p00w316y" etc
  31.  
  32. Using full path:
  33.  
  34. /home/user/Scripts/Get-IPlayer/get_iplayer-c3aafce/get_iplayer "http://www.bbc.co.uk/..." etc
  35.  
  36.  
  37. Step 2:
  38.  
  39. Get your event's video link from:
  40.  
  41. http://www.bbc.co.uk/sport/olympics/2012/live-video
  42. http://www.bbc.co.uk/sport/olympics/2012/schedule-results
  43.  
  44. The bottom link above has the full Olympics schedule, click on the event/day and select "Catch Up" to get to the video page.
  45.  
  46. e.g.
  47.  
  48. Men's Individual Foil heats and quarter-finals
  49. http://www.bbc.co.uk/sport/olympics/2012/live-video/p00w2xsr
  50.  
  51. By the way, wait until the event's session has actually finished before starting the chunk download.
  52.  
  53.  
  54. Step 3:
  55.  
  56. If the session you are downloading is long, the chances are you will need to download in chunks. There is a 4GB video file size limit (not due to a file system limitation but to 32 bit memory addressing limit). However with the software versions I've got, I rarely got as far as 4GB with each chunk, typically my downloads were of 1.5 to 2.5 hours in video length before RTMPDump crashed - but that's ok the files left after the crash work fine. Sometimes the crash happened sooner, when the downloaded chunk was less than an hour I just deleted the file and started that chunk again. NOTE: 90 mins of BBC Olympics 1280x720 video is a bit over 2GB of file size.
  57.  
  58. Keep notes about your downloaded chunks and ALWAYS REMEMBER to rename your files or get_iplayer will try to resume them which will not work (at least didn't for me), and unless you rename them you'll lose track of what you've got and which is which.
  59.  
  60. There will be lots of time calculation needed, the Time Calculator on this page was useful: http://www.grun1.com/utils/timeCalc.html
  61.  
  62.  
  63. Download the first chunk, no start time necessary:
  64.  
  65. ./get_iplayer "http://www.bbc.co.uk/sport/olympics/2012/live-video/p00w2xsr" --force -o "/big/storage/location/"
  66.  
  67. Download stopped at 5260 secs [Find this in the terminal, you may need to scroll up above errors.]
  68.  
  69. 5260 = 87 mins 40 secs = 01:27:40
  70.  
  71. Rename the file: Mens_Team_Foil_final_00.00.00-01.27.40.mp4.flv
  72.  
  73. Check the video. I used VLC and skipped through it using <CTRL><RIGHT_ARROW> / <SHIFT><RIGHT_ARROW> / <ALT><RIGHT_ARROW> to jump quickly through the video, VLC allows you to manually set both the jump size lengths and the keys to use for them in this settings location:
  74.  
  75. VLC Prefs --> Show All --> Interface --> Hotkeys Setting
  76.  
  77.  
  78. Download the second chunk, start time is now necessary:
  79.  
  80. The first chunk stopped at 01:27:40. You can start the next chunk with a few mins overlap, say at 01:25:00 (stick to whole minutes and the calculations will be easier). However this ignores the editing process - it may be that between 01:25:00 and 01:27:40 there is continuous sporting action and no convenient join/edit point for later. My technique was to look in the video for a specific join point, a 5-15 min break, or some kind of pause in the action. Let's say I found a convenient 5 min break while viewing in VLC at 01:20:25, so I'll start the next chunk at 01:20:00 (believe me whole mins will make your life easier).
  81.  
  82. ./get_iplayer "http://www.bbc.co.uk/sport/olympics/2012/live-video/p00w2xsr" --force --start "01:20:00" -o "/big/storage/location/"
  83.  
  84. Download stopped at 12285 secs. This is ACCUMULATIVE, it includes the start time of 01:20:00.
  85.  
  86. Calculate length of chunk:
  87.  
  88. Started at:
  89. 01:20:00 = 80 mins
  90. 80 * 60 = 4800 secs
  91.  
  92. Duration of chunk is: Stop Pos - Start Pos
  93. Duration of chunk is: 12285 - 4800 = 7485 seconds
  94.  
  95. 7485 / 60 = 124.75
  96. 7485 mod 60 = 45
  97. 124 mins 45 secs = 02:04:45
  98.  
  99. Use VLC to check that the chunk is 02:04:45 in length.
  100.  
  101. Chunk finish time in relation to the overall video is therefore:
  102. Chunk Start At + Chunk Duration
  103. 01:20:00 + 02:04:45 = 03:24:45
  104.  
  105. Rename the file: Mens_Team_Foil_final_01.20.00-03.24.45.mp4.flv
  106.  
  107.  
  108. Download the third chunk and subsequent chunks...:
  109. [DO NOT SKIP READING THIS BIT - IT IS NOT QUITE THE SAME AS DOWNLOADING THE 2ND CHUNK.]
  110.  
  111. Now you need to find a convenient start point again, so view the 2nd chunk. VLC will specify the time from 00:00:00, when in terms of the overall you're actually starting at 01.20.00. Let's say you find a good start-the-next-chunk start point at approx. 10 mins from the actual end of the 2nd chunk, at 01:55:00 (as reported by VLC).
  112.  
  113. Start next chunk at:
  114. Pos Start of 2nd Chunk + Pos into 2nd Chunk
  115. 01.20.00 + 01:55:00 = 3:15:00
  116.  
  117. ./get_iplayer "http://www.bbc.co.uk/sport/olympics/2012/live-video/p00w2xsr" --force --start "03:15:00" -o "/big/storage/location/"
  118.  
  119. Download stopped at...
  120.  
  121. Repeat this process till all the chunks are done. I deleted any chunks that downloaded less than an hour of video and started them again on the command line (but you must remember to delete the small chunks or get_iplayer will try appending to the chunk which won't work). Sometimes I had to restart a chunk several times.
  122.  
  123.  
  124. Step 4:
  125.  
  126. Edit and remux - this will fix the video timecodes and give you nice .mp4 files.
  127.  
  128. Now you need to view your chunks and work out the points at which you want each chunk to start and stop, so that they will all fit nicely together - I also used this process to edit out long breaks in the action, this meant I sometimes used the same chunk to create 2 different new files (clearly with different start and end points). By doing so I turned 9 hours of video into 6.5 hours of actual action.
  129.  
  130. I created a table of what I needed to do in order to edit out the breaks and remove overlaps in the chunks. The start and end times are from the actual times shown in VLC of each chunk, the overall time is now no longer an issue.
  131.  
  132. ffmpeg needs a start point and then the duration (not the end point), so I used a Time Difference Calculator to get the duration. Put the end pos in the top text box and the start pos in the bottom text box: http://www.grun1.com/utils/timeDiff.cfm
  133.  
  134. NOTE: When creating your own table, note that ffmpeg is not perfectly accurate to the exact second, for instance the 2nd entry in the table below resulted in a file output by ffmpeg which was 00:57:34 in duration and not 00:57:35, 1 second went AWOL. So choose your edit points wisely, start a second or 2 early, finish a second or 2 late. I was only doing a rough 'just get it all without the breaks and overlaps' edit.
  135.  
  136. Fencing - Men's Indiv. Foil 1st Session (I also had to do the 2nd session):
  137.  
  138. Note the multiple use of Chunk_1 and Chunk_2 to remove long breaks.
  139.  
  140. ------------------------------------------------------------
  141. DL Chunk Start End Duration Output
  142. ------------------------------------------------------------
  143. Chunk_1 00:00:00 00:40:00 00:40:00 Output_1.mp4
  144. Chunk_1 01:21:00 02:18:35 00:57:35 Output_2.mp4
  145. Chunk_2 00:11:00 00:29:00 00:18:00 Output_3.mp4
  146. Chunk_2 00:37:00 00:53:00 00:16:00 Output_4.mp4
  147. Chunk_3 00:17:30 01:32:30 01:15:00 Output_5.mp4
  148. Chunk_4 00:16:30 00:52:30 00:36:00 Output_6.mp4
  149. ------------------------------------------------------------
  150. Total duration of all output files: 4:02:35
  151. ------------------------------------------------------------
  152.  
  153. The basic ffmpeg command is as follows - PARAMETER ORDER IS IMPORTANT:
  154.  
  155. ffmpeg -i INFILE.mp4.flv -ss StartAt -t Duration -vcodec copy -acodec copy Output_1.mp4
  156.  
  157. -i = input file
  158. -ss = start pos time
  159. -t = duration (not end pos time)
  160.  
  161. For -ss and -t you can use seconds or "hh:mm:ss". e.g. these 2 command lines below do the same thing, they start 10 mins from the beginning and provide a video of the next 1 hour, put another way they do 00:10:00 to 01:10:00.
  162.  
  163. ffmpeg -i infile.mp4.flv -ss 600 -t 3600 -vcodec copy -acodec copy Output_1.mp4
  164. ffmpeg -i infile.mp4.flv -ss "00:10:00" -t "01:00:00" -vcodec copy -acodec copy Output_1.mp4
  165.  
  166. The above table translated into these commands:
  167.  
  168. e.g.
  169. ffmpeg -i DL_Chunk_01.mp4.flv -ss "START_AT" -t "DURATION" -vcodec copy -acodec copy Output.mp4
  170.  
  171. I actually did (though I've renamed the input file names below for clarity):
  172.  
  173. ffmpeg -i DL_Chunk_01.mp4.flv -ss "00:00:00" -t "00:40:00" -vcodec copy -acodec copy Output_1.mp4
  174. ffmpeg -i DL_Chunk_01.mp4.flv -ss "01:21:00" -t "00:57:35" -vcodec copy -acodec copy Output_2.mp4
  175. ffmpeg -i DL_Chunk_02.mp4.flv -ss "00:11:00" -t "00:18:00" -vcodec copy -acodec copy Output_3.mp4
  176. ffmpeg -i DL_Chunk_02.mp4.flv -ss "00:37:00" -t "00:16:00" -vcodec copy -acodec copy Output_4.mp4
  177. ffmpeg -i DL_Chunk_03.mp4.flv -ss "00:17:30" -t "01:15:00" -vcodec copy -acodec copy Output_5.mp4
  178. ffmpeg -i DL_Chunk_04.mp4.flv -ss "00:16:30" -t "00:36:00" -vcodec copy -acodec copy Output_6.mp4
  179.  
  180. View and check all the output files using VLC.
  181.  
  182.  
  183. Step 5:
  184.  
  185. Joining together the output files from step 4 is very easy.
  186.  
  187. Start Avidemux:
  188.  
  189. a) On the left side set video to 'copy', audio to 'copy', and format to 'MP4'.
  190.  
  191. b) Click 'open' at the top and select your first output file from step 4, in my example this would be 'Output_1.mp4'.
  192.  
  193. Avidemux will say:
  194.  
  195. "H.264 detected - If the file is using B-frames as reference it can lead to a crash or stuttering. Avidemux can use another mode which is safe but YOU WILL LOSE FRAME ACCURACY. Do you want to use that mode?"
  196.  
  197. Select 'Use Safe Mode'.
  198.  
  199. I got a big green screen after the video had loaded, click play (and then stop after a while) if you're worried that this is not normal, it frightened the pants off me, the green screen should immediately be replaced by your video playing.
  200.  
  201. c) Now go to the File menu and click 'Append', select your next output file, in my example this would be 'Output_2.mp4'. Select 'Use Safe Mode' again. Then select 'Append' again for each of your remaining output files (in order) from step 4, of course selecting 'Use Safe Mode' each time. You can keep an eye on the overall time length at the bottom and watch it grow as you append each file and check that it is growing by the right amount each time if you're worried.
  202.  
  203. d) When all the files have been appended... double check on the left side that video is still set to 'copy', audio to 'copy', and format to 'MP4' as per step 5 a). Then select the file menu, then 'save', then 'save video' and chose a save location and a name for your file, e.g. 'Full_Final_Edit.mp4' (add the '.mp4' bit at the end of the filename as this is not automatically added). Avidemux will now join all the files together nicely, muxing into a nice stable, .mp4 file. It may take a while, depending on your hard disk's speed and length of the video, 6.5 hours took my speedy desktop PC about 5 mins (but I have a nice new big and fast hard disk).
  204.  
  205.  
  206. That's it - end of tutorial.
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!