API tools faq
paste
Advertisement
Guest User

mozjpeg-optimizer-v2.sh

a guest
Jun 14th, 2021
680
0
Never
Add comment
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
Bash 9.43 KB | None | 0 0
raw download clone embed print report
  1. #!/bin/bash
  2. #This script optimizes JPG and PNG files recursively to JPGs of smaller file size without significant loss in visual quality with "mozjpeg" (https://github.com/mozilla/mozjpeg). This script then asks whether the original files should be overwritten with the optimized files or should both be preserved. Optimized files are always smaller than the original files because optimized files are not generated if they are larger than the original ones. This script tests 21 mozjpeg parameter sets on each file about to be optimized to find which set outputs the tiniest file size. Out of all possible sets, only these sets were selected because they were found to output tiniest file sizes most of the time.
  3.  
  4. #8 june 2021 v1
  5. #mozjpeg-optimizer-v1.sh was written in 8 june 2021 and tested to be functional with mozjpeg v 4.0.4 on Debian 10. This script works with files/folders containing at least %, $, *, white spaces horizontal tabs or newlines. "mozjpeg" command denoted in this script in mozjpeg v 4.0.4 is actually "cjpeg" unless a symbolic link after mozjpeg installation is created like so:
  6. #sudo ln -s /opt/mozjpeg/bin/cjpeg /usr/bin/mozjpeg
  7. #If no symbolic link is created, replace all instances of "mozjpeg" in this script with "cjpeg" to be able to use this script.
  8. #mozjpeg-optimizer-v1.sh is available here: https://pastebin.com/vfqnSHne
  9. #mozjpeg-extreme-simulation-v1.sh is available here: https://pastebin.com/wPYd0Hzw
  10.  
  11. #16 june 2021 v2
  12. #mozjpeg-optimizer-v2.sh (this script) was written in 16 june 2021 and tested to be functional with mozjpeg v 4.0.4 on Debian 10. Main improvements since v1: parallelization added to make optimization faster on multicore CPUs.
  13.  
  14. #COPYRIGHT: this script is made available under the Creative Commons CC0 1.0 Universal Public Domain Dedication (https://creativecommons.org/publicdomain/zero/1.0/deed.en). The original creator of this script has no affiliation with mozjpeg or Mozilla.
  15.  
  16. printf 'CAUTION:\n- This script does not add JFIF to the optimized JPGs. This reduces file size by to 18 bytes, but breaks standards and may cause compatability issues. To preserve JFIF, edit this script and remove all instances of \"-nojfif\".\n- Transparent PNGs will lose transparency upon optimization to JPGs. If this is an issue, either edit this script to exclude PNGs from processing or do not process transparent PNGs.\n\nSTART: %s\nSize in bytes before (old) and after (new) optimization. %%: new size of old size in percentage. micros.: total microseconds spent on processing the file.\n\nold\tnew\t%%\t(micros.)\tname and path\t(parameters used by mozjpeg to achieve new size)\n' "$(date)"
  17.  
  18. #Temp files to RAM as variables.
  19. R0="$(mktemp -p /dev/shm/)"
  20. R1="$(mktemp -p /dev/shm/)"
  21.  
  22. while IFS= read -r -d '' i; do
  23.     S=${EPOCHREALTIME/./}
  24.     #Filename and size saved. If the name has newlines or tabs, they are converted to spaces so the names display well in terminal.
  25.     name="$(stat --printf="%n" "$i" | tr '\n' ' ' | tr '\t' ' ')"
  26.     size="$(stat --printf="%s" "$i")"
  27.  
  28.     #Optimization is simulated once. Optimized size in bytes is extracted and saved to $n. $n and the used parameters without -memdst (this parameter signifies simulation instead of actual file generation) are written to $R0 simultaneously overwriting (>) any info from previous loops in it.
  29.     n=$(mozjpeg -memdst -dct float -quant-table 1 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9')
  30.     printf '{%s\t-dct float -quant-table 1 -nojfif -dc-scan-opt 2}' "$n" > "$R0"
  31.  
  32.     #If optimized size is larger than original size, then file is skipped. Else: other parameters are tested.
  33.     if((size<n)); then
  34.         printf '%s\n' "$size    $name was SKIPPED as optimization did not reduce size."
  35.         printf '%s\n' "$size    $name was SKIPPED size could not be reduced by optimization."
  36.  
  37.     #"{...} &" split simulations to 20 parallel sims. This maximizes speed by occupying all cores of CPUs that have up to 20 cores. Speed may be increased modestly by removing brackets so that the number of bracketed command groups (parallel processes) is equal to the number of cores in your CPU.
  38.     #Simulated sizes and corresponding parameters are appended to $R0. "{...}" in printf help identifying sim data from each other. These brackets are used momentarily and later on replaced by newlines. Newlines are not used directly for identification to prevent output errors to $R0, which sometimes occurs with parallelization.
  39.     else
  40.         {
  41.         mozjpeg -memdst -dct float -quant-table 2 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -quant-table 2 -nojfif -dc-scan-opt 2}' >> "$R0"
  42.         } &
  43.         {
  44.         mozjpeg -memdst -dct float -quant-table 3 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -quant-table 3 -nojfif -dc-scan-opt 2}' >> "$R0"
  45.         } &
  46.         {
  47.         mozjpeg -memdst -dct float -tune-ms-ssim -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ms-ssim -nojfif -dc-scan-opt 2}' >> "$R0"
  48.         } &
  49.         {
  50.         mozjpeg -memdst -dct float -tune-ms-ssim -quant-table 3 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ms-ssim -quant-table 3 -nojfif -dc-scan-opt 2}' >> "$R0"
  51.         } &
  52.         {
  53.         mozjpeg -memdst -dct float -tune-ssim -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ssim -nojfif -dc-scan-opt 2}' >> "$R0"
  54.         } &
  55.         {
  56.         mozjpeg -memdst -dct float -tune-ssim -quant-table 0 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ssim -quant-table 0 -nojfif -dc-scan-opt 2}' >> "$R0"
  57.         } &
  58.         {
  59.         mozjpeg -memdst -dct float -tune-ssim -quant-table 1 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ssim -quant-table 1 -nojfif -dc-scan-opt 2}' >> "$R0"
  60.         } &
  61.         {
  62.         mozjpeg -memdst -dct float -tune-ssim -quant-table 2 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ssim -quant-table 2 -nojfif -dc-scan-opt 2}' >> "$R0"
  63.         } &
  64.         {
  65.         mozjpeg -memdst -dct float -tune-ssim -quant-table 3 -nojfif -dc-scan-opt 1 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ssim -quant-table 3 -nojfif -dc-scan-opt 1}' >> "$R0"
  66.         } &
  67.         {
  68.         mozjpeg -memdst -dct float -tune-ssim -quant-table 3 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ssim -quant-table 3 -nojfif -dc-scan-opt 2}' >> "$R0"
  69.         } &
  70.         {
  71.         mozjpeg -memdst -dct float -tune-ssim -quant-table 4 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-dct float -tune-ssim -quant-table 4 -nojfif -dc-scan-opt 2}' >> "$R0"
  72.         } &
  73.         {
  74.         mozjpeg -memdst -quant-table 2 -nojfif -dc-scan-opt 1 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-quant-table 2 -nojfif -dc-scan-opt 1}' >> "$R0"
  75.         } &
  76.         {
  77.         mozjpeg -memdst -quant-table 2 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-quant-table 2 -nojfif -dc-scan-opt 2}' >> "$R0"
  78.         } &
  79.         {
  80.         mozjpeg -memdst -tune-ssim -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-tune-ssim -nojfif -dc-scan-opt 2}' >> "$R0"
  81.         } &
  82.         {
  83.         mozjpeg -memdst -tune-ssim -quant-table 1 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-tune-ssim -quant-table 1 -nojfif -dc-scan-opt 2}' >> "$R0"
  84.         } &
  85.         {
  86.         mozjpeg -memdst -tune-ssim -quant-table 2 -nojfif "$i" |& tr -cd '0-9' | xargs printf '{%s\t-tune-ssim -quant-table 2 -nojfif}' >> "$R0"
  87.         } &
  88.         {
  89.         mozjpeg -memdst -tune-ssim -quant-table 2 -nojfif -dc-scan-opt 0 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-tune-ssim -quant-table 2 -nojfif -dc-scan-opt 0}' >> "$R0"
  90.         } &
  91.         {
  92.         mozjpeg -memdst -tune-ssim -quant-table 2 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-tune-ssim -quant-table 2 -nojfif -dc-scan-opt 2}' >> "$R0"
  93.         } &
  94.         {
  95.         mozjpeg -memdst -tune-ssim -quant-table 3 -nojfif -dc-scan-opt 1 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-tune-ssim -quant-table 3 -nojfif -dc-scan-opt 1}' >> "$R0"
  96.         } &
  97.         {
  98.         mozjpeg -memdst -tune-ssim -quant-table 3 -nojfif -dc-scan-opt 2 "$i" |& tr -cd '0-9' | xargs printf '{%s\t-tune-ssim -quant-table 3 -nojfif -dc-scan-opt 2}' >> "$R0"
  99.         }
  100.         wait
  101.         #sed adds a newline between each }{ in $R0 and tr removes these brackets. Each sim data (now separated by newlines) is sorted by bytesize in ascending order and writted to $R1.
  102.         sed -e 's/}{/}\n{/g' "$R0" | tr -d '{}' | sort -n > "$R1"
  103.  
  104.         #Parameters producing the tiniest file size are extracted from $R1 via head+cut and fed to mozjpeg as $par to produce an actual optimized file.
  105.         par=$(head -n1 "$R1" | cut -f2)
  106.         mozjpeg $par "$i" > "$i.opti.jpg"
  107.  
  108.         #Time spent on processing and new vs old size in percentage are calculated and displayed.
  109.         newsize=$(head -n1 "$R1" | cut -f1)
  110.         percent=$((200*newsize/size%2+100*newsize/size))
  111.         E=${EPOCHREALTIME/./}
  112.         T=$((E-S))
  113.         printf '%s\n' "$size    $newsize    $percent    ($T)    $name   ($par)"
  114.     fi
  115. #Recursively finds files with variable extensions. Add "-maxdepth 1" after "find ." to search non-recursively. Remove " -o -iname '*.png' to optimize JPG files only.
  116. done < <(find . -type f \( -iname '*.jpg' -o -iname '*.jpeg' -o -iname '*.png' \) -print0)
  117.  
  118. #Temp files are removed from RAM.
  119. rm -f "$R0" "$R1"
  120.  
  121. printf '\nEND: %s\n' "$(date)"
  122. read -r -p "Should original files be replaced with optimized .opti.jpg backups (see output for errors before deciding)? Press Y for YES or N for NO and press ENTER: " response
  123. if [[ "$response" =~ ^([yY][eE][sS]|[yY])$ ]]
  124. then
  125.     find . -type f -name '*.opti.jpg' -exec bash -c 'mv "$1" "${1/%.opti.jpg/}"' -- {} \;
  126.     printf 'Done. Original files were replaced.\n'
  127. else
  128.     printf 'Done. Both files were preserved.\n'
  129. fi
  130.  
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement
Public Pastes
Advertisement
We use cookies for various purposes including analytics. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy.  OK, I Understand
Not a member of Pastebin yet?
Sign Up, it unlocks many cool features!