User:Djr13/scripts/fftcycle

This bash script was created out of frustration with repeatedly needing to relearn the workflow for properly descreening scanned images to remove half-toning and the resulting moire effects. It is semi-automated, interactive in the sense that all you need to do is play around with the spectrum image and each time you save it, results will be automatically produced. No more messing around with plugins, and less need to revert your image editor after every FFT attempt. There are, however, many deficiencies, but honestly it's rarely worse than most manual use. Please do contribute improvements, suggestions, etc.

Requires that ImageMagick and inotifywait are installed. Currently tested only under Linux. The actual image processing snippets were completely ripped from the guide on the ImageMagick website. I recommend you read that guide and understand each step that this script cycles through, and again, try to improve it. Inotify and some other script structure ripped from denilsonsa's sleep_until_modified.sh.

fftcycle.sh

#!/bin/bash
# Documentation: https://commons.wikimedia.org/wiki/User:Djr13/scripts/fftcycle
# Interactive FFT script for descreening images. Requires ImageMagick and inotifywait.
# Snippets completely ripped from the guide on the ImageMagick website:
# https://www.imagemagick.org/Usage/fourier/#noise_removal
# I recommend you read that guide and understand each step that this script cycles through.
# Inotify and some other script structure ripped from:
# https://bitbucket.org/denilsonsa/small_scripts/src/78fb99fcb0444b4da31201cc41791b08099919bc/sleep_until_modified.sh
# Due to ripping from the ImageMagick usage guide, if this script contains any copyrightable content, it is under the ImageMagick License:
# https://www.imagemagick.org/script/license.php

TARGETORIG="$1"
TARGET="${TARGETORIG%.*}_fft"

mkdir "$TARGET/"

convert "$TARGETORIG" -fft \
	\( +clone  -write "$TARGET/phase.png" +delete \) +delete \
	-write "$TARGET/magnitude.png"  -colorspace gray \
	-auto-level -evaluate log 100000  "$TARGET/spectrum.png"

echo "
Edit \"$TARGET/spectrum.png\" to cover unwanted patterns and save as \"$TARGET/spectrum_edited.png\". The script will watch and automatically process the image each time the file is saved. Press Ctrl-C to end the script.
"

while "`dirname "$0"`/sleep_until_modified.sh" "$TARGET/spectrum_edited.png" ; do

	convert "$TARGET/spectrum_edited.png" "$TARGET/spectrum.png" \
		-compose difference -composite \
		-threshold 0 -negate "$TARGET/spectrum_mask.png"

	convert "$TARGET/magnitude.png" "$TARGET/spectrum_mask.png" \
		-compose multiply -composite \
		"$TARGET/phase.png" -ift "$TARGET/filtered.png"

	convert "$TARGETORIG" "$TARGET/filtered.png" -compose difference \
		-composite -normalize "$TARGET/noise.png"

	convert "$TARGET/spectrum_mask.png" \
		-blur 0x5 -level 50x100%  "$TARGET/mask_blurred.png"

	convert "$TARGETORIG" -fft \
		\( -clone 0 "$TARGET/mask_blurred.png" -compose multiply -composite \) \
		-swap 0 +delete -ift "$TARGET/filtered_2.png"

	convert "$TARGETORIG" "$TARGET/filtered_2.png" -compose difference \
		-composite -normalize "$TARGET/noise_2.png"

done

echo -e "Files processed:
Original:\t$TARGETORIG
\t$TARGET/phase.png
\t$TARGET/magnitude.png
Edit this but don't overwrite it:\t$TARGET/spectrum.png
Save your edits here:\t$TARGET/spectrum_edited.png
Mask extracted from edits:\t$TARGET/spectrum_mask.png
Result (via hard mask):\t$TARGET/filtered.png
Changes from original (via hard mask):\t$TARGET/noise.png
Mask after Gaussian blur:\t$TARGET/mask_blurred.png
Result (via blurred mask):\t$TARGET/filtered_2.png
Changes from original (via blurred mask):\t$TARGET/noise_2.png"

sleep_until_modified.sh

This is a companion script, needed because I gave up on the trouble inotifywait was giving me when I tried to embed the functions directly into the above script. Place this script into the same folder as the above script, and make sure it's named sleep_until_modified.sh. This is identical to denilsonsa's sleep_until_modified.sh except for the addition of quotes that otherwise broke the script for me for paths containing spaces.

#!/bin/bash

SCRIPTNAME=`basename "$0"`

print_help() {
	cat << EOF
Usage: $SCRIPTNAME filename
Uses 'inotifywait' to sleep until 'filename' has been modified.

Inspired by http://superuser.com/questions/181517/how-to-execute-a-command-whenever-a-file-changes/181543#181543

TODO: rewrite this as a simple Python script, using pyinotify
EOF
}

# check dependencies
if ! type inotifywait &>/dev/null ; then
	echo "You are missing the inotifywait dependency. Install the package inotify-tools (apt-get install inotify-tools)"
	exit 1
fi

# parse_parameters:
while [[ "$1" == -* ]] ; do
	case "$1" in
		-h|-help|--help)
			print_help
			exit
			;;
		--)
			#echo "-- found"
			shift
			break
			;;
		*)
			echo "Invalid parameter: '$1'"
			exit 1
			;;
	esac
done

if [ "$#" != 1 ] ; then
	echo "Incorrect parameters. Use --help for usage instructions."
	exit 1
fi

FULLNAME="$1"
BASENAME=`basename "$FULLNAME"`
DIRNAME=`dirname "$FULLNAME"`

coproc INOTIFY {
	inotifywait -q -m -e close_write,moved_to,create "${DIRNAME}" &
	trap "kill $!" 1 2 3 6 15
	wait
}

trap "kill $INOTIFY_PID" 0 1 2 3 6 15

# BUG! Não vai funcionar com arquivos contendo caracteres estranhos
sed --regexp-extended -n "/ (CLOSE_WRITE|MOVED_TO|CREATE)(,CLOSE)? ${BASENAME}\$/q" 0<&${INOTIFY[0]}

Bugs and potential features

Your help needed!

1. Maybe a proper repo? This was quick enough and in proximity to the projects that might benefit. I'd happily see this moved to a non-userspace page.
2. Should integrate the inotifywait function into the main script.
3. Didn't integrate much documentation or error handling.
4. A way to exit the script without Ctrl-C.
5. Handle file deletions.
6. Even with the supplied Gaussian-blurred version, the mask method still seems rather crude.
7. The blur radius also isn't scaled to the size of the image, should it be?
8. The blur is unshaped, and EG with small astroid masks, it'll completely blur them into circles.
9. There's no way to do blur-masks, or whatever other masks might be preferred, instead of multiply-overlay masks as currently used.
10. The color/tone of the overlays might be considered, too, but maybe the guide author chose for good reasons, beyond simplicity?
11. The script likely can't properly handle color half-toning where colors aren't totally merged.
12. Per the guide, the masking "[...]could posibly be improved further by adjusting that mask to fit the 'star' shapes better." As masking shapes are (sadly) a manual step, this isn't directly related to the quality of the script (until masking as a whole can be better automated/assisted). However, due to not having a suitable brush in GIMP, I created an astroid shape SVG which when rasterized can be imported into GIMP. Whether this improves results or not, I have no idea.
13. It should be pretty easy to adapt the script to allow it to produce revisions for each edit, rather than overwriting the results. Symlinks to the latest revision of each step might allow keeping the feature that allows having an image viewer refresh after every update.
14. I can imagine masking being much more automated and easy, but I can't write it. Perhaps a script could identify distinct patterns and present options to independently isolate and remove each, without requiring (nearly as much) of the guesswork of throwing blotches at a spectrum and hoping the unwanted pattern vanishes.
15. There might be dynamic range concerns, or there might not be. Per the guide, "Many of the examples use a HDRI Version of ImageMagick which is needed to preserve accuracy of the transformed images. It is recommened that you compile a personal HDRI version if you want to make the most of these techniques. [...] HDRI is thus vital when you plan to use extremely heavy mathematical processing of images, involving the temporary use of negative values, or strong scaling to very small or very large values. It is especially important for users that want to make full use of new Fast Fourier Transforms (FFT) capabilities, and it is here that you will see the most examples of a HDRI version of IM." If these concerns exist, it might be desirable to check that the installed ImageMagick is compiled with the HDRI feature (identify -version), and maybe there are more appropriate intermediate file formats than PNG. Or go all-out and package everything needed.

See also

• Help:Scanning
• Commons:Cleaning up interference with Fourier analysis
• wikibooks:GIMP/Remove Coherent Noise
• Template:CleanFFT
• Category:Images containing regular noise