Lesson 13: Layer masking

Layer masking is a powerful tool in the hands of the creative image author, combining the capabilities of non-destructive editing and selective masking.

Engaging the concepts and techniques presented in this Lesson on your images, you will realize in action the amazing power of layer masking.

So what is a layer mask?

It is actually a sort of a selection -- an 8bit channel like the Quick mask in the Quick Mask View -- which acts on a specific layer not as a selection really but as a visibility mask. It is implemented as a channel with reference not to the whole image as the Quick Mask but to a specific layer. Every pixel of the layer mask controls the visibility of the corresponding pixel of the layer at a given degree, receiving a value from the 8bit mask channel (0-255): if 0 (black on the mask) the layer pixel becomes 0% visible (= invisible), if 255 (white on the mask) the layer pixel becomes 100% visible, and with indermediate values in this dynamic range defining the degree of the layer pixels' visibility.

A layer mask is a permanent information saved inside the file formats which support layers - the GIMP's XCF and the Photoshop's PSD.

The layer's mask is irrelevant with the layer's alpha channel. A layer that does not have an alpha channel may well have a layer mask.

A layer mask refers and acts on a layer immediately at will, constituting a separate entity (drawable). You can edit the mask leaving the layer untouched, or you can edit the layer leaving the mask untouched. This offers the exquisite possibility of non-destructive editing: you can work with the layer keeping the layer untouched by itself.

The Add Layer Mask dialog

The dialog offers the following options to help you quickly initialize the mask:

• White (full opacity) - All pixels of the mask at a value of 255 - all layer pixels 100% opaque.
• Black (full transparency) - All pixels of the mask at a value of 0 - all layer pixels 100% transparent.
• [For advanced users. Layer's alpha channel - If the layer has an alpha already, every pixel's transparency will double up because it will be affected from both the alpha channel and the mask. After that you will probably want to 'Remove Alpha Channel', so to make it easier GIMP offers the 4th option, 'Transfer layer's alpha channel', performing the previous two actions in one shot and all pixels on the alpha channel getting a value of 255.]
• Selection - GIMP takes the current selection for the mask.
• Grayscale copy of layer - let us say like if you paste on the mask a 8bit version of your layer after a 'Desaturate'.
• Channel - You select from the drop-down-list anyone of the channels (if any).
• Finally with Invert mask you have the option to invert the mask values (in the range 0-255), so getting an inverted mask whichever your initialization choice may have been. In other words, you simply invert the mask channel, thus having the freedom to flexibly decide between the two logical directions of your masking: positive either negative.

Gathering the magic bits

Now let me try to put some tools together inside the GIMP's interface, to get my point across with a rough example for what i could name as multiple selective masking or variant selective masking.

If you have followed along the Lessons of this School so far, i am sure that you would understand me if i told you that: in the last example 5, my intention is to illustrate the power of knowing how to combine the GIMP's tools effectively. The test shot has deliberately been a difficult case, in order to expose this tremendous power: the figure's hair does not have good contrast neither on value nor on color.

In real world scenarios, a professional knows exactly what he wants from the shooting phase, making sure that the optical item he wants to separate does have enough contrast in value or color or both. A simple example: in commercial studio photography, a photographer can take an extra shot of a red object changing its background to cyan, so that later at the editing phase he can easily achieve a separation for the object, in order to use it on any background or composition. As explained in the theory in the first Lesson, the RGB and the CMYK color models represent two different logics on composing color -- so this means that using the Decompose Tool you have two complementary approaches to analyze color:

• RGB,
• CMYK or CMY.

So don't miss that point! Since any color has its complementary color, if the decomposing does not work on RGB good enough, it might work better in CMYK or CMY - or vice versa.

By elaborating the mask using both non-continuous masking and continuous masking as explained in Lesson 8, you will possibly be able to achieve a good enough separation result in most cases -- well, you know, there are cases when some work pays off.

Creating selections out of masks

• Layer > Mask > Mask to Selection (or RightClick a layer > Mask to Selection): you get a selection from the active layer's layer mask.
• The next 3 commands on 'Layer > Mask' submenu (Add to Selection, Subtract from Selection, Intersect with Selection): you can play using masks from different layers to build complex selections!

Two more commands on layer masks

• Layer > Mask > Apply Layer Mask (or RightClick a layer > Apply Layer Mask): converts the layer mask into an alpha channel (if the layer does not have, it gets it) and removes the layer mask.
• Layer > Mask > Delete Layer Mask (or RightClick a layer > Delete Layer Mask): rejects the layer mask - permanently deleted.

Think twice before doing these two actions, though:

• when you save your project, the layers together with their masks are saved, so you can edit them again in the future,
• when you export to a usable image file, GIMP takes into acount only the enabled layers together with their masks (if enabled). This means that you can save data for future use in extra layers + masks and deploy them at will inside your project.