by ·1 Comment

ReSharper C++In the Visualization Team, we’ve recently started using ReSharper C++ 10.0.2 in Microsoft Visual Studio to assist with writing C++ code. We also have some home-grown (and predictably ugly) C++ preprocessor macros specifically for the Microsoft compiler to help with various code quality issues such as automatic coverage reports, execution profiling, unit test generators and so on. The problem is that when ReSharper parses the C++ source code, it does so with a slightly different “interpretation” of the the C++ preprocessor specification than Microsoft’s compiler and the IntelliSense parser. I’m not going to get involved with any argument as to which is “most correct”; let’s just say that they are different. Different enough that ReSharper complains about our macros, even though the compiler thinks they’re fine. If we could detect when the ReSharper parser is looking at our code, we could simplify the macro definitions and stop ReSharper complaining.

Microsoft Visual Studio helpfully provides a preprocessor macro named ‘__INTELLISENSE__‘ that allows you to detect when it is IntelliSense that is parsing your source code, but I couldn’t find the equivalent for ReSharper. That’s not to say that one doesn’t exist, but I couldn’t find any on-line documentation for one.

However, there obviously is a difference between the Microsoft and JetBrains parsers (otherwise we wouldn’t need to distinguish between them!) so can we use that variation to detect who is parsing our source code? The difference that is causing our macros problems is the way that macro argument tokens are pasted together and joined. Here’s an example:

#define LITERAL(a) a
#define JOIN(a,b) LITERAL(a)LITERAL(b)
#define BEFOREAFTER 1

The first question is: why aren’t we using the token pasting operator? Ironically, that operator, ‘##‘, solves all our problems (in this case). Therefore it’s not a candidate for distinguishing the two parsers. So, given the macros above, what does the following expand to?

JOIN(BEFORE,AFTER)

Well, the Microsoft products (as of MSVC 2013) expand it to ‘1‘ whereas ReSharper expands it to two tokens: ‘BEFORE‘ immediately followed by ‘AFTER‘. Fascinating, but not particularly useful, surely? Ah, but consider this:

#if JOIN(BEFORE,AFTER)
   // We're being parsed by Microsoft products
#else
   // We're being parsed by ReSharper
#endif

Inside, the ReSharper parser is no doubt bitterly fuming about the malformed ‘#if‘ condition; but it does so silently and the test condition ultimately fails.

Putting it all together gives us:

#define RESHARPER_LITERAL(a) a
#define RESHARPER_JOIN(a,b) RESHARPER_LITERAL(a)RESHARPER_LITERAL(b)
#define RESHARPER_DISABLED 1
#if RESHARPER_JOIN(RESHARPER,_DISABLED)
#define __RESHARPER__ 0
#else
#define __RESHARPER__ 1
#endif
#undef RESHARPER_DISABLED
#undef RESHARPER_JOIN
#undef RESHARPER_LITERAL

Of course, this code snippet is preceded by a huge comment explaining why we’re abusing the preprocessor quite so badly, and suggesting that the reader pretends she never saw it.

by ·0 Comments

In the first part of this series of posts, we introduced the idea of trying to detect flesh in images by looking at the colour values of individual pixels in the image. This produces reasonable results, but far too many “false positives” due to the fact that other items in the scene, such as hair and clothes, may be flesh-coloured too.

Boolean Pixel Function

In the example below, pixels in the left-hand image that are fleshy (R > G > B) are rendered red in the right-hand image, whereas non-fleshy pixels are rendered green:

Simple flesh detection

Simple flesh detection

Fuzzy Pixel Function

We can improve things slightly by using fuzzy logic. Our original fleshy function (R > G > B) is actually made up of two conditions: a pixel is “fleshy” if the red component is greater than the green component and the green component is greater than the blue component. These two conditions are binary, but they could be made fuzzy. Consider the following JavaScript function:

function fuzzy(x, false_limit, true_limit) {
  var y = (x - false_limit) / (true_limit - false_limit);
  return Math.min(Math.max(y, 0), 1);
}

This produces a fuzzy truth value between zero, meaning definitely false, and one, meaning definitely true:

fuzzy_graph

We can then compose a fuzzy logic expression for fleshiness (notice that the fuzzy AND operator is simply multiplication):

var rg = fuzzy(r - g, 0, 0.10);
var gb = fuzzy(g - b, 0, 0.02);
var fleshiness = rg * gb;

The values 0.10 and 0.02 were derived empirically. Effectively, we’re saying that we expect the red value to be quite a bit greater than the green channel; the difference between the green and blue values is less important.

The fuzzy approach gives us marginally better results. Parts of the hair are deemed to be less likely to be fleshy, as are some portions of the dress pattern.

flesh2_dress_fuzzy

But, as mentioned at the end of Part One, we need a radically different approach to consistently find accidentally-rendered “naughty bits” in an image.

Chameleon Detector

Fortunately, we have control over the rendering pipeline of these images, so there’s nothing stopping us from rendering the scene twice with slightly different parameters. Let us pretend that belly buttons are considered “naughty” and that we want to detect renders that show some or all of this body part. When we render body parts, we use texture mapping on to a 3D mesh. If we “paint” over the naughty bits in the source texture maps with a known colour (say, green) and render the scene, we may get the following for two different outfits:

fleshy2_star_green

For the purposes of clarity, we’ve painted a large star over the belly button. In reality, the painted region would be smaller and more accurately shaped. If we render the scene again with the naughty bits over-painted with the same shape but a different colour, say, red, we get:

fleshy2_star_red

Obviously, the image on the right is unchanged by this modification to the skin texture, but the image on the left is. All we need to do is run the two sets of images through a very simple (fuzzy) comparator to find visible naughty bits:

flesh2_star_fuzzy

As can be seen, this “chameleon” technique produces a strong signal. And even though it requires two renders per image, there are other advantages too:

  1. The regions considered “naughty” are hand-painted into the source skin textures. This is both intuitive and flexible.
  2. Different “naughtiness maps” can easily be used for different regions and cultures.
  3. One of the outputs of the technique is an image illustrating which naughty bit is visible and where.
  4. It is body shape agnostic.
  5. It is viewpoint agnostic.
  6. It handles translucent garments gracefully, particularly if a fuzzy comparator is used.
  7. It does not matter how complex the scene is.
  8. The code used to run the test is identical to the final rendering code: only input texture data is modified.

by ·0 Comments

As part of the Visualization Team at Metail, I spend a large proportion of my time staring at renders of models, hoping that too much virtual flesh isn’t being exposed. It’s a onerous task and any automation to make my life easier is always appreciated. Can we get computers to make sure “naughty bits” aren’t being accidentally shown to our customers?

Detecting Flesh Colours

The diversity of flesh colours is quite large; the following is a small selection of MeModel renders:

Feet Skin Colours

Feet Skin Colours

The range of colours considered “fleshy” is further compounded by the synthesised lighting environment: shadows tend to push the colours towards black, highlights towards white.

The first question to consider is which colour space to use for detecting flesh colours. There’s quite a bit of informal debate about this. Some propose perceptual spaces such as HSL or HSV, but I’m going to stick my neck out and say that there’s no reason not to use plain, old RGB.1 Or, more precisely, to overcome the effects of fluctuating lighting levels, some form of normalized RGB. For example:

M = max(R, G, B) or 1 iff R ≡ G ≡ B ≡ 0
R* = R ÷ M
G* = G ÷ M
B* = B ÷ M

Two observations can be made at this point:

  1. If R ≡ G ≡ B ≡ 0, we’re in such a dark place that we cannot tell whether we’re looking at flesh or not; and
  2. The dominant colour channel in all (healthy) human skin is red (green-skinned lizards and blue Venusians are unsupported), so M = R.

Typically, we further assume that the least dominant colour channel is blue. This leads to a pleasing heuristic:

(R, G, B) is flesh only if R > G > B

Marrying this up with a hue/brightness wheel gives us a generous segment of potentially “fleshy” colours:

Hue/Brightness Wheel

Hue/Brightness Wheel

There are various refinements to this linear programming technique to further partition the colour space, including:

  • J. Kovac, P. Peer, and F. Solina, “Human skin colour clustering for face detection” in Proceedings of EUROCON 2003. Computer as a Tool. The IEEE Region 8, 2003
  • A. A.-S. Saleh, “A simple and novel method for skin detection and face locating and tracking” in Asia-Pacific Conference on Computer-Human Interaction 2004 (APCHI 2004), LNCS 3101, 2004
  • D. B. Swift, “Evaluating graphic image files for objectionable content” US Patent US 6895111 B1, 2006
  • G. Osman, M. S. Hitam and M. N. Ismail, “Enhanced skin colour classifier using RGB Ratio model” in International Journal on Soft Computing (IJSC) Vol.3, No.4, November 2012

For example, one in-house heuristic we tried could be coded in JavaScript as:

function IsSkin(rgba) {
  if ((rgba.a > 0.9) && (rgba.r > rgba.g) && (rgba.g > rgba.b)) {
    var g = rgba.g / rgba.r;
    if ((g > 0.6) && (g < 0.9)) {
      var b = rgba.b / rgba.r;
      var expected_b = g * 1.28 - 0.35;
      return Math.abs(b - expected_b) < 0.05;
    }
  }
  return false;
}

However, there are some fundamental flaws with merely classifying each pixel as either “fleshy” or “non-fleshy”:

  1. The portion of the colour space that is taken up by human hair, although larger, overlaps the space taken up by flesh tones. This makes distinguishing between long hair lying on top of clothes and actual flesh very difficult.
  2. Many clothes or parts of clothes are deliberately designed to be flesh-coloured or “nude” looking.
  3. You do not know if the “fleshy” pixels are at naughty locations of the body or not.
  4. As the body shape parameters of the MeModel changes, the location of the “naughty bits” changes.

We’ll address these issues in the next part. However, even with fairly simplistic heuristics, the techniques discussed thus far reduce the number of images that have to be pored over by humans, as opposed to automated systems, by up to 90%.

Footnotehsv

The maximal range of hues generally considered as skin tones by most flesh pixel detectors is 0° ≤ H ≤ 60° (red to yellow).

This range equates to R ≥ G ≥ B, as can be seen in the chart on the right.

Furthermore, the saturation and value (or lightness) components of HSV (or HSL) detectors are usually discarded. Therefore, the “flesh predicate” can be constructed purely from relative comparisons between R, G and B, or pairwise differences thereof. This is why I believe the RGB colour space to be no less accurate than other colour spaces.