It really depends on who Joe Blow is. Who is your target audience?
Are you targeting only Windows users? Or are you going after Linux and handhelds? If Windows Only isn't OK, I would rule out C# right away (you don't want Joe Blow to have to navigate the WINE/MONO landscape).
For Windows only...what kind of consumer? Home or commercial? How many will still be running XP? Commercial users tend to be further behind than home users, and aren't always even allowed to install stuff like the .NET framework.
For home users, I would guess that more of them have at least .NET 2.0 installed than Java, although I couldn't find any hard numbers to back that up. Best I could find is the link below, but that is browser compatibility, not desktop. I'm surprised there aren't better numbers on this (or maybe I just missed them).
You might have to restrict yourself to .NET 2.0 to get the fewest complaints. But the same is true of Java. You certainly can't use 1.7.
C# is without a doubt hands down better for writing Windows forms based desktop apps. The language and the IDE are much more powerful for that sort of task. If you limit yourself to .NET 2.0, and your typical Joe Blow isn't a full 10 years behind the times, you will probably hit 90+% on the...it just works scale. But that's just a guess.