I'd recommend a slightly different angle. Since you just want to get going with programming, I'd suggest one of the currently-hot dynamic languages: Ruby, Python, PHP, or Perl.
The advantage of these languages is that they're simple to learn. C and C++ are primitive languages: they expect you to understand concepts like pointers, memory allocation, buffer overflows, etc. Java has, in my opinion and experience, a higher cost-of-entry: it takes more code to get stuff done. My guess is that you just want to start getting some code working.
The languages I mentioned above all make it easy to just get stuff going. I'd probably recommend Ruby, then Perl, then Python, then PHP in terms of "just programming".
Ruby is very hot at the moment; learning it could be very advantageous to you. Perl is a language you'll either love or hate - if it fits the way you think, it'll be a piece of cake; if not, you'll curse it and wish you never had to look at it. Python is a little more like Java than the previous two, so there's a little more "initial cost" in learning to use it. Many folks love it - I sort of like it, but it's not what I pick when I'm just banging out a project. PHP is really a web templating toolkit with an attached programming language, but it's easy to use and many people who otherwise don't program at all love it; it's gotten a bad rep as insecure because of people not using its built-in security functions.
Disclaimer: I love Perl, like Ruby, and an neutral on Python and PHP.
You should be able to run any of these on Windows, Linux, or Mac OS X. Just search Yahoo! for your OS and the language of choice: "Windows Perl"; "Linux Ruby"; "OS X Python". Have fun.