In Windows API programming, the classes, device contexts and objects (e.g. file system and GUI objects) have nothing to do with OOP; The term, "object" is just used to refer to something.
Also:
A window class is NOT something related to OOP. They just use the 'class' word because a window class defines the properties of a window. Once a class is defined (e.g. with RegisterClassEx), you can use that class (i.e. window properties) to create other windows with those same properties, without having to specify them again.
Windows API programming isn't as bad as you think. I mean, you REALLY wouldn't want to try using stupid crap like the Borland VCL framework or Microsoft MFC framework libraries. The closest you get to OOP is when you need to acess those idiotic COM objects for DirectX and some other things. Despite the name, COM objects are not exactly like an OOP object. Rather, COM objects are just a method of distributing code. In fact, you won't find a more brain-dead method of providing code functions anywhere else. (To illustrate how stupid those Adobe programmers are, those idiots based their Director code libraries on COM objects.)
By the way, did you get Petzold's book? Seriously, unless you want to waste a lot of time, you're better off learning from his book rather than just looking at the Windows API documentation. The only caveat is that his books were written for Win9x, so some functions may have changed for Win2000 and later. Win2000 is a combined version of business-oriented Windows NT and the regular consumer version of Windows (Win9x). So, you have additional functions from NT, plus the extra security.