Basically the PC is run by the CPU (Central Processing Unit) . IT is continuously cycling through a set of instructions at the speed it is clocked at. I am not sure how many instructions there are, but it is not many. The motherboard is usually a bit slower and therefore halves the CPU speed (misses out every one in two cycles)This is why a Mac is faster. It may have a slower clock speed but it has a smaller instruction set called RISC (reduced Instruction set computer). The CPU has many connection pins for input/output of digital signals. As you say on or off/ power or no power. Now one of these instructions is called an interupt. Every time the CPU cycles through the instructions it queries if this interupt is receiving a signal or not. If for example, a key is pressed the interupt will be triggered. Of course something (presumably the keyboard controller, is scanning the whole keyboard loads of times every millisecond and again is relying on an interupt) Now, instead of going to the next instruction in the cycle it branches off to another circuit, possibly through the memeory where the fonts are stored and then to the screen print instruction set.Once that 'job' has been done the CPU continues it's normal cycle again. Of course it doesn't actually work like this because 1. The CPU multi tasks and 2. it has a lot of stuff stored in its cache memory. While it is attending to that interupt it is returning, back and forth, to the cycle, doing other jobs and seeing to other interupts.
To put it another way lets say you are sitting in achair reading a book. Without realising it you are continually and subconciously listening to see if the 'phone or the doorbell is ringing. If the 'phone rings you pick it up. (more likely you instruct the wife to answer it, get on with reading the book, but remembering you told the wife and waiting for her to report back that sjhe has finished that task!You are still listening to see if the doorbell is ringing or not and memorising where you were in the book. You talk on the phone, still listening for doorbell. Finish phone call, listen foor doorbell, go back to reading, scrap memorising where you were in book. While you are doing this you are also doing a thousand other things.
The CPU is very powerfull: It has memory, it can add up huge numbers at great speed (and therefore multiply and everything else) and it can use logic (IF, AND, OR etc.). It is also connected to other powerful and fast task-specific MPUs (Micro Processor Units) such as Graphic, Video, Harddrive controller, I/O (input output) chips. The I/O chip looks after input and output. Input would be the keyboard, mouse etc., Output would be screen (video controller) or printer etc.
So the CPU 'knows' to pass requests for different tasks to these other parts of the PC.
You should look at a schematic diagram of the motherboard to get an idea of the flow of data around a PC
The basic requirements of the computer to be 'live' are the CPU, Power, and a BIOS (Basic Input Output System which is permanently held in ROM). You then latch on everything else. Firstly a DOS (Disc Operating System) which can read and write to a hard drive. On the hard drive you put your OS (Operating System) which would now be MS Windows. Strangely enough a modern PC still works exactly the same as the original IBM PC. On start up it goes through the same processes as it always has done, although more hardware and software has been added on top. You can still run a PC in Command Mode (without the operating system). You can see this if you go to Start/All programs/Accessories/Command Prompt. Although in this case it is still running through Windows. Somehow you can get the PC to run in Command prompt without starting Windows and you will see you have no GUI (Graphical User Interface), just text commands. In a way the system is very simple because, as you say, it all works on binary (on or off, zero or one).