It's a "hypertext preprocessor".
To start, it's a program that runs on a server.
If you take an HTML file and put it on a website, then visit the URL of that file, the HTML will be sent to the visitor and displayed by the browser.
With a PHP file, you write a script in the PHP language, and upload it to the server. Assuming that the server has PHP (the program) installed, the sever will open up the script on the SERVER's end, interpret the PHP language that you wrote.
So, the PHP script tells the server what to do. Usually, you would use the PHP to tell the server that you want to output HTML. You could say, print "", and then the server will send "" to the visitor. Then you could tell it to print "", and so on, for the rest of the HTML page.
On the visitor's end, there is no difference between a PHP page and an HTML page. With a PHP page, the web browser still receives a bunch of HTML and displays it. The only difference is that, instead of a pure HTML file that was uploaded and downloaded in the same form, the PHP file was a bunch of HTML that was generated according to the instructions in a PHP script.
What's the advantage to this if you can write HTML without PHP?
The advantage is that you can use PHP to identify certain conditions, and use different HTML for different conditions, in order to make a "dynamic" page.
For example, you could use PHP to determine the time of day. Then you could say, "if" it's in the afternoon, I want to ouput such-and-such HTML. "if" it's in the morning, I want to output different HTML. So on.
A very common use for PHP is to connect to a database. You could tell PHP to get information from a database and display it on the page. That way, any time the database information changes, the PHP page changes.
Another common use is to handle web forms--for example, the web form you used to submit your question about PHP. Once you submit the form, it could be sent to a PHP page. Then the PHP makes sure that you didn't submit an empty form. In conjunction with that, PHP is used to put information into a database, so then the PHP page takes the question you asked and put it into a database.
PHP has many different functions. It can deal with databases (one kind of database is called MySQL, but that's not the only kind), email, file processing, dates, times, time zones, and many other types of data.
PHP is not the only kind of language that provides this type of capability. ASP, Ruby, Python, Java, and C can all be used on websites. But, PHP is very common. It's easy to start learning (but that doesn't mean it's easier to get good at it).
For a very simple introduction to the PHP language, see http://www.w3schools.com/