The first advantage is that this allows PHP to execute code that can alter the files on the server. This can be done by using some built-in PHP functions. You could think of it as the CMD promt on you Windows machine. There are commands that can create directories, change filenames, compress images and all kinds of other stuff that directly alters the content of the directories of your website.
You will never find a piece of PHP code by inspecting the website element (unless someone fucked up the syntax). The code that is executed is invisible for the visitor, which is quite convenient since we are going to need to store a password in our PHP code.