Download WAMP or XAMPP (free- they both do the same thing). Saves downloading and configuring all the components for a web server separately. It creates a web server on your computer so you go to
http://localhost/ and that is your website.
You can copy the files onto the htdocs folder that will be created on your computer. With your MySQL database you will have to go into phpmyadmin on your current host, export the database as an SQL file, then going to phpmyadmin on your computer and copy & paste the SQL and execute it.
Obviously this means you will only be able to access your "website" from your computer.