You can only embed images directly in New Sites. To insert documents you can upload them to Google Drive, share them publicly* and insert them using the From Drive option in the INSERT menu.

See also a PDF example

Here are some more detailed instructions from sites guru Steegle:

*You can also publish the document in order to embed it into your site. See this information from a docs help file:

To make a document, spreadsheet, or presentation, available for a large audience to see, publish a file. After you publish your file, you’ll have a URL that you can send to anyone or embed into your website.

If you have several files to insert you can create folder in Drive, share it appropriately and place your files in that folder.

In the first file below I have disabled the download capability. See the instructions here:

Demo Doc
Example Document