You're probably viewing your pages by opening them as local files rather than loading them from a web server. So instead of the URL of the page in your browser starting with http:// it starts with C:\ or file://
Here's an example of how the buttons show up if viewed via http:// or https://
And here's what they look like when viewing the same file from the file:// protocol:
We currently are using protocol-free URLs in our code - ones that start with // instead of http:// or https:// - to make sure our SSL implementations are cross compatible without extra work. However, if you're loading the test page with the file:// protocol it causes the CSS and other JavaScript files not to load, since it's looking for them on your local machine rather than our web servers.
I've submitted a ticket to our developers to fix this for people previewing pages using their file system rather than a web server, but until they can get that implemented you should either understand that the buttons won't show up if using the file:// protocol or use a web server in order to test how your sites display in browsers.