In the index page for the gallery you are working with there is a div like this on the page:
<div class="fullsize">
This is the div that the larger picture will be in. You can add your code inside of this div to have the button next to the image. If you are wanting to add an image to your cart or something similar you will probably have to do some customization of the js for the gallery to set the id of the image for the add to cart button when the images change out.