Shape displays can be used to render both 3D physical content and user interface elements. We propose to use shape displays in three different ways to mediate interaction: facilitate, providing dynamic physical affordances through shape change; restrict, guiding users through dynamic physical constraints; and manipulate, actuating passive physical objects on the interface surface. We demonstrate this on a new, high-resolution shape display.