Sandstone caves are formed when wind or storms knock into the very unsteady sand like rock.  Sometimes these caves are no deeper then a couple of yards.  Another unusual way these caves are formed is by streams when they smash into the rock ebbing away at it.  The cave dwelling Indians, which you might of heard of build amazing structures around the caves, which they call their home.  Some more people who would use these caves were settlers.  When settlers would get caught in storms they would use these caves for cover.

Image courtesy of http://www.dgl.microsoft.com