The college is a rather spectacular collection of buildings in the village of Kirkoswald. It lays to the North of the moat near the church, and to the West of Kirkoswald Castle. The limited view of the buildings from the road, prevent a really detailed view of the buildings, but my guess is that the thinly walled, late 15th century tower (not shown in the photo below) lays at the front of the building. Check the photos from Visit Cumbria, taken from the air by Simon Ledingham.
The tower would originally have been of four storeys, but with its flat roof rebuilt into a steepled roof it lost a storey to become the three storeyed building we see today.