We investigate linear elliptic, second-order boundary value problems with mixed boundary conditions on domains with a rough boundary. Assuming only boundedness/ellipticity on the coefficient function and very mild conditions on the geometry of the domain, including a very weak compatibility condition between the Dirichlet boundary part and its complement, we prove Hölder continuity of the solution. Moreover, Gaussian Hölder estimates for the corresponding heat kernel are derived. This is joint work with J. Rehberg.