Retrofitting language models to operate over bytes