Can you have no theme on WordPress?

Every WordPress site technically needs to have a theme installed and activated, but that doesnt necessarily mean you need to use your themes design.