Over my years studying Chinese Medicine and living in different parts of the country, I have encountered different viewpoints on this issue. There are some distinct pros and cons that I have heard from roommates, healthcare providers, and from my own body talking to me. Today I'd like to offer both sides and a compromise.
On Removing Shoes
The arguments to remove shoes upon entering a house start in the energetic and/or cultural realm. It is seen in Japanese culture and many others as respectful to remove one's shoes when entering a house.
Roots for some cultural motivation go along with some of the more physical reasons. Water, dirt, and germs can track inside on shoes. Allergens like pollen or pet dander can also stay with shoes at the door.
Wearing Shoes or Slippers
Wearing shoes with quality arch support can be beneficial for maintaining joint alignment and preventing or alleviating back pain and other structural issues. While shoes are also doubly important for protecting the feet while out of the house, arch support is best maintained throughout the day at home as well.
Slippers can keep your feet warm indoors, but often lack arch support.
It can be helpful to delegate a pair of high quality shoes with arches as house shoes. These shoes are then not (possibly ever) worn outside, or are easy to clean and remain mostly for in the home.
Some arch supports can be fit into shoes that don't come with them, or might even fit in some slippers. But remember to commit to something sturdy enough that they do not cause a trip hazard.