I don't really work remotely but at home for myself for the last year. The biggest challenge for me in a given day is lack of social interaction. It is nice to have a quiet place to work, but hard to deal with (some days) not speaking to a single person all day. Frankly my last job had many downfalls, but I did enjoy some of the people I worked with.
I'm just about to start a 100% remote work job for a company with no real office to work from. I'm renting a full-time desk at a co-working space just so I can have some casual social interaction that doesn't involve me ordering an espresso beverage.