I have definitely noticed this as well. There were a lot of people I had to go in and re-follow because I was somehow not following them anymore.