Rode through here last week with my partner on the back of the cargo bike and I had to slow way down to go over the curbs without launching her off the seat... The corridor would be incredible (once complete) if it was actually a comfortable ride.

This goes a bit out of the way as it goes to Pandora, but Haultain is very calm and has almost no through-traffic. Going down Vancouver to Pandora and over the bridge will get you on the trail.
Once the construction on Shelbourne is done, that will definitely be your best option.

These people will always find something to complain about. Today it's bike lanes, tomorrow it'll be speed limits.
Victoria is rapidly becoming 'the' cycling city in Canada precisely because the network is being built so rapidly.