There are five main things that affect the movement of the actuator arm, in order of common-ness:
1) Vac hoses - Cost about $20 to replace the whole lot from the vac pump on the intake side of the engine all the way to the turbo via the solenoid. If your hoses can turn on the connections, they've probably hardened. A small leak is enough.
2) Solenoid. You've replaced yours, it's included here because it's a common failure point.
3) Electrical connection to solenoid. Happens a fair bit to those that use pressure-washers in their engine bay. Undo and redo the connector a couple of times.
4) Brake booster. It's also vacuum controlled and a couple of people have reported this as an issue in the past. Both cases also experienced poor (more difficult) braking performance, so if your brakes are good, cross this one off the list
5) Actuator diaphragm. Bloody uncommon and in your case probably not it since the actuator is doing what it's supposed to do most of the time. It's included because it's part of the system.
I'd guess at 3, but wouldn't be surprised if the hose to the actuator had hardened a little and was leaking vacuum on you every now and again.