I once made the mistake of agreeing to setup someones iCloud. Set both their iPhone and iPad to sync.. Then 90% of the entries everywhere disappeared. Crap.
I have issues with contact sync too - my contacts are inconsistently synced if they use iCloud between apple devices & never seems to resolve itself, yet if I put everything in an Exchange (ActiveSync) account it just works.
I've managed to avoid iCloud so far. Apple's CardDAV server seems to do the job pretty well.