The conditional perfect, would have, refers to a missed opportunity in the past. In informal speech, it contracts to would’ve, not "would of."
Would have can never be written "would of." However, the latter does exist: when the present conditional would is followed by an expression that begins with of.
The Bottom Line
The erroneous phrase "would of" is probably due to the very similar pronunciation of would’ve. I would’ve told you this sooner, but I needed to save something for the bottom line.