The use of a verb is always good. You can’t ‘of’ anything. A quite common mistake, if you don’t know.
We should have, or should’ve (to use the abbreviation commonly used).
If the other centre halves we have are better than Raggett then I have no problem with him leaving but I think he has been an excellent servant and will always do his best for Pompey. If we had another ten players who showed as much commitment then we probably wouldn’t still be in league 1.