I think one of the biggest problems is that they make it hard to write good libraries. When libraries cannot easily deal with user types that's just not good. Imagine trying to write an ORM for instance.... in Go you are not given many options...you either have to deal with casting via interface{}, making coding both more verbose and lacking in compiler enforced type-safety, or you force the user to implement your interface on all their types.

When a language has a different feature set, it's often helpful to approach the problem from a different direction. Not all common design patterns apply to all programming languages and it may be more work than it should to shoehorn a concept into a language where it can't fit.