You may have no problem in a metre of swell, but as soon as the waves steepen and get choppy, or start to break, it becomes had work or even dangerous. Sometimes landing the boat is the hardest bit because a lazy swell becomes a boiling surf as the water gets shallower. Build up your experience step by step. In such a small boat, you sometimes have to treat each sizeable wave as an individual obstacle with constant adjustments to the speed and heading.
There is no shame in backing down when the conditions are beyond what you're used to, but you can only develop your skills by pushing the envelope a bit at a time.
__________________
|