Hi Tonus,
No problem with your plan of using dowels to fill the existing holes but they do need to be an interference fit. You still néed to use an adhesive, but there's no need for great expense, any 2 pack epoxy resin will do the job. Can't say I'd be that keen on little bits of ply glued to the existing ply spacers to support the new brackets, think they will eventually fall off.
I am surprised you can't get the old packers off. I assume you have tried a broad sharp chisel at the point where the ply mates upto the transom. A good hit with a hammer should either get them free or split the ply. If it does split the ply you can then at least chisel/sand the rest off.
No doubt it will mark your transom, but you will be covering it and the existing holes with a new bigger ply packer. It would, IMO, be a much neater and more permanent job.
__________________
|