I'm a shortie, too, and I'm amazed how many people treat me like a 12 year old because I'm the same height as one. I'm 37, FFS!
It does sound as though the site manager was dropping a rather heavy hint. I wonder whether your unappointmented mentor had designs on someone he knew (friend? family?) getting your plot, and has been kicking off about it, and is now trying to make your life tough hoping you'll chuck in the towel and he'll get his way.
As you settle in, and start chatting to others on the allotment, you might find that this chap is nothing more than the (albeit annoying) local comedy act; it would be interesting to see if anyone else mentions him.
With our community garden, we've just allocated the raised beds. The local 'know-it-alls' are well known, and we've allocated them raised beds all together so they drive each other insane...

!! Mind you, we have also made an effort to put friendly, non-patronising gardeners next to complete novices (with their permission, of course!) to hopefully spread their knowledge. Me, I've plumped myself right in the middle of the garden. My excuse was that it would be easier to keep an eye on the kids, truth is that I've got the best view of what everyone else is up too......
