This is an odd shrug of a song, suggesting two men in a bar or party who have both recently been dumped but the one singing has no interest in hearing the other's story, in part because he is too numbly sad to really feel another guy's pain at this time, and anyway his story is much likely the same one. Musically it flows much as the lyrics do, blending sadness and helplessness with the bland knowledge that this happens all the time; Rod Allen's voice is almost pre-rock. It is a smooth song that hints at the coming split of rock and pop, the pop here being cool (but coming off as blase, even cold) while rock was getting hotter and more frenzied all the time. (It was kept off by "Help!" which is the opposite of this song, for instance.)
The real fortune here (if I can put it that way) was for Roger Cook & Roger Greenaway, who wrote this song and thence many others, including one for The Fortunes again - a song I have heard on a dreaded workplace cd compilation* (unlike "You've Got Your Troubles"). The two Rogers went on to fame and renown with songs including "I'd Like To Teach The World To Sing" and The Fortunes still tour with a typically ever-evolving line-up & with songs like these - amiable, modest, simple - they could just go on forever. That their manager was killed in a pirate radio showdown a year later is about the most 'rock' thing about them, but that is 1966, when anything could happen, as opposed to 1965, when everyone is desperately trying to keep their cool.
*The workplace was dreaded, not the cd. There must be a thousand songs that are partially heard with involuntary shudders every day because of this same situation, and it's never the song's fault. Feel free to mention yours, if you wish.