" . . . even if it's for only one second" is probably a figure of speech. I doubt he'd be happy if the guy was only stopped for one second.
Perhaps it's a problem with translation. Japanese may have a phrase that means "hold him as long as possible, but it's okay if it's only a short time" but without being so long; the English equivalent would thus be, "hold him, even if it's for only a second." But I don't know. It just seems that the translation isn't god-tier, since "Although this is bad for you young people's pride, but I have to move on" is an incorrect sentence. (The "but" shouldn't be there.)